Commit Graph

685 Commits

Author SHA1 Message Date
Vinh Tran
83f88cdc8b Remove creation of has_stubs in cc_library_shared by bp2build
Because when stubs_symbol_file is defined on cc_library_shared, has_stubs is already implied to be true.

Bug: 260771171
Test: go test
Change-Id: I3b9a2cfb9000411d1b5f22da87dc339267775799
2022-12-19 13:56:11 -05:00
Cole Faust
abc182cc94 Merge "Allow exporting bazel mixed build allowlists to simple text files" 2022-12-19 03:09:11 +00:00
Kevin Dagostino
32edd1a608 Add tradefed attributes to cc_test bazel macro
The BUILD file will now have the needed attributes to run the tradefed
test rules.

Test: b test //packages/modules/adb:adbd_test --platforms=//build/bazel/platforms:android_target
Change-Id: I5cb84ccdde27a7fadbef479269df594344a29187
2022-12-18 04:38:16 +00:00
Yu Liu
f01a0f0ca9 Enable code coverage for cc shared library
Bug: None
Test: Manual and unit tests
Change-Id: I5064e4407d3451ff172ab203256f3c073cc0ad1b
2022-12-16 14:16:22 -08:00
Cole Faust
705968d043 Allow exporting bazel mixed build allowlists to simple text files
So that they the allowlists can be read from build_mainline_modules_bundles.sh.

Bug: 262302253
Test: m out/soong/bazel_prod_mixed_builds_enabled_list.txt out/soong/bazel_staging_mixed_builds_enabled_list.txt && cat out/soong/bazel_prod_mixed_builds_enabled_list.txt && echo && cat out/soong/bazel_staging_mixed_builds_enabled_list.txt
Change-Id: Id5f89ebb651a52dc5368681a20d6183703e05bd4
2022-12-15 15:40:36 -08:00
Alix
266bbf2f9c Libs support for * -> java_library_edge with kt sources
Bug: 244210934
Test: bo2build tests and built kotlinx_atomicfu
Change-Id: Id6eac2f104878e4d7902a32e846ef1cc87dd7863
2022-12-15 06:27:59 +00:00
Treehugger Robot
6441a125cf Merge changes from topics "bp2build_cc_object_dynamic_deps", "bp2build_cc_prebuilt_object"
* changes:
  Add support for headers from dependencies to bazel cc_object
  Add bp2build support for cc_prebuilt_object
2022-12-13 16:25:58 +00:00
Vinh Tran
6916999cd0 Merge changes from topic "afdo-in-cc-rules"
* changes:
  Change out path to android_target-opt in bazel integration tests
  Add check for handcrafted BUILD file in bp2build
  Add bp2build conversion to reference to fdo_profile targets
2022-12-13 00:12:03 +00:00
Treehugger Robot
331360e7d6 Merge "Add support for variant_prepend properties in bp2build for cc library" 2022-12-12 22:46:33 +00:00
Vinh Tran
bc9c8b494c Add check for handcrafted BUILD file in bp2build
ag/20469925 added a handcrafted BUILD file (allowlisted in ag/20597986)
to internal master. ag/c/platform/build/soong/+/20583593/12/cc/bp2build.go#818 generates references to the targets in the BUILD file. However in tm-dev, the BUILD file doesn't exist while the .afdo files do.

One solution is to cherry-pick the BUILD file to tm-dev. However, tm-dev doesn't have vendor/google/build/soong/bp2build_allowlist.go to check in the BUILD file in bp2build.

This CL adds a check that the BUILD file exists to avoid the failure as in https://android-build.googleplex.com/builds/pending/P45724500/aosp_cf_x86_64_phone-userdebug/latest/view/logs/build_error.log in tm-qpr-dev-plus-aosp branch.

Bug: 253540178
Test: go test
Change-Id: I47fb853015ca230afe3cefe1d37728bf714624be
Merged-In: I47fb853015ca230afe3cefe1d37728bf714624be
2022-12-12 12:17:49 -05:00
Vinh Tran
99270ea3c3 Add bp2build conversion to reference to fdo_profile targets
The fdo_profile targets are handcrafted in ag/20469925. Bp2build creates fdo_profile attribute on cc_library_shared and references to the fdo_profile target if the profile exists. This works under assumption that all afdo profiles have an associated Bazel fdo_profile target declared in the same folder/package.

As noted in b/253540178, this won't work when we check in the BUILD files (which is likely 6+ months away) because some profiles might only exist internally but not in AOSP. We will implement a long-term solution once we figure out all the requirements we need to support afdo both in Soong and Bazel.

Bug: 253540178
Test: go tests
Change-Id: Iebd5dd7a76583b4b2bf0d9ee56d58f247d313a54
Merged-In: Iebd5dd7a76583b4b2bf0d9ee56d58f247d313a54
2022-12-12 12:16:43 -05:00
Zi Wang
1cb11800f7 Add support for variant_prepend properties in bp2build for cc library
Some properties in soong are marked "variant_prepend", which means
in bp2build output, variant properties(select ...) should come before
general properties.
Prepend property is added in StringListAttribute
stuct so that bp2build can handlle variant_prepend properly, if
Prepend is properly set in different module logics.
This change also coordinates cc library with Prepend.
Coordination with other module types will be in separate CLs.

Bug: 261644490
Test: TestCcLibraryHeadersSimple,
TestCcLibraryHeadersArchAndTargetExportSystemIncludes and TH

Change-Id: I3b61c491c54bbe91ba9aa2af9b4c84193c91ae7f
2022-12-09 16:41:23 -08:00
Colin Cross
65ebc429e0 Add support for headers from dependencies to bazel cc_object
The libc_musl_crt* cc_object modules use header_libs to add
headers to the search path.  Propagate static_libs, shared_libs
and header_libs to includes_deps.

Bug: 259266326
Test: TestCcObjectHeaderLib
Change-Id: I8db4d6886761426d3ece38c43ac868d3248f7a9f
2022-12-09 13:58:58 -08:00
Colin Cross
c5075e917e Add bp2build support for cc_prebuilt_object
Add bp2build support for cc_prebuilt_object to allow buiiding
musl crt objects that use prebuilt clang_rt.crtbegin and
clang_rt.crtend.

Bug: 259266326
Test: //extenral/musl:libc_musl builds
Change-Id: Ic2b0375bc6e7336d31c9e2cf51bdc2ea894185aa
2022-12-09 13:58:58 -08:00
Trevor Radcliffe
682bfb2923 Revert "bp2build conversion for Sanitize.Integer_overflow"
This reverts commit f82482db31.

Reason for revert: http://b/261935841

Change-Id: I8155c2b17c6052b734efea43be06b1099ffedd32
2022-12-09 20:39:12 +00:00
Trevor Radcliffe
f82482db31 bp2build conversion for Sanitize.Integer_overflow
and Sanitize.Misc_undefined.

This is a redo of aosp/2277186 with changes to account for issues
in mixed builds. Specifically, for now we're disabling mixed builds
for modules that use ubsan. This bug tracks enabling it:
b/261058727.

Bug: 253428057
Test: Unit tests
Change-Id: I9dbec1d99353f8cbe6751d5241283ab19592dc27
2022-12-06 20:13:21 +00:00
Paul Duffin
4c0765a669 Support running bp2build tests in fixtures
Previously, the fixture test infrastructure was hard coded to
initialize itself in preparation for invoking the PrepareBuildActions()
method. That meant it could not be used for testing the bp2build and
apiBp2build modes which required different initialization and called a
different method.

This change extracts that behavior into a FixtureTestRunner and adds an
implementation that allows it to test the above two modes. It then uses
that to implement the runBp2BuildTestCaseWithSetup method.

The TestPrebuiltLibraryAdditionalAttrs was the only test which broke as
it supplied an invalid path to the MockFS (it does not support using a
trailing / to represent an empty build directory). So, it was modified
to add a couple of files inside those directories instead.

Test: m nothing
Change-Id: I6798a4f761160af8d1bfed81d46de9628bda3eb9
2022-12-02 11:14:17 +00:00
Jingwen Chen
c4c34e1eab Convert apex_available (for supported modules) to bazel tags.
The tags will then be read by an aspect applied from apex attributes to
validate that the deps are explicitly opt-in to the appropriate apex.

The semantics of this is the same as the Soong apex_available property.

This also adds the "base_apex_name" attr to override_apex conversion so
that apex_available checks continue to work on the non-Google and non-Go
apex names.

Test: presubmits
Fixes: 218841706
Change-Id: I19b3637da9aa47573b252d74f62b601fbdc3784d
2022-12-01 06:25:25 +00:00
Yu Liu
734fefdf90 Merge "Support abi check in bazel." 2022-11-30 17:57:42 +00:00
Alix
341484b3da Bp2Build for libs property in java_library
Bp2build for both *->java_library and java_library->* edges.

Change-Id: I2540c9af2ae2fe7677767d507647b1c6669b9bb7
Bug: 244210934
Test: ./bp2build testcase, manually inspected build files for hamcrest-library, hamcrest targets
2022-11-29 19:44:50 +00:00
Yu Liu
56ccb1aecd Support abi check in bazel.
Bug: 253498204
Test: Manual and unit test.
Change-Id: Id23c4f772f67c4ba05704eaad77322133474b42b
2022-11-29 11:25:10 -08:00
Jingwen Chen
ed6240d1bc Merge "Revert "bp2build conversion for Sanitize.Integer_overflow"" 2022-11-29 02:35:07 +00:00
Jingwen Chen
0ead79bd11 Revert "bp2build conversion for Sanitize.Integer_overflow"
Revert submission 2277186-ubsan_bp2build

Reason for revert: b/260630869 broke mixed_droid
Reverted Changes:
Idd9b33e53:bp2build conversion for Sanitize.Integer_overflow
I0d636e439:Add Sanitizer libs up the dependency tree
If113de7bc:Introduce toolchain features for sanitizers

Change-Id: Idf7e7b62f6fb6d4ff92341cc4c9c4b7426b70951
2022-11-28 22:44:35 +00:00
Trevor Radcliffe
202f2afbf8 Merge "bp2build conversion for Sanitize.Integer_overflow" 2022-11-28 21:45:07 +00:00
Trevor Radcliffe
548fc5dcec bp2build conversion for Sanitize.Integer_overflow
and Sanitize.Misc_undefined.

Bug: 253428057
Test: Unit tests
Change-Id: Idd9b33e537d1b1c80b0c96a9590327dae13fada7
2022-11-28 19:31:34 +00:00
Liz Kammer
ffc17e4edb Use excludes from OS axis for OsAndInApex
The axis for OS and in APEX was previously ignoring the excludes from
OS, which would result in excludes for non-Android OSes being
incorrectly ignored.

Test: go test Soong tests
Test: b build //hardware/libhardware:all
Bug: 260131489
Change-Id: Ie77f763bea0f473ac69a0c6b6bc3216e7359ad12
2022-11-23 10:01:30 -05:00
Sam Delmerico
775f2cb3cd Merge "export neverallow include dir list to Bazel" 2022-11-16 20:52:17 +00:00
Sam Delmerico
46d08b4e39 export neverallow include dir list to Bazel
Bazel needs to recreate the neverallow behavior for absolute includes,
so export the list of include dirs to soong_injection.

Bug: 250931909
Test: aosp/2302925
Change-Id: I3bd345ea4cfb3181f07e4570466f2fc8c5fe32da
2022-11-15 17:33:21 -05:00
Alix
8062f4dee7 bp2build for java_library with .kt srcs or common_srcs
java_library modules with .kt srcs or common_srcs will be converted into
module type kt_jvm_library

Test: m bp2build, manually inspected build files for allowlisted modules
Bug: 258688914
Change-Id: I8293a11c8247b4b76358d0991f82c6b61b58adc3
2022-11-15 20:01:29 +00:00
Sam Delmerico
1bde1cb668 Merge changes from topics "bp2build-tidy-disabled-srcs", "bp2build-tidy-timeout-srcs"
* changes:
  bp2build for tidy_timeout_srcs
  bp2build for tidy_disabled_srcs
2022-11-10 16:27:24 +00:00
Sam Delmerico
4c902d696a bp2build for tidy_timeout_srcs
Bug: 195029134
Test: m bp2build
Change-Id: I59fdc16d8c224f42167e3b5a3857f4ff2f546825
2022-11-08 17:18:52 -05:00
Sam Delmerico
c9b8fbdda0 bp2build for tidy_disabled_srcs
Bug: 195029134
Test: m bp2build
Change-Id: I9591439213dbf0ef68cd33151b3e32f6f6c68551
2022-11-08 17:17:52 -05:00
Paul Duffin
c639059fd8 Make CodegenContext reference *android.Context
Previously, it stored an android.Context (no pointer) which while it
worked (because the current contents are themselves pointers) it is
inconsistent with how the rest of the code references it and could
cause weird behavior if additional fields were added to the Context.

Test: m nothing
      m bp2build
Change-Id: I9c70f925dde85e5b15431cc232b0c2eb6371ec6d
2022-11-05 17:43:12 +00:00
Lukács T. Berki
72ab470a29 Merge "Multithread symlink forest removal." 2022-11-02 07:45:22 +00:00
Usta (Tsering) Shrestha
3bb3763afd Merge "include symlink metrics in bp2build_metrics.pb" 2022-10-31 20:00:36 +00:00
Lukacs T. Berki
bc5f731791 Multithread symlink forest removal.
This is a second attempt at aosp/2273288 which got rolled back because
it turned out that:

1. We make ~120K symlinks in AOSP (!), all of which need to be deleted
2. System calls are sometimes slow
3. Golang spawns a new OS-level thread for each blocking system calls to
   keep cores busy

All this together means that we sometimes had 10K system calls in
flight, which meant 10K OS-level threads, which is when Go gives up and
says "I created too many threads, please help".

The fix is to move the system calls into a pool of goroutines, which
soon end up on a pool of threads (since they mostly do blocking system
calls)

Test: Presubmits.
Change-Id: Ia9aefff3b0ed373f09bb6c8b2ec1d8b0f00b213b
2022-10-31 16:04:13 +00:00
usta
4f5d2c1e97 include symlink metrics in bp2build_metrics.pb
Bug: b/256212479
Test: Prior to thi CL bp2build.symlink_forest event was missing in bp2build_metrics.pb after a clean mixed build
Change-Id: I53bfc4114a383c0d1f9c4c7945e7d4c69bc50b0c
2022-10-31 11:38:10 -04:00
Alix
b4e09a0ada Bp2build Java libs for java_binary -> java_import edge
Since Bazel's java_import requires a jars attribute to be specified,
the generated neverlink-duplicated module is of type java_library

Change-Id: I14a866dfc583507a9462add50d95060cbfe540c5
Bug: 244210934
Test: m bp2build, go test ./bp2build, manual inspection of generated Build and jar files
2022-10-28 20:26:38 +00:00
Christopher Parsons
ed2873aea0 Revert "Multithread symlink forest removal."
This reverts commit 6b236f1607.

Reason for revert: Breaks ab/aosp-master-bazel incremental builds. Details on b/254338319

Change-Id: I37eeeda50cff0475d91e7926fdf74216975a0037
2022-10-28 15:43:48 +00:00
Lukacs T. Berki
6b236f1607 Multithread symlink forest removal.
This makes symlink forest creation ca. 2x faster again, taking 2-3
seconds instead of 5.

Who would have thought that os.RemoveAll() is slow.

Test: Presubmits.
Change-Id: I91e41319c972dbf1113cf723e383c785433c18b9
2022-10-28 07:15:55 +00:00
Lukács T. Berki
4bb30b59a9 Merge changes from topic "separate-symlink-forest-invocation"
* changes:
  Build the symlink tree on multiple threads.
  Create Bazel symlink forest in a separate process.
2022-10-28 07:05:00 +00:00
Lukacs T. Berki
647e7abfa2 Build the symlink tree on multiple threads.
This makes it take ~5 seconds on AOSP instead of ~10. Frankly, the
speedup is somewhat disappointing but at least the code is not
complicated.

Test: Presubmits.
Change-Id: Icf94d7ca8bd80c458d014f4cf4cc1be7138deaa6
2022-10-27 10:17:20 +00:00
Lukacs T. Berki
c541cd27fa Create Bazel symlink forest in a separate process.
This helps with incrementality a lot: the symlink forest must depend on
almost every directory in the source tree so that if a new file is added
or removed from *anywhere*, it is regenerated.

Previously, we couldn't do this without invoking bp2build, which is
quite wasteful because bp2build takes way more time than the symlink
forest creation, even though we do the latter in a very suboptimal way
at the moment.

This means that if a source file is added or removed (which does not
affect globs), we don't pay the cost of bp2build anymore.

Also refactored symlink_forest.go on the side. Too much state was being
passed around in arguments.

This change reimplements aosp/2263423 ; the semantics of not touching an
output file is the exact same as order-only inputs and the latter is a
bit fewer lines of code.

Test: Presubmits.
Change-Id: I565c580df8a01bacf175d56747c3f50743d4a4d4
2022-10-27 08:08:45 +00:00
Yu Liu
2ff53da9f8 Merge "Support apex_test." 2022-10-26 16:37:59 +00:00
Usta Shrestha
783ec5c72c Remove a self-dependency of bp2build
bp2build (i.e. bootstrap.ninja#bp2build_workspace_marker) generate $OUT/soong/bp2build/**/BUILD.bazel files
Having them as a dependency would thus make bp2build_workspace_marker stale upon incremental builds (because we don't re-touch the marker file if it already exists)

See Also: https://android-review.googlesource.com/c/platform/build/soong/+/2263423

Bug: b/253450880
Test: NINJA_ARGS='-d explain' m --bazel-mode adbd
      repeat and see if bp2build is rerun
prior to this CL, one would see the input /usr/local/google/home/usta/aosp/out/soong/bp2build/build/make/tools/BUILD.bazel trigger bp2build over and over again

Change-Id: I904cd333a5d6ef506fc4019eda7623ef96a1daa3
2022-10-25 13:46:20 +00:00
Yu Liu
4c212ce058 Support apex_test.
Bug: 249744489
Test: Manual and unit test.
Change-Id: I866debf7ffee54a0f4649b5e5425a8e3cc3042a1
2022-10-24 10:35:12 -07:00
Usta Shrestha
49d04e89d8 cosmetics
Test: m --bazel-mode nothing
Bug: b/239044236
Change-Id: Iaffc315c696f2fa19a2525009010d5964cf5a7d1
2022-10-21 20:09:23 +00:00
Treehugger Robot
95ac23eeeb Merge "Introduce BazelStringOrLabelFromProp." 2022-10-20 18:26:20 +00:00
Jingwen Chen
6817bbb3c8 Introduce BazelStringOrLabelFromProp.
Soong supports string properties, but they are overloaded, and can mean
one of three things:

* path reference
* module reference
* string literal

Bazel has different types: label and string attributes. Thus there needs
to be a way to categorize them correctly in bp2build.

This CL introduces a new function to be used on properties like
apex_key.private_key / apex_key.public_key, as well as
android_app.certificate / apex.certificate.

It is important to disambiguate the prop betenn a string literal
attribute or file/rule target label attribute, so this functions does
just that.  The new attributes are then further handled by their
respective macros (apex_key, android_binary, apex).

Bug: 253557437
Fixes: 253557437
Test: presubmits, new tests

Change-Id: Id8111cdd60d3aabcae7d17fe9da84d0ee3966023
2022-10-20 14:57:37 +00:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
c5184ec089 Rename prebuilt_library_* to cc_prebuilt_library_*
Follow the convention of everything else in `cc`

Test: CI
Change-Id: I133d0b7119d4e9b357aada3467bc079f20dc0f60
2022-10-20 14:51:55 +00:00