Commit Graph

70240 Commits

Author SHA1 Message Date
Colin Cross
e5dcad264d Merge "Revert "Use depsets for transitive manifests and assets"" into main am: 0a30ce7c4b am: 20d572b050
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/2658795

Change-Id: I471e7a0819043bcc22e1aa16ce33fcd62f404c84
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-07-14 18:02:23 +00:00
Treehugger Robot
294e07f8de Merge "Fix a typo. NFC." into main am: 59276876cd am: 4e0f3a19f3
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/2658057

Change-Id: I43d95e0eca366221da0681d163c08c9b60c49864
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-07-14 18:02:07 +00:00
Colin Cross
20d572b050 Merge "Revert "Use depsets for transitive manifests and assets"" into main am: 0a30ce7c4b
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/2658795

Change-Id: I64ce1247270525779aeabb8471282abaa65eb904
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-07-14 16:13:30 +00:00
Treehugger Robot
4e0f3a19f3 Merge "Fix a typo. NFC." into main am: 59276876cd
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/2658057

Change-Id: I75a9c33ea7ef1aef3133b376ba4737276bc44e37
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-07-14 16:13:16 +00:00
Colin Cross
0a30ce7c4b Merge "Revert "Use depsets for transitive manifests and assets"" into main 2023-07-14 16:10:31 +00:00
Colin Cross
d8c711f884 Revert "Use depsets for transitive manifests and assets"
This reverts commit 4c90f99cb3.

Reason for revert: b/291252863

Change-Id: I646b039a305a1297167e0c56b0bce0f541da260d
2023-07-14 15:52:41 +00:00
Treehugger Robot
59276876cd Merge "Fix a typo. NFC." into main 2023-07-14 15:19:33 +00:00
Jiakai Zhang
b3cf567244 Merge changes from topics "art-boot-image-jars", "configured-jar-location-overrides" into main am: a1140990a1 am: 21abd9bd8e
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/2655305

Change-Id: I244fd36b866a16548bab41f970cbe5a12b4b5d2c
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-07-14 13:56:51 +00:00
Jiakai Zhang
47ee73173c Add a Make variable that overrides configured jar locations. am: 4d90da29a8 am: a701f676d9
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/2655304

Change-Id: I070a444d947d0e028d111abf7077e55acaf1489e
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-07-14 13:56:47 +00:00
Jiakai Zhang
d69968ad43 Refactor dexpreopt for boot jars to make it flexible to config changes. am: cb13b5d1bd am: 0c03078eb1
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/2652081

Change-Id: I9afab30e356694ac1f4cf93a2519c5f5b611c750
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-07-14 13:56:42 +00:00
Jiakai Zhang
21abd9bd8e Merge changes from topics "art-boot-image-jars", "configured-jar-location-overrides" into main am: a1140990a1
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/2655305

Change-Id: Ia823058a5d905666bbeeb1b9ae4bb7ecf15dd7ed
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-07-14 12:26:08 +00:00
Jiakai Zhang
a701f676d9 Add a Make variable that overrides configured jar locations. am: 4d90da29a8
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/2655304

Change-Id: If54785cc69a8fd40a7022342c0a726a0212a0686
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-07-14 12:26:06 +00:00
Jiakai Zhang
0c03078eb1 Refactor dexpreopt for boot jars to make it flexible to config changes. am: cb13b5d1bd
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/2652081

Change-Id: I08692cbb4fa9bb813b3b8bfef98999567f87da9c
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-07-14 12:26:03 +00:00
Jiakai Zhang
a1140990a1 Merge changes from topics "art-boot-image-jars", "configured-jar-location-overrides" into main
* changes:
  Add a Make variable to determine ART boot image jars for testing.
  Add a Make variable that overrides configured jar locations.
  Refactor dexpreopt for boot jars to make it flexible to config changes.
2023-07-14 11:47:59 +00:00
Jiakai Zhang
4d90da29a8 Add a Make variable that overrides configured jar locations.
We had hardcoded logic to override "framework-minus-apex", but we need
to override more jars, so this CL adds a general solution for this.

Bug: 290583827
Test: m nothing
Change-Id: I211ebda7aa2803886ab6e6d081d26327665e49a6
2023-07-14 12:13:30 +01:00
Jiakai Zhang
556bdf8e96 Add a Make variable to determine ART boot image jars for testing.
Bug: 290583827
Test: m nothing
Change-Id: I6eb0c83dd14682905ebaf3af55171856300eb2e3
2023-07-14 12:13:30 +01:00
Jiakai Zhang
cb13b5d1bd Refactor dexpreopt for boot jars to make it flexible to config changes.
In the past, dexpreopt for boot jars was very inflexible, and it was
incredibly hard to make a change that is as simple as adding a jar to a
boot image. Boot image generation was handled by
"platform_bootclasspath" and "bootclasspath_fragment" separately. This
caused not only code duplication but also the inflexiblity as such a
design did not fit today's use cases, where a boot image may take jars
from multiple mainline modules and the platform, and a mainline module
can contribute to multiple boot images. The design casued a huge
maintenance burden as any change to the boot image cost multi-week
efforts.

In recent years, efforts have been made to improve this a bit by a bit.
This change is another step towards making dexpreopt reasonable.

After this change, all boot images are generated by "dex_bootjars",
which is in build/soong and is therefore available on both the full
source tree and the thin manifest (master-art). The change decouples
profile generation/extraction from boot image generation. Profiles for
mainline modules are still handled by "bootclasspath_fragment"
because they need to be packed into APEXes when building mainline
modules and extracted from APEXes whem building the system image from
prebuilt modules. Boot images are not handled by
"bootclasspath_fragment" anymore.

Bug: 290583827
Test: m (all existing tests are still passing)
Test: Manually checked that the boot images are exactly the same as
  before.
Change-Id: Ib5a5f401bee334ffcab5c26618e0c8888b84575a
2023-07-14 12:13:28 +01:00
Yi Kong
8f9ca23c1e Fix a typo. NFC.
Test upload from Cog...

Change-Id: I68964d14d0a01be2bdc0ad276bdfde9f879795f2
Test: N/A
2023-07-14 06:18:38 +00:00
Treehugger Robot
86c53a60e3 Merge "Add flag to skip writing output from soong_zip." into main am: c38523cd33 am: 90ff013f58
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/2657395

Change-Id: I97e8c1917c1f5897d4646c40092f5e7afea459ed
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-07-14 01:24:01 +00:00
Treehugger Robot
031d1ea87f Merge "Add unconverted_reasons_from_heuristics fields to the Module message" into main am: 4d9dbdab7a am: c41e27c39a
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/2642132

Change-Id: Id11a599735a0991fc3915203fad899806e889770
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-07-14 01:23:25 +00:00
Treehugger Robot
90ff013f58 Merge "Add flag to skip writing output from soong_zip." into main am: c38523cd33
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/2657395

Change-Id: If3153bba13125d40f395824d740b6ccb51587a30
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-07-13 23:34:11 +00:00
Wei Li
d4fddcc224 Some new libraries are added to system partition and are not captured in am: 9a8df6669b am: b3c2ccd466 am: dfd2b44d76
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/2657275

Change-Id: Ia334fe842e0831e9e9e56f08ac8f7c20b78d9c25
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-07-13 23:34:04 +00:00
Treehugger Robot
c41e27c39a Merge "Add unconverted_reasons_from_heuristics fields to the Module message" into main am: 4d9dbdab7a
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/2642132

Change-Id: I0575994181b740c0bbd57a6e9720c81a36de2845
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-07-13 23:33:14 +00:00
Treehugger Robot
c38523cd33 Merge "Add flag to skip writing output from soong_zip." into main 2023-07-13 23:32:26 +00:00
Wei Li
dfd2b44d76 Some new libraries are added to system partition and are not captured in am: 9a8df6669b am: b3c2ccd466
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/2657275

Change-Id: I5f7b91dccecc7f1e990a2f838ef0aa6ed21da9d8
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-07-13 22:49:57 +00:00
Treehugger Robot
4d9dbdab7a Merge "Add unconverted_reasons_from_heuristics fields to the Module message" into main 2023-07-13 22:45:17 +00:00
Wei Li
b3c2ccd466 Some new libraries are added to system partition and are not captured in am: 9a8df6669b
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/2657275

Change-Id: I48fe79981771442c243367e4f78e95626347edb6
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-07-13 21:59:19 +00:00
Spandan Das
dd4d7a25f8 Merge changes from topic "go_bp2build" into main am: 06f3c9bb42 am: c16d84cd70 am: 0541f6d7ba
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/2655342

Change-Id: Ibc595b732be5735aab8e41fad531bef3cfe80bd8
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-07-13 19:04:10 +00:00
Dan Albert
1cec6d5000 Add flag to skip writing output from soong_zip.
We want to measure how much of build time is spent on writing large
zips to disk. We can do this by doing the rest of the work but
skipping the write itself.

Some local and trivial testing shows this is probably a lot of the
cost of these rules:

    $ python -m timeit -s "import os" -r 10 "os.system('soong_zip -write_if_changed -o foo.zip -D prebuilts/sdk')"
    1 loop, best of 10: 8.37 sec per loop
    $ python -m timeit -s "import os" -r 10 "os.system('soong_zip -o foo.zip -D prebuilts/sdk')"
    1 loop, best of 10: 5.39 sec per loop
    $ python -m timeit -s "import os" -r 10 "os.system('soong_zip -n -o foo.zip -D prebuilts/sdk')"
    1 loop, best of 10: 3.66 sec per loop

Bug: None
Test: treehugger
Change-Id: I43acdb08d0e00efaba9e5d7add972a7ec41646d2
2023-07-13 18:45:32 +00:00
Spandan Das
0541f6d7ba Merge changes from topic "go_bp2build" into main am: 06f3c9bb42 am: c16d84cd70
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/2655342

Change-Id: I3e36863026e962bd94b16cd2c4777583420187ce
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-07-13 17:56:35 +00:00
Wei Li
9a8df6669b Some new libraries are added to system partition and are not captured in
product SBOM of cuttlefish devices, ignore these diffs for now in the
SBOM integration test to fix the CI breakage.

Bug: 291073898
Test: build/soong/tests/sbom_test.sh
Change-Id: Idf5569f67642784ee422847301c59dddc8ff0907
2023-07-13 10:29:55 -07:00
Spandan Das
c16d84cd70 Merge changes from topic "go_bp2build" into main am: 06f3c9bb42
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/2655342

Change-Id: I19a5b10c616e10a370b3df1e2b876b00348cb535
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-07-13 17:12:36 +00:00
Spandan Das
06f3c9bb42 Merge changes from topic "go_bp2build" into main
* changes:
  Respect package boundaries in bp2build conversion of go modules
  Create a temporary denylist for go binaries used in mixed builds
  Partial bp2build conversion of blueprint_go_binary
  Partial bp2build conversion of bootstratp_go_package
2023-07-13 16:16:00 +00:00
Igor Foox
61efedb0cf Merge "Remove openwrt modules from genrule denylist." into main am: ae2e23a4ba am: 502ea558ce am: 9fbad45be7
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/2651203

Change-Id: Icbd7fa177a81e0efaa68e744b46c58d5c1da4055
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-07-13 16:05:13 +00:00
Igor Foox
9fbad45be7 Merge "Remove openwrt modules from genrule denylist." into main am: ae2e23a4ba am: 502ea558ce
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/2651203

Change-Id: Ifff1dca4d02d10ece70324ba7e94fe2c7abbcd91
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-07-13 15:20:35 +00:00
Igor Foox
502ea558ce Merge "Remove openwrt modules from genrule denylist." into main am: ae2e23a4ba
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/2651203

Change-Id: I5769f4d7fc4e7c1f488ebf181a733f6de25541d4
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-07-13 14:17:18 +00:00
Ayush Budhwani
62efa1d650 Add unconverted_reasons_from_heuristics fields to the Module message
A heuristics is implemented in the build/bazel/scripts/bp2build_progress/bp2build_progress.py script to determine the reason for why a module is not converted.

Test: b build //build/bazel/scripts/bp2build_progress:bp2build_progress

Change-Id: I9d4eaf309dbf26bbb6de18e1af0d9cdc8fe09e94
2023-07-13 13:31:09 +00:00
Igor Foox
ae2e23a4ba Merge "Remove openwrt modules from genrule denylist." into main 2023-07-13 13:29:13 +00:00
Wei Li
2181addcde [automerger skipped] Add integration test for m build SBOM of APEXs/APKs. am: b68bfc65d8 -s ours
am skip reason: Merged-In Id8002042f607f8e158f561ca47e0a84a8b8927e2 with SHA-1 59df0ee442 is already in history

Original change: https://googleplex-android-review.googlesource.com/c/platform/build/soong/+/24029573

Change-Id: I5c6c22f2f65f5ee4b1425319b5025233bfa931a8
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-07-13 00:33:55 +00:00
Wei Li
8cfaa2fdac [automerger skipped] Cleanup SBOM test script for later adding tests of generating SBOM of am: 1b2badc435 -s ours
am skip reason: Merged-In I2bb3e8405dad716837d3004d4c6fd54ae80527c1 with SHA-1 cd9948f66e is already in history

Original change: https://googleplex-android-review.googlesource.com/c/platform/build/soong/+/24029572

Change-Id: I91bd3494b95e7d1abf92138977ddbf14d6c683ac
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-07-13 00:33:52 +00:00
Wei Li
8b1d67cd94 [automerger skipped] Exclude diffs caught by sbom_test. am: 92275ff913 -s ours
am skip reason: Merged-In I152c2e2c55b6547b2dddf7b1b2524681c9af952e with SHA-1 704146094a is already in history

Original change: https://googleplex-android-review.googlesource.com/c/platform/build/soong/+/24029571

Change-Id: I82a04848a43946256350f29c2bf915c491552ca8
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-07-13 00:33:48 +00:00
Spandan Das
0a8a27500e Respect package boundaries in bp2build conversion of go modules
bp2build's codegen context does not implement
BazelPathConversionContext. To reuse the utility function
transformPackagePaths, update its signature

(Also make deps of go_library unique to make the conversion resilient)

Test: go test ./bp2build
Change-Id: I126b1057d2b26bc6c7d3be2780f1b62d28323cf0
2023-07-13 00:22:26 +00:00
Spandan Das
69afa98fbd Create a temporary denylist for go binaries used in mixed builds
This allows us to rollout building _some_ go targets using rules_go
without affecting mixed builds.

Test: Presubmit
Bug: 284483729
Change-Id: I0ccb4c9b90614369147a380f44f7ae372ef9396e
2023-07-13 00:22:26 +00:00
Colin Cross
5bf41f6f96 Merge "Use depsets for transitive manifests and assets" into main am: 654f176379 am: b54cec3cb9 am: 44891a09ac
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/2633850

Change-Id: I4434825dbf0a6e12669c6c1620f77ceaad876e47
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-07-13 00:13:11 +00:00
Vadim Spivak
c38550c945 Merge "AIDEGen: Collect apt generated sources" into main am: 7a2e48bd36 am: 0f9737719e am: 8303ac5f5f
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/2620724

Change-Id: I0c3fc967680ca6d7f3876ba20ef5dc62e91e2fc2
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-07-13 00:11:17 +00:00
Colin Cross
44891a09ac Merge "Use depsets for transitive manifests and assets" into main am: 654f176379 am: b54cec3cb9
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/2633850

Change-Id: I4ffe5422da8d1eadc4bcb99223850e170e1cdb11
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-07-12 23:29:23 +00:00
Vadim Spivak
8303ac5f5f Merge "AIDEGen: Collect apt generated sources" into main am: 7a2e48bd36 am: 0f9737719e
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/2620724

Change-Id: I79780f60f5ad97864476b49d5054a0cd58a505bf
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-07-12 23:28:00 +00:00
Colin Cross
b54cec3cb9 Merge "Use depsets for transitive manifests and assets" into main am: 654f176379
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/2633850

Change-Id: I7b16496a65e5177edc599f6070eb1ca414087567
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-07-12 22:42:02 +00:00
Vadim Spivak
0f9737719e Merge "AIDEGen: Collect apt generated sources" into main am: 7a2e48bd36
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/2620724

Change-Id: I37daac50c8a192f49168b6b83aeddfb7e26d1404
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-07-12 22:39:58 +00:00
Colin Cross
654f176379 Merge "Use depsets for transitive manifests and assets" into main 2023-07-12 21:49:13 +00:00