Commit Graph

822 Commits

Author SHA1 Message Date
Colin Cross
04d9c96afe Merge "Remove global state from apex modules" am: 43159bd1b7 am: 21bb1d491c am: 2c76247f51 am: 393b394b85
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1428815

Change-Id: I264afeceec4bdbc344ba812b1c5fd88fed043214
2020-10-07 20:02:47 +00:00
Colin Cross
56a8321c21 Remove global state from apex modules
A global variant was used to store the global mapping between
modules and APEXes.  Replace it with storing pointers to APEX
contents inside each module so that they can query the contents
of any APEXes they belong to.

Bug: 146393795
Test: all Soong tests
Test: single line change to build.ninja host install dependency ordering
Test: no Android-${TARGET_PRODUCT}.mk, make_vars-${TARGET_PRODUCT}.mk or late-${TARGET_PRODUCT}.mk
Change-Id: Id2d7b73ea27f8c3b41d30820bdd86b65c539bfa4
2020-10-06 13:39:57 -07:00
Jooyung Han
276f33f175 Merge "apex: emit file_contexts for flattened apex" am: 51ce6b0acc am: 605b7a6bde am: 0d7b03fc88 am: 2c4999b7f7
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1435292

Change-Id: Ic61a5e6645c34c63f5b67cc15811ed2164dd6717
2020-10-06 04:37:48 +00:00
Jooyung Han
51ce6b0acc Merge "apex: emit file_contexts for flattened apex" 2020-10-06 03:26:14 +00:00
Treehugger Robot
0ef34709fc Merge "Remove stale apex_available dependency." am: 0400ede359 am: e74f6092de am: 92071533db am: b10e28c727 am: c467935850
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1435497

Change-Id: I10d6d40e8c360a31e2d6cd4a333829ec9e6a360c
2020-10-05 01:10:38 +00:00
Martin Stjernholm
3b8ca4c317 Remove stale apex_available dependency.
Test: m
Bug: 142944931
Change-Id: I9c69ef64eb9eecd45f6272a1d79bf77ee910815c
2020-09-30 17:40:17 +01:00
Treehugger Robot
579b2388da Merge "Don't include artifacts for host-cross targets" am: 6c76f18aae am: ec2752052e am: 33649cab1a am: f8602ac7f7 am: d0e7b28a6a
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1440496

Change-Id: If75c580964e9abb50606b37e68c5197045080bb6
2020-09-29 15:51:27 +00:00
Jiyong Park
ccb406fc9c Don't include artifacts for host-cross targets
When building an APEX, the build system included artifacts from all
targets that are returned from ctx.MultiTargets(). This however has
became a problem as we add new host targets like linux_bionic_arm64.
When there are multiple host targets having different OSes,
ctx.MultiTargets() returns one target per a host OS. For example, when
linux_bionic_arm64 is enabled, ctx.MultiTargets() returns the following
two targets because linux_bionic != linux_glibc.

* linux_glibc_86_64
* linux_bionic_arm64

Some dependencies (like soong_zip, etc.) are not enabled for the new
host target (because they don't have to) and therefore the build fails.
Since the purpose of host APEX is to package some host tools for
testing, etc., we actually don't need the second target for APEX.

Fixing the problem by not skipping "cross host" targets which can't be
natively executed on the host machine.

Bug: 169454252
Test: HOST_CROSS_OS=linux_bionic HOST_CROSS_ARCH=arm64 m
Test: DIST_DIR=$(pwd)/out/dist \
        ./art/tools/dist_linux_bionic.sh com.android.support.apexer
Change-Id: Idaded56e4b4fc0195c8bbc760e4b4728016bd5b4
2020-09-29 11:22:23 +09:00
Dan Albert
063452b1c1 Replace FutureApiLevel with an ApiLevel. am: 0b176c8038 am: d5e15a13df am: 818d1b4e6e am: d997f3cd33 am: 07f0cae3c7
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1388351

Change-Id: Ife2f7c6a1d6dc9828c07ef6341b774e23d40fe3e
2020-09-25 06:00:59 +00:00
Dan Albert
d2a7987456 Replace ApiStrToNum uses with ApiLevel. am: c8060536e8 am: 6e3d089818 am: c81c409966 am: 97825a1e23 am: 049b18bb14
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1371896

Change-Id: Id26f8d0f0323da9fbf924fcb42dda23d326e2909
2020-09-25 06:00:55 +00:00
Jooyung Han
7f146c0da1 apex: emit file_contexts for flattened apex
Flattened APEXes are installed in /<partition>/apex/<name>, so its
file_contexts file should be amended accordingly.
For example, file_contexts of an apex "com.android.foo" with should be
modified by prepending actual install path to the path regex.

/bin/foo <sepolicy label>
=> /system/apex/com\.android\.foo/bin/foo <sepolicy label>

File_contexts file is emitted to "make" via LOCAL_FILE_CONTEXTS variable
for apex_manifest.pb module.

Make collects these files and use them to build file_contexts.bin.

Also removed global state which is used to keep the list of all
flattened apexes.

Bug: 166518492
Test: m file_contexts.bin
      check intermediate file_contexts.modules.tmp file
      in $OUT/obj/ETC/file_contexts.bin_intermediates/
Change-Id: I780370b6e576964a5c09f9477f3361f341dd576b
2020-09-23 15:43:39 +00:00
Dan Albert
0b176c8038 Replace FutureApiLevel with an ApiLevel.
Keeping the int constant around for now as FutureApiLevelInt because
it's still useful in places that haven't adopted ApiLevel yet for
testing if their non-ApiLevel API level is current or not.

Test: treehugger
Bug: http://b/154667674
Change-Id: I47a7012703f41fdeb56f91edf9c83afa93042deb
2020-09-22 15:04:48 -07:00
Dan Albert
c8060536e8 Replace ApiStrToNum uses with ApiLevel.
Test: treehugger
Bug: http://b/154667674
Change-Id: I2954bb21c1cfdeb305f25cfb6c8711c930f6ed50
2020-09-22 15:04:48 -07:00
Colin Cross
680b11571f Merge "Use alias for test_per_src apex dependencies" am: a018b3df16 am: b524b7a9f6 am: a6aafd9d18 am: bf6baa6494 am: 2b90325fa0
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1406260

Change-Id: I368bf31438820af7de33c2a5f0931763cde2cb4a
2020-09-12 22:42:10 +00:00
Colin Cross
90dab3498d Use alias for test_per_src apex dependencies
AddFarVariationDependencies was broken, which allowed apex to add
dependencies on tests by requesting an empty test_per_src variation
even though some test dependencies did not have a test_per_src
variation.  Add an alias from the pre-test_per_src variation
variant to the empty test_per_src variation, and drop the
test_per_src variation when requesting the depencency.

Test: TestApexWithTests
Change-Id: I2ed7bdd761027956141e25d4d81f9074afe628b6
2020-09-11 16:02:24 -07:00
Xin Li
429c23c1fa Merge Android R
Bug: 168057903
Merged-In: I9d5d0da0f409bd6b131f7e0f6363be061d3045c1
Change-Id: Ie7feaf6a3d0787c750de17540969b876a4306b0a
2020-09-09 20:21:02 -07:00
Treehugger Robot
8c0773bcee Merge "Notice files for prebuilt_etc are included in APEX" am: 4f35976c87 am: 410b20dba2 am: dae2bca0a8 am: 66df7b4c0f am: 02303504dd
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1421373

Change-Id: Ic177090ef0b64e525d464f46dc28038097f50201
2020-09-10 02:07:27 +00:00
Treehugger Robot
410b20dba2 Merge "Notice files for prebuilt_etc are included in APEX" am: 4f35976c87
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1421373

Change-Id: Ifbfd71ff4416b07742e70c9d9aba523d60af1d2b
2020-09-10 00:30:29 +00:00
Jiyong Park
41f637d653 Notice files for prebuilt_etc are included in APEX
This change fixes a bug that notice files for some module type (e.g.
prebuilt_etc) were not included in APEX. This happened because we relied
on WalkPayloadDeps which actually doesn't traverse module types that
don't implement ApexModule interface. prebuilt_etc is one such module
type. Fixing the problem by also iterating the filesInfo array which
has info about all modules that are included in the APEX.

Bug: 166575301
Test: m com.android.tzdata and inspect the built artifact.
NOTICE.html.gz is there.

Change-Id: Iceb055b60184aef2a3e65e44680304853eb79a53
2020-09-09 13:19:56 +09:00
Mark Chien
fd7dad5016 Merge "Include bpf program in APEXes" am: 66a0ae60f8 am: 288f40ba51 am: 05e47c0c5d am: 10893ea830 am: c98c7e3b28
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1418308

Change-Id: I25796c021f4eac123ad324156bcb810a795b7266
2020-09-04 10:39:09 +00:00
Mark Chien
288f40ba51 Merge "Include bpf program in APEXes" am: 66a0ae60f8
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1418308

Change-Id: I86983c1012615e95d1d9aac875a52e92248a17b8
2020-09-04 09:11:22 +00:00
Mark Chien
66a0ae60f8 Merge "Include bpf program in APEXes" 2020-09-04 08:58:21 +00:00
Colin Cross
7a142a4b78 Merge changes I59b7a32a,Ida7bc75a am: 0329f2398c am: 90d0500699 am: 3fa94fc983 am: 78395a8673 am: b443661414
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1406259

Change-Id: I0ab4e4c0775aaf9e08a582e714512e26094640ea
2020-09-03 02:34:00 +00:00
Colin Cross
90d0500699 Merge changes I59b7a32a,Ida7bc75a am: 0329f2398c
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1406259

Change-Id: I9b2b709c59630394aa528c64b4fbe60b52d50be5
2020-09-03 00:21:49 +00:00
Colin Cross
0329f2398c Merge changes I59b7a32a,Ida7bc75a
* changes:
  Only request image and version variations for device SDK dependencies
  Create os and arch variants for GoBinaryTool modules
2020-09-02 23:51:53 +00:00
markchien
2f59ec98d1 Include bpf program in APEXes
bpf program is put to an APEX via 'bpfs' property. It is
placed under etc/bpf directory in it.

Fix: 167530625
Test: m
Change-Id: Ia36b486f0cffb619ecc4f7a318cde881abc5baf4
2020-09-02 22:18:25 +08:00
Jooyung Han
3be2358038 Merge commit '0703fd873fe2ed492266a4457a1e217971c2f4f0' into manual_merge_1e3d9fb376aaf1be6de1bd16a9004ccc95226ab0 am: 4c92cad10e am: 4d565700b5 am: 378ff68eb5 am: 1b06154e8f
Original change: https://googleplex-android-review.googlesource.com/c/platform/build/soong/+/12501269

Change-Id: I0e9a408b9547fad45983f39a5b7dbacd19b29c1d
2020-09-02 08:41:05 +00:00
Jooyung Han
4c92cad10e Merge commit '0703fd873fe2ed492266a4457a1e217971c2f4f0' into manual_merge_1e3d9fb376aaf1be6de1bd16a9004ccc95226ab0
Change-Id: I7115a50254e23e3f7d9cdab960c671195746978e
2020-09-02 14:19:18 +09:00
Xin Li
ca52d2ff53 Merge "Merge Android R (rvc-dev-plus-aosp-without-vendor@6692709)" into stage-aosp-master 2020-09-01 20:01:23 +00:00
Jooyung Han
0703fd873f apex: support prebuilt_firmware for vendor apexes
prebuilt_firmware module is one of many prebuilt_etc-like modules. When
it is soc-specific, it is installed in /vendor/firmware. Similarly, when
prebuilt_firmware is embeded in a vendor apex, installing it in
<apex>/firmware instead of <apex>/etc.

Bug: 162701747
Test: lunch sunfish-userdebug
      m && device boots && vibrator works
Change-Id: I00d28cde42259aaf8221e3897df77efc42b0c1ca
2020-09-01 10:45:13 +09:00
Colin Cross
e50ee06781 Merge "Remove hack for platform availability of modules with missing apexes" am: 9c35c9ae63 am: 207037e03e am: cc1cae5634 am: b2022ea731 am: 17d8eb7d79
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1414830

Change-Id: I374ab32e1e66b696e51c4742baf762a995c96321
2020-08-31 06:32:30 +00:00
Colin Cross
cc1cae5634 Merge "Remove hack for platform availability of modules with missing apexes" am: 9c35c9ae63 am: 207037e03e
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1414830

Change-Id: Idee1003b69dcd845548362effd26d832f8ddf96d
2020-08-31 05:21:46 +00:00
Colin Cross
9c35c9ae63 Merge "Remove hack for platform availability of modules with missing apexes" 2020-08-31 04:20:18 +00:00
Xin Li
65cb5c69ea Merge Android R (rvc-dev-plus-aosp-without-vendor@6692709)
Bug: 166295507
Merged-In: Ifca7b65f4e27bf14cdc30f72f790b0de90130bae
Change-Id: I3a39be5f0b8736de4822c6a14072c78d4e4ad89d
2020-08-29 01:17:45 -07:00
Nikita Ioffe
a0254b5289 Merge "Add F2FS support for APEX build system" 2020-08-28 21:55:24 +00:00
Colin Cross
d2c81a7933 Remove hack for platform availability of modules with missing apexes
The affected modules weren't actually built in the affected build,
delay the platform availability check in Make instead to allow
the build to proceed.

Bug: 154888298
Test: m checkbuild
Change-Id: I21778e38e04635545b352dee2043a33ec9da246b
2020-08-28 13:32:02 -07:00
Theotime Combes
cb70bdb346 Add F2FS support for APEX build system
Add the new property payload_fs_type in apex module type. Either 'f2fs'
or 'ext4'. Default 'ext4'.

Exempt-From-Owner-Approval: Jiyong reviewed this change
Test: m

Bug: 158453869
Merged-In: I36f373251bd597e11acb62af75437a87e2a531ec
Change-Id: I36f373251bd597e11acb62af75437a87e2a531ec
(cherry picked from commit b20d6629aefd2100ec95583ed946de852f1d6bf4)
(cherry picked from commit 1c751e1453)
2020-08-28 15:00:28 +01:00
Theotime Combes
4ba38c1a3f Add F2FS support for APEX build system
Add the new property payload_fs_type in apex module type. Either 'f2fs'
or 'ext4'. Default 'ext4'.

Test: m

Bug: 158453869
Change-Id: I36f373251bd597e11acb62af75437a87e2a531ec
Merged-In: I36f373251bd597e11acb62af75437a87e2a531ec
2020-08-28 12:03:00 +00:00
Theotime Combes
1c751e1453 Add F2FS support for APEX build system
Add the new property payload_fs_type in apex module type. Either 'f2fs'
or 'ext4'. Default 'ext4'.

Exempt-From-Owner-Approval: Jiyong reviewed this change
Test: m

Bug: 158453869
Change-Id: I36f373251bd597e11acb62af75437a87e2a531ec
(cherry picked from commit b20d6629aefd2100ec95583ed946de852f1d6bf4)
2020-08-28 11:28:40 +00:00
Colin Cross
42507337e5 Only request image and version variations for device SDK dependencies
AddFarVariationDependencies was broken, which allowed sdk modules to
request dependencies using image and version variations, even for
host modules that do not have image or version variations.  Make
the image and version variations conditional on device sdk modules.

Test: go test ./sdk
Change-Id: I59b7a32a3782254fd5feb828a5258ee13d4db812
2020-08-26 11:51:51 -07:00
Treehugger Robot
952140ab59 Merge "Reland: Deduplicate APEX variants that would build identically" am: 68bfe074c6 am: a3d1c127a2 am: 4d9984eeb1 am: 76309aab5d am: a106e405e9
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1405087

Change-Id: I5ba652222b6c9ed40fd8c643f92485b26842bd9b
2020-08-21 08:43:24 +00:00
Treehugger Robot
4d9984eeb1 Merge "Reland: Deduplicate APEX variants that would build identically" am: 68bfe074c6 am: a3d1c127a2
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1405087

Change-Id: I7cf0c8d2c27757395e54fb4cc8592f438aa6ccaa
2020-08-21 07:38:47 +00:00
Treehugger Robot
68bfe074c6 Merge "Reland: Deduplicate APEX variants that would build identically" 2020-08-21 06:46:51 +00:00
Colin Cross
aede88c1c7 Reland: Deduplicate APEX variants that would build identically
APEX variants that share the same SDK version and updatability
almost always use identical command line arguments to build but
with different intermediates directories.  This causes unnecessary
build time and disk space for duplicated work.

Deduplicate APEX variants that would build identically.  Create
aliases from the per-APEX variations to the new shared variations
so that the APEX modules can continue to depend on them via the
APEX name as the variation.

This has one significant change in behavior.  Before this change,
if an APEX had two libraries in its direct dependencies and one
of those libraries depended on the other, and the second library
had stubs, then the first library would depend on the implementation
of the second library and not the stubs.  After this change, if
the first library is also present in a second APEX but the second
library is not, then the common variant shared between the two
APEXes would use the stubs, not the implementation.

In a correctly configured set of build rules this change will
be irrelevant, because if the compilation worked for the second
APEX using stubs then it will work for the common variant using
stubs.  However, if an incorrect change to the build rules is
made this could lead to confusing errors, as a previously-working
common variant could suddenly stop building when a module is added
to a new APEX without its dependencies that require implementation
APIs to compile.

This change reduces the number of modules in an AOSP arm64-userdebug
build by 3% (52242 to 50586), reduces the number of variants of the
libcutils module from 74 to 53, and reduces the number of variants
of the massive libart[d] modules from 44 to 32.

This relands I0529837476a253c32b3dfb98dcccf107427c742c with a fix
to always mark permissions XML files of java_sdk_library modules as
unique per apex since they contain the APEX filename, and a fix
to UpdateUniqueApexVariationsForDeps to check ApexInfo.InApexes
instead of DepIsInSameApex to check if two modules are in the same
apex to account for a module that depends on another in a way that
doesn't normally include the dependency in the APEX (e.g. a libs
property), but the dependency is directly included in the APEX.

Bug: 164216768
Test: go test ./build/soong/apex/...
Change-Id: I2ae170601f764e5b88d0be2e0e6adc84e3a4d9cc
2020-08-19 10:21:17 -07:00
Treehugger Robot
2172bcbfd6 Merge "Revert "Deduplicate APEX variants that would build identically"" am: fc4d79598c am: b1d3c4c4d7 am: 671aed59c9 am: d65caf6abb am: a9862dd9d3
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1403647

Change-Id: I4a6c1fd673b35a260e865b183e0cfd6b3d67bd4e
2020-08-18 18:13:38 +00:00
Treehugger Robot
671aed59c9 Merge "Revert "Deduplicate APEX variants that would build identically"" am: fc4d79598c am: b1d3c4c4d7
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1403647

Change-Id: Ia2db2038480c3e09e7c82c1766f9b922e5cc6a11
2020-08-18 17:42:08 +00:00
Pete Bentley
0c7b26e0e2 Revert "Deduplicate APEX variants that would build identically"
This reverts commit d6b2525b00.

Reason for revert: <Breaks tests - confirmed by Forrest: b/165188843>

Change-Id: I7cb68cb87522415004390c0672dc774e0067b122
2020-08-18 13:44:59 +00:00
Treehugger Robot
86555c4c41 Merge "Deduplicate APEX variants that would build identically" am: ca29a25d38 am: aa3235edcc am: 1f160e818b am: a485326840 am: 888957e15e
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1399832

Change-Id: I81e8534c2a28478bdf896b245615087def124f11
2020-08-18 05:50:40 +00:00
Treehugger Robot
1f160e818b Merge "Deduplicate APEX variants that would build identically" am: ca29a25d38 am: aa3235edcc
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1399832

Change-Id: I4fe0cc33956d110e6bb1de49183470eb21992fbf
2020-08-18 05:19:52 +00:00
Colin Cross
d6b2525b00 Deduplicate APEX variants that would build identically
APEX variants that share the same SDK version and updatability
almost always use identical command line arguments to build but
with different intermediates directories.  This causes unnecessary
build time and disk space for duplicated work.

Deduplicate APEX variants that would build identically.  Create
aliases from the per-APEX variations to the new shared variations
so that the APEX modules can continue to depend on them via the
APEX name as the variation.

This has one significant change in behavior.  Before this change,
if an APEX had two libraries in its direct dependencies and one
of those libraries depended on the other, and the second library
had stubs, then the first library would depend on the implementation
of the second library and not the stubs.  After this change, if
the first library is also present in a second APEX but the second
library is not, then the common variant shared between the two
APEXes would use the stubs, not the implementation.

In a correctly configured set of build rules this change will
be irrelevant, because if the compilation worked for the second
APEX using stubs then it will work for the common variant using
stubs.  However, if an incorrect change to the build rules is
made this could lead to confusing errors, as a previously-working
common variant could suddenly stop building when a module is added
to a new APEX without its dependencies that require implementation
APIs to compile.

This change reduces the number of modules in an AOSP arm64-userdebug
build by 3% (52242 to 50586), reduces the number of variants of the
libcutils module from 74 to 53, and reduces the number of variants
of the massive libart[d] modules from 44 to 32.

Bug: 164216768
Test: go test ./build/soong/apex/...
Change-Id: I0529837476a253c32b3dfb98dcccf107427c742c
2020-08-17 15:18:31 -07:00