Commit Graph

430 Commits

Author SHA1 Message Date
Treehugger Robot
f1c48afc31 Merge "arch specific dependencies are supported for apex" 2020-12-17 00:15:26 +00:00
Jiyong Park
892a98f0a3 versioning macro is exported from prebuilt stubs as well
This change fixes a bug that the versioning macro (__LIBNAME_API__) is
omitted for prebuilts providing a stub.

Bug: 175166063
Test: m nothing

Change-Id: I1cce5ab58ef245622861200ec0d8b0f84e3178ed
2020-12-16 03:54:44 +09:00
Jiyong Park
f7c3bbe433 Prebuilt stub not available to platform is handled correctly
When a shared library providing stubs is included in an APEX, only the
stub variant (i.e. version:"1", etc.) gets emitted to Android.mk. This
enforces that everything in the Make world to link to the stub providing
only the public APIs of the library. The non-stub variant (i.e.
version:"") isn't exposed, otherwise, others will be able to access
private part of the lib which isn't guaranteed to be stable.

This has been done by unhiding the stub variant when it is known that
the library is actually included in an APEX. Note that stub variants are
by default hidden.

The above mechanism however doesn't work when the shared library is
replaced by a prebuilt and when the APEX is also replaced by a prebuilt.
Then, the fact that the prebuilt library is actually in the APEX gets
lost.  In that case, AnyVariantDirectlyInAnyApex() returns false for the
prebuilt library. As a result, the stub variant remains hidden and not
emitted to Android.mk.

This change fixes the problem by checking if the lib isn't available for
the platform at all. If not available for the platform (e.g.
apex_available doesn't have "//apex_available:platform"), the lib is
assumed to be included in an APEX even when it actually didn't go
through the apex mutator (... because it's a prebuilt).

Bug: 175166063
Test: m nothing

Change-Id: I41fabd5b368baecf4dc3c5a080b466f8bcd79d77
2020-12-15 09:17:36 +09:00
Jiyong Park
59140307ec arch specific dependencies are supported for apex
The apex module type now supports arch specific native dependencies.

apex {
    name: "myapex",
    arch: {
        arm64: {
	    native_shared_libraries: ["arm64_only_lib"],
	},
    },
}

Bug: 174639241
Test: m nothing
Change-Id: I3f00aa87c480d4127b27d33e9620c4336824e937
2020-12-14 18:44:04 +09:00
Paul Duffin
4d934af2ee Merge "Extract testDexpreoptWithApexes() for reuse" 2020-12-11 10:06:07 +00:00
Jiyong Park
b555609098 Merge "rust ffi libraries can be included in APEX" 2020-12-10 23:54:34 +00:00
Jiyong Park
0d399b5915 Merge "Add min_sdk_version to Rust modules." 2020-12-10 23:54:34 +00:00
Paul Duffin
c3bbb96667 Extract testDexpreoptWithApexes() for reuse
Extracts testDexpreoptWithApexes() from testDexpreoptWithApexes() to
reuse in other tests.

Test: m nothing
Bug: 171061220
Change-Id: Iee62b68d72133b4cc066e2875bd27d7051206cbc
2020-12-10 20:16:57 +00:00
Steven Moreland
35989ae98b Merge "target.apex.exclude_[shared|static]_libs to cc_* modules" 2020-12-10 19:02:02 +00:00
Jiyong Park
2920d2cec0 Merge "test_for is available for all cc_* module types" 2020-12-10 00:16:29 +00:00
Ivan Lozano
3e9f9e47cf Add min_sdk_version to Rust modules.
Add the min_sdk_version property to Rust modules so they can declare
a minimum SDK version they support for use with APEX modules.

Test: New Soong test passes.
Bug: 174862583
Change-Id: I2829053a320f50c218783dee5adbeff9cef81e8e
2020-12-09 11:30:44 -05:00
Jiyong Park
46a512f1f5 test_for is available for all cc_* module types
Sometimes, the ordinary cc_library_* modules need test_for property when
they are part of a bigger cc_test. Instead of propagating the test_for
property from cc_test to its dependencies, this change requires the very
dependency which needs access to the private part of an APEX to
explicitly have the test_for property.

Bug: 161575591
Test: m
Change-Id: Ie1ffe9a60cd2ab02d41bbe5a98225a40392470f6
2020-12-09 12:59:19 +09:00
Jiyong Park
f2cc1b77f3 rust ffi libraries can be included in APEX
This CL fixes a bug that rust_ffi_shared libraries can't be included in
an APEX either directly (via native_shared_libs property) or indirectly.

Bug: 175105284
Test: m
Change-Id: I2e2d3e11b3901f3412b9725e47a8ebf31f93847b
2020-12-09 00:29:54 +09:00
Mohammad Samiul Islam
3cd005d347 Enable soong build tool to handle APEX compression
1. Soong can now detect PRODUCT_COMPRESSED_APEX flag
     We don't want APEX to be compressed on all devices. Only those that
     have explicitely set PRODUCT_COMPRESSED_APEX flag.
2. Handle "compressible" field in soong build rule
     On devices that supports APEX compression, all APEX will be
     compressed by default. If any apex does not want to be compressed,
     they will need to state that by setting "compressible" field to
     false
3. Can use apex_compression_tool to compress APEX
     Note we compress the APEX after it has been signed. That way, when
     we decompress we will get a signed APEX.
4. Place the compressed APEX in system with .capex extension
     This makes it easy to identify. We still preserve the original
     extension so that when we decompress, we can just rename by cuttif
     off the .capex extension.

Note: with this change, we can create a system image with compressed
APEX, but we cannot boot with it since platform doesn't know how to
handle .capex files. Platform support will be added on follow up CLs.

Bug: 172911362
Test: OVERRIDE_PRODUCT_COMPRESSED_APEX=true m (apex_test.go)
Test: observed $OUT/system/apex has .capex files
Change-Id: I20ac4c4ceb521924c751a6017f979b2d808fdded
2020-12-08 13:06:25 +00:00
Jiyong Park
e386754898 target.apex.exclude_[shared|static]_libs to cc_* modules
The property is used to exclude some shared and static libs when the
module is built for an APEX.

Bug: 166468760
Test: m
Change-Id: I0dcaa4ae94c01aa00dc5539c60d3054c57fd8824
2020-12-04 18:09:39 +09:00
Treehugger Robot
d348c41af5 Merge "Don't use module names with "prebuilt_" prefixes in make dependencies." 2020-12-04 06:23:51 +00:00
Martin Stjernholm
2856c66c99 Don't use module names with "prebuilt_" prefixes in make dependencies.
The distinction between foo and prebuilt_foo doesn't exist in make, so
this could create invalid dependencies when both source and prebuilt
modules exist and the prebuilts are preferred.

Test: `m` with ART module prebuilts with prefer:true in the tree
Test: m nothing
Bug: 172480615
Change-Id: I90b76a8f38493882b3330d1b6789159852e59d55
2020-12-04 01:00:23 +00:00
Dan Albert
b19953d04a Rename __ANDROID_SDK_VERSION__.
__ANDROID_SDK_VERSION__ is ambiguous. Rename it
__ANDROID_APEX_MIN_SDK_VERSION__ so it's clear which SDK version it
refers to.

Note that this is still different from the minSdkVersion of the module
being compiled. This is the max of *all* the minSdkVersions of modules
that this module shares an APEX with.

Test: treehugger
Bug: None
Change-Id: Id9cbd80a6bc99db8227daef4e1db9c893e63ad1e
2020-11-30 14:39:01 -08:00
Jiyong Park
99644e92c8 rust modules can be included in apex
We will have some APEXes having rust binaries and libraries. So, adding
the support for the types of modules.

rust.Module now inherits from android.ApexModuleBase and implements
the android.ApexModule interface.

Bug: 172414324
Test: m

Exempt-From-Owner-Approval: rebased after +2 from the owner
Change-Id: I356ef4c45f782a6460f001e83af96d1710642d80
2020-11-30 15:40:48 +00:00
Colin Cross
1af783fae7 Merge "Replace android.WriteFile rule with android.WriteFileRule" 2020-11-16 23:11:41 +00:00
Jooyung Han
74f576de58 Remove "apex_uses" mutator
which was added to support built-time common apex. But the common apex
was abandoned and it is used nowhere.

This is a manual revert of 5c998b9ff8.

Bug: 173329465
Test: m
Change-Id: I684e6b8bfe2ef5b53943329f4aaa5a5ccba27738
2020-11-16 09:08:30 +00:00
Colin Cross
cf371cc1f7 Replace android.WriteFile rule with android.WriteFileRule
The android.WriteFile rule takes careful escaping to produce the
right contents.  Wrap it in an android.WriteFileRule that handles
the escaping.

Test: compare all android.WriteFile outputs
Change-Id: If71a5843af47a37ca61714e1a1ebb32d08536c31
2020-11-14 16:26:00 -08:00
Colin Cross
ae8600b507 Pass Config to NewTestContext instead of ctx.Register
Prepare for using Config when adding singletons by passing
Config to NewTestContext and NewContext instead of to ctx.Register.
This will enable a followup change to store SingletonMakeVarsProviders
registered on the Context in the Config, which is necessary to run
multiple tests in parallel without data races.

Test: all soong tests
Change-Id: Id229629a4e42ff4487d317241673837726c075fc
2020-11-12 10:07:49 -08:00
Justin Yun
63e9ec70bb Define product_available property
To make a module available to product variants, it must define
`product_available: true`. `vendor_available: true` will not create
product variants any more.
However, in this CL, we don't change the behavior of
`vendor_available` property. It still creates both variants. After we
update all Android.bp files that need to provide product variants
with `product_available: true`, we may upload the remaining patches.

Bug: 150902910
Test: lunch aosp_arm64-userdebug && m
Change-Id: I0fd5be7bbae2c45d5cab3c3c2ca49f53a9b6f975
2020-11-08 23:53:22 +00:00
Colin Cross
a717db7304 Don't create stubs variants of static libraries
Statically linking stub implementations doesn't make sense, don't
create stubs variants of static libraries.

Bug: 170784825
Test: all soong tests
Change-Id: Ie73635244516edf6da884e3a7a275971a9bd7839
2020-10-29 14:17:37 -07:00
Paul Duffin
e10dfa4e3d Cleanup usages of CreateConfiguredJarList
After previous refactorings the CreateConfiguredJarList function is now
only used in tests and are supplied with a PathContext that will cause
ReportPathErrorf() to panic. So, this change removes the ctx parameter,
calls panic directly on any error and renames the method to make it
clear that it is for testing only.

Bug: 171479578
Test: m nothing
Change-Id: Icfb4bdfe720afa855b64ecf0e74a0b030882d029
2020-10-27 10:36:29 +00:00
Paul Duffin
69d1fb1e39 Switch BootJars/UpdatableBootJars to ConfiguredJarList
This change:
* Switches BootJars/UpdatableBootJars fields of config.productVariables
  from []string to ConfiguredJarList.
* Updates BootJars() method to simply concatenate the jars list from
  the BootJars/UpdatableBootJars fields.
* Adds an UnmarshalJSON(..) method to ConfiguredJarList to support
  unmarshalling from a single string array to avoid having to change the
  format of the JSON file from which the configuration is loaded.
* Adds some additional calls to ConfiguredJarList(..) in tests to
  convert from []string to ConfiguredJarList. They pass nil as the
  ctx argument as there is no suitable PathContext which will cause any
  errors to be thrown using panic. That is reasonable for hard coded
  values in tests. A follow up change will clean up the calls to
  ConfiguredJarList(..).

Bug: 171479578
Test: m nothing
Change-Id: I59b94dafb479ccd8f0471ed802be175af57be271
2020-10-27 10:36:29 +00:00
Colin Cross
ed5dee0a8f Merge "Add llndk_stubs property" 2020-10-23 18:09:42 +00:00
Steven Moreland
6e36cd6104 apex dependency error: note 'apex_available'
This error tracks `apex_available` closure, but the message doesn't
indicate this variable specifically. Calling it out to try to make
the problem more searchable/discoverable/intuitable.

Bug: N/A
Test: apex_test.go
Change-Id: Ib254ace1dbd4e77d073ed4f98ee181dd86adfcfc
2020-10-22 01:23:33 +00:00
Colin Cross
0477b42276 Add llndk_stubs property
Prepare for making the relationship between an llndk_library stubs
module and the cc_library implementation module explicit by
adding an llndk_stubs property.  Each cc_library will be updated
to point to its llndk_library, and the llndk_library name will
be changed to make the .llndk suffix explicit.  Then the implicit
connection and suffix can be removed.

Bug: 170784825
Test: m checkbuild
Change-Id: I6b0482a3f286ec29b2e928551aa4317749f2b499
2020-10-21 10:55:33 -07:00
Colin Cross
d4620701ad Merge "Start using Providers instead of direct module access" 2020-10-13 21:48:26 +00:00
Evgenii Stepanov
2080bfe79a Support asan/hwasan versions of prebuilts.
In apex_set and cc_prebuilt_library_*, provide a way to specify an
alternative source to use when build with sanitizers.

Test: prebuilt_test, apex_test
Change-Id: I1ab8091bf696d94da3547cf5248853df489bdee6
2020-10-13 03:37:32 +00:00
Colin Cross
0de8a1e17b Start using Providers instead of direct module access
Export information about static libraries, shared libraries and
exported flags through Providers instead of accessing the module
directly.  Much more is left to be converted, but this significantly
simplifies the dependencies on libraries with stubs by making it easy
for a module to masquerade as another by simply exporting the
providers from the other module.  Instead of depending on all the
versions of a library and then picking which one to use later, it
can depend only on the implementation variant and then select the
right SharedLibraryInfo from the variant.

Test: m checkbuild
Test: only expected changes to build.ninja
Change-Id: I1fd9eb4d251cf96ed8398d586efc3e0817663c76
2020-10-12 16:55:47 -07:00
Colin Cross
405af07859 Revert "Make lots of tests run in parallel"
This reverts commit 323dc60712.

Reason for revert: Possible cause of test instability
Bug: 170513220
Test: soong tests

Change-Id: Iee168e9fbb4210569e6cffcc23e60d111403abb8
2020-10-09 18:34:24 -07:00
Colin Cross
323dc60712 Make lots of tests run in parallel
Putting t.Parallel() in each test makes them run in parallel.
Additional t.Parallel() could be added to each subtest, although
that requires making a local copy of the loop variable for
table driven tests.

Test: m checkbuild
Change-Id: I5d9869ead441093f4d7c5757f2447385333a95a4
2020-10-06 15:12:22 -07:00
Colin Cross
56a8321c21 Remove global state from apex modules
A global variant was used to store the global mapping between
modules and APEXes.  Replace it with storing pointers to APEX
contents inside each module so that they can query the contents
of any APEXes they belong to.

Bug: 146393795
Test: all Soong tests
Test: single line change to build.ninja host install dependency ordering
Test: no Android-${TARGET_PRODUCT}.mk, make_vars-${TARGET_PRODUCT}.mk or late-${TARGET_PRODUCT}.mk
Change-Id: Id2d7b73ea27f8c3b41d30820bdd86b65c539bfa4
2020-10-06 13:39:57 -07:00
Artur Satayev
4e1f2bd0d8 Track allowed transitive deps in any updatable module.
Instead of tracking per module and per module variant, track allowed
list of dependecies for all modules combined. This avoids issues with
different products and different downstream branches having different
build graphs.

To compare allowed_deps.txt vs head, run:
:; m -j out/soong/apex/depsinfo/new-allowed-deps.txt.check

To update source allowed_deps.txt, run:
:; build/soong/scripts/update-apex-allowed-deps.sh

Bug: 149622332
Test: m
Change-Id: Ic518fbd9ebfe1b46aaf9a58df731780a7e5a676b
Merged-In: Ic518fbd9ebfe1b46aaf9a58df731780a7e5a676b
(cherry picked from commit 453555083b)
(cherry picked from commit e5207cd9a6)
2020-09-30 21:17:42 +00:00
Colin Cross
7812fd3814 Fix ChooseSdkVersion after api levels
I2954bb21c1cfdeb305f25cfb6c8711c930f6ed50 switched normalizeVersions
to work on ApiLevels, which inadvertantly caused it to return "current"
instead of "10000" for libraries that specify "current" in their stubs
property.  ChooseSdkVersion couldn't handle "current" because it was
manually converting the version to an int.  Switch ChooseSdkVersion
to use ApiLevels instead so that it can handle "current".

Test: m checkbuild
Change-Id: Id412359e092483ba419118dd03bc206fae702a96
2020-09-26 00:53:03 +00:00
Dan Albert
4f378d75aa Convert more versions in config to ApiLevel.
The test case I removed is invalid. The codename has had its int
assigned, but the config claims it is not final.

If this ever does need to be supported it's just a matter of making
sure the Q -> 29 mapping (or whatever) in the finalized codenames map
in android/api_levels.go.

Test: treehugger
Bug: http://b/154667674
Change-Id: I4f42ec2fd4a37750519ee3937938a1c65b6bb1e8
2020-09-22 16:01:56 -07:00
Dan Albert
c8060536e8 Replace ApiStrToNum uses with ApiLevel.
Test: treehugger
Bug: http://b/154667674
Change-Id: I2954bb21c1cfdeb305f25cfb6c8711c930f6ed50
2020-09-22 15:04:48 -07:00
Jooyung Han
8e5685ddca apex: prebuilt_firmware installs in /etc/firmware
Installation path of prebuilt_firmware varies accoding to target
partitions. When it is for system, it installs a file in
/system/etc/firmware while it installs in /vendor/firmware for vendor.

We'd better be consistent about installation path when it is
for APEXes regardless of target partition. Otherwise, ueventd would need
to scan both /apex/*/etc/firmware and /apex/*/firmware.

Having /etc prefix for prebuilt modules helps module owners to predict
the layout of the contents.

Bug: 162701747
Bug: 167942098
Test: soong tests
Test: loading vibrator firmware from vibrator apex (sunfish)
Change-Id: I7a7105026426f8a7a156bc947304930f761c81f3
2020-09-21 11:02:57 +09:00
Mark Chien
288f40ba51 Merge "Include bpf program in APEXes" am: 66a0ae60f8
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1418308

Change-Id: I86983c1012615e95d1d9aac875a52e92248a17b8
2020-09-04 09:11:22 +00:00
Mark Chien
66a0ae60f8 Merge "Include bpf program in APEXes" 2020-09-04 08:58:21 +00:00
markchien
2f59ec98d1 Include bpf program in APEXes
bpf program is put to an APEX via 'bpfs' property. It is
placed under etc/bpf directory in it.

Fix: 167530625
Test: m
Change-Id: Ia36b486f0cffb619ecc4f7a318cde881abc5baf4
2020-09-02 22:18:25 +08:00
Jooyung Han
4c92cad10e Merge commit '0703fd873fe2ed492266a4457a1e217971c2f4f0' into manual_merge_1e3d9fb376aaf1be6de1bd16a9004ccc95226ab0
Change-Id: I7115a50254e23e3f7d9cdab960c671195746978e
2020-09-02 14:19:18 +09:00
Jooyung Han
0703fd873f apex: support prebuilt_firmware for vendor apexes
prebuilt_firmware module is one of many prebuilt_etc-like modules. When
it is soc-specific, it is installed in /vendor/firmware. Similarly, when
prebuilt_firmware is embeded in a vendor apex, installing it in
<apex>/firmware instead of <apex>/etc.

Bug: 162701747
Test: lunch sunfish-userdebug
      m && device boots && vibrator works
Change-Id: I00d28cde42259aaf8221e3897df77efc42b0c1ca
2020-09-01 10:45:13 +09:00
Xin Li
65cb5c69ea Merge Android R (rvc-dev-plus-aosp-without-vendor@6692709)
Bug: 166295507
Merged-In: Ifca7b65f4e27bf14cdc30f72f790b0de90130bae
Change-Id: I3a39be5f0b8736de4822c6a14072c78d4e4ad89d
2020-08-29 01:17:45 -07:00
Colin Cross
9e93b1bb12 Remove TestApexWithJniLibs_Errors
AddFarVariationDependencies was broken, which allowed dependencies
on non-libraries to be added even though a link variation was
requested.  When AddFarVariationDependencies is fixed, attempting
to add a non-library to jni_libs results in the usual missing
variant error.

Test: none
Change-Id: I6aa281491b8af5f141a2d132e248c75f95d68ab5
2020-08-21 16:18:11 -07:00
Colin Cross
aede88c1c7 Reland: Deduplicate APEX variants that would build identically
APEX variants that share the same SDK version and updatability
almost always use identical command line arguments to build but
with different intermediates directories.  This causes unnecessary
build time and disk space for duplicated work.

Deduplicate APEX variants that would build identically.  Create
aliases from the per-APEX variations to the new shared variations
so that the APEX modules can continue to depend on them via the
APEX name as the variation.

This has one significant change in behavior.  Before this change,
if an APEX had two libraries in its direct dependencies and one
of those libraries depended on the other, and the second library
had stubs, then the first library would depend on the implementation
of the second library and not the stubs.  After this change, if
the first library is also present in a second APEX but the second
library is not, then the common variant shared between the two
APEXes would use the stubs, not the implementation.

In a correctly configured set of build rules this change will
be irrelevant, because if the compilation worked for the second
APEX using stubs then it will work for the common variant using
stubs.  However, if an incorrect change to the build rules is
made this could lead to confusing errors, as a previously-working
common variant could suddenly stop building when a module is added
to a new APEX without its dependencies that require implementation
APIs to compile.

This change reduces the number of modules in an AOSP arm64-userdebug
build by 3% (52242 to 50586), reduces the number of variants of the
libcutils module from 74 to 53, and reduces the number of variants
of the massive libart[d] modules from 44 to 32.

This relands I0529837476a253c32b3dfb98dcccf107427c742c with a fix
to always mark permissions XML files of java_sdk_library modules as
unique per apex since they contain the APEX filename, and a fix
to UpdateUniqueApexVariationsForDeps to check ApexInfo.InApexes
instead of DepIsInSameApex to check if two modules are in the same
apex to account for a module that depends on another in a way that
doesn't normally include the dependency in the APEX (e.g. a libs
property), but the dependency is directly included in the APEX.

Bug: 164216768
Test: go test ./build/soong/apex/...
Change-Id: I2ae170601f764e5b88d0be2e0e6adc84e3a4d9cc
2020-08-19 10:21:17 -07:00
Ulyana Trafimovich
910eb70891 Merge "Add structured representation for colon-separated jar lists." 2020-08-06 15:07:27 +00:00