Commit Graph

843 Commits

Author SHA1 Message Date
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
447f6c99c9 Bp2Build common properties auto-handling
Introduce `commonAttributes` & `fillCommonBp2BuildModuleAttrs used in
CreateBazelTargetModule

Adapt `bp2BuildInfo` to use `commonAttrs` instead of `Name`.
And thus also all downstream users of `CreateBazelTargetModule`.

As initial user, the Soong `required` property will be
translated to Bazel's `data`.

Bug: 198146582, 196091467
Test: build_converstion_test.go:TestCommonBp2BuildModuleAttrs
Test: go test
Test: mixed_{libc,droid}.sh
Change-Id: Ib500e40f7e2cb48c459f1ebe3188962fc41ec124
2021-10-04 14:43:04 +00:00
Martin Stjernholm
8be1e6db16 Propagate the dex jar path as an OptionalPath which is either valid or
invalid with a message.

This will allow propagating any error from the deapexer module for
prebuilt APEXes to the location where the dex jars get used. It's only
at those points that we can raise errors about not being able to
extract files from the deapexer modules if they are invalid, and this
way we avoid encoding knowledge there about why they may be invalid.

To keep the refactoring limited it intentionally does not change any of
the existing logic for when dexJarFiles are set or not (non-nil vs nil
prior to this change), although there may be opportunity to use this
for more conditions when dex jars aren't available.

The refactoring is also not extended to
dexpreopt.ClassLoaderContextMap.

Test: m nothing
Bug: 192006406
Change-Id: I68986dccd9a9b3fee4d24caa1947ea17a36caedc
2021-09-23 17:19:55 +01:00
Jiakai Zhang
519c5c82e5 Revert^2 "Preopt APEX system server jars."
This reverts commit 92346c4832.

Reason for revert: Fixed build error.

The build error is fixed by ag/15841934. This CL remains unchanged. This
CL will be submitted AFTER ag/15841934 is submitted.

Bug: 200024131
Test: 1. Patch this CL and ag/15841934 into internal master.
  2. sudo vendor/google/build/build_test.bash

Change-Id: I5f2b8357846fc7dda56e25ebe6ffb095e8047ec8
2021-09-16 06:25:26 +00:00
Adrian Roos
92346c4832 Revert "Preopt APEX system server jars."
This reverts commit ca9bc98e0c.

Reason for revert: breaks build
Bug: 200024131

Change-Id: Ide07b4c4d267370ae31107b1598b2f878c701282
2021-09-15 14:11:07 +00:00
Jiakai Zhang
ca9bc98e0c Preopt APEX system server jars.
The path to the artifacts will in the form of
/system/framework/oat/<arch>/<encoded-jar-path>@classes.{odex,vdex,art},
where <encoded-jar-path> is the path to the jar file with "/" replaced
by "@". For example,
/system/framework/oat/x86_64/apex@com.android.art@javalib@service-art.jar@classes.odex

There will be a follow-up CL to update ART runtime to recognize
artifacts in that path.

Test: m com.android.art
Bug: 194150908
Change-Id: Ic89fd63c4b1cd565684cead83fc91dae3bc97a4c
2021-09-15 09:01:07 +00:00
Samiul Islam
7c02e26cba Relax restriction on suffix of apex_set filename to include .capex
Compressed APEX files generated by the build system have the .capex
extension. This CL allows us using those .capex files with their actual
suffix.

Bug: 197258691
Test: manual + unit test added
Change-Id: I79173ef942326b48b5e659f1873d80f12b212339
2021-09-08 18:29:47 +01:00
Daniel Norman
5a3ce13c6a Allows prebuilts in override_apex.
Bug: 197787336
Test: Create an override_apex that uses a different `prebuilts` than its
      base. Observe built contents.
Test: apex_test.go
Change-Id: I7666ed6cfe3f2fa5dd81e5f8c1961477dabbbd3c
2021-09-02 09:28:52 -07:00
markchien
7c803b8746 Make bpfs properties overridable
To support different variable bpfs file between mainline module and
non-updatable module(e.g. Android GO). Make bpfs properties overridable.

Test: m
Bug: 190523685
Change-Id: I4c63e35f74230de94b21f3ceb2beb90f0f9ddb11
2021-08-27 03:03:45 +00:00
Liz Kammer
380dbb9327 Merge "Don't create a new module for bp2build conversion." 2021-08-18 19:28:20 +00:00
Liz Kammer
2ada09a546 Don't create a new module for bp2build conversion.
A performance improvement for bp2build as Blueprint/Soong no longer have
the overhead of additional modules. The creation of these modules
results in:
* traversal of additional modules for each subsequent mutator
* synchronization over a go channel to collect newly created modules:
https://cs.android.com/android/platform/superproject/+/master:build/blueprint/context.go;l=2594,2600;drc=1602226f23181b8c3fbfcaf3358f0297e839d7d3

We avoid both of these by storing the information directly in the
underlying module.

Also as a fringe benefit, removes some necessary boilerplate for
conversion.

For benchmarks, reduces runtime ~1% for 1% converted, ~24% for 100%
converted. See more: go/benchmarks-for-https:-r.android.com-1792714

Test: ran benchmarks/tests in bp2build
Test: build/bazel/ci/bp2build.sh
Change-Id: Ie9273b8cbab5bc6edac1728067ce184382feb211
2021-08-17 15:57:09 -04:00
Martin Stjernholm
0e17c1a57e Remove fallback workaround for APEX variants which is now unnecessary.
Test: m nothing
Bug: 191269918
Change-Id: I13091d5ffdd46947899db273b9cb2399dda23907
2021-08-15 22:32:37 +01:00
Colin Cross
8355c1524f Collect lint reports from apps in apexes
Lint reports were only being collected from jars in apexes, not
apps.

Bug: 188772607
Test: mainline_modules_arm64 build
Change-Id: Ie84819343f8b8ab3f43496bbc9b7876d0d0d8e70
2021-08-10 19:25:59 -07:00
Treehugger Robot
03beb0ec77 Merge changes Ie2012adb,Ib0283ca6
* changes:
  static rust binaries are also prohibited in APEXes
  Revert "Revert "Prohibit static executable in APEX""
2021-08-04 04:35:20 +00:00
Jiyong Park
d12979d09a static rust binaries are also prohibited in APEXes
Also fixes a bug that the test runs for host APEXes like
com.android.art.host.

Bug: 185971244
Test: m
Test: build mainline_modules target on aosp-master
Change-Id: Ie2012adbf2f4eda5454d5eaa30f128fb1e20ad37
2021-08-03 07:52:46 +00:00
Jiyong Park
192600a7d0 Revert "Revert "Prohibit static executable in APEX""
This reverts commit 2125aab504.

Reason for revert: relanding along with a forward fix

Change-Id: Ib0283ca6beefa2f3073860287d00553ad0af6317
2021-08-03 07:52:17 +00:00
Jiyong Park
2fe71a5165 Merge "Revert "Prohibit static executable in APEX"" 2021-08-03 07:50:26 +00:00
Yen-Chao Chen
2125aab504 Revert "Prohibit static executable in APEX"
This reverts commit 3a8130c797.

Reason for revert: Build failed.

Change-Id: I28c72c8c3adaeb23bff9bffe3994971aa3f2e4f4
2021-08-03 04:51:12 +00:00
Jiyong Park
d8a3513155 Merge "Prohibit static executable in APEX" 2021-08-03 03:48:01 +00:00
Rupert Shuttleworth
9447e1ee12 Add support for prebuilts attribute to the bp2build APEX converter.
Test: Updated unit test.

Change-Id: I73dfa84d7a55c0b94393ad6257b5f16313422852
2021-07-28 06:03:17 -04:00
Jiyong Park
3a8130c797 Prohibit static executable in APEX
If a library is statically included in a static executable, it's
behavior is different from the case where the same library is statically
(or dynamically) linked to a non-static executable. To reduce the number
of test configs that we need to consider, it is required to prohibit
static executables in APEXes.

Bug: 185971244
Test: m
Change-Id: I0cd8fca669d55088e2329fef1f859eeb297502dd
2021-07-27 22:42:23 +09:00
Rupert Shuttleworth
6e4950a061 Add some more properties to the bp2build APEX converter (second try)
In particular:

- AndroidManifest
- file_contexts
- key
- certificate
- min_sdk_version
- updatable
- installable
- native_shared_libs
- binaries

Test: Updated unit test

Change-Id: I1c6e8c4b6b24ce487f64e5d37bd594dbb000fe6f
2021-07-27 05:02:50 -04:00
Rupert Shuttleworth
0358478150 Revert "Add some more properties to the bp2build APEX converter."
This reverts commit 69bf4c0d60.

Reason for revert: Possibly broke CI (https://android-build.googleplex.com/builds/quarterdeck?branch=aosp-master-bazel&target=bp2build-incremental&lkgb=7266256&fkbb=7237028), rolling back just in case...

Change-Id: I14ebac4f47576a73285d1f724ae4499036e8e678
2021-07-27 02:37:32 +00:00
Rupert Shuttleworth
69bf4c0d60 Add some more properties to the bp2build APEX converter.
In particular:

- AndroidManifest
- file_contexts
- key
- certificate
- min_sdk_version
- updatable
- installable
- native_shared_libs
- binaries

Test: Updated unit test

Change-Id: I9ea13f021d63ec0f75006b3b0b23ec8178c7e9da
2021-07-26 07:32:36 -04:00
Jingwen Chen
f59a8e117e Fix apex bp2build.
apexBundle was missing InitBazelModule for it to have bazel_module properties.

Test: soong unit test
Fixes: 193866723
Change-Id: I5fb38c8b2e8edaecef44162491d399c0223d3fb1
2021-07-16 09:32:49 +00:00
Rupert Shuttleworth
a9d76dd423 Add partial bp2build support for APEX targets (second try).
Test: Added unit test
Change-Id: Icbf0475251aac5d1f13bd35a18f90b6c70f3ae29
2021-07-02 07:36:07 -04:00
Weng Su
7ce730b90c Revert "Add partial bp2build support for APEX targets."
Revert submission 1733232-roboleaf-apex

Reason for revert: b/192523357 Broken build 7512065 on aosp-master on sdk_x86_64-sdk
Reverted Changes:
Ie82c263f6:Add a stub rule for building APEX bundles.
I93e2a25fa:Add partial bp2build support for APEX targets.

Change-Id: I8e836fe7f466e02f3108680f9653412299e929e0
2021-07-01 06:51:11 +00:00
Rupert Shuttleworth
22b606b5f0 Add partial bp2build support for APEX targets.
Test: Added unit test

Change-Id: I93e2a25fa43eec1ffb8fb30c346a9b523195a0fd
2021-06-30 22:20:23 -04:00
Martin Stjernholm
bfffae7bec Fix inconsistencies in the apex names used for the apex variations.
Add an apex_name property to prebuilt APEX modules to allow specifying
the "runtime" name of the APEX, i.e. the one it gets mounted as in /apex/,
which is also the one used for the apex variations.

Introduce a callback to retrieve that name consistently for all APEX
modules (apex, override_apex, prebuilt_apex, and apex_set), and update
some apex mutator code paths to use it.

For APEX source modules (apex and override_apex), it's necessary to add
a new field in apexBundle, since the name property gets overridden for
the override variant that override_apex creates.

Test: m nothing
Bug: 191269918
Change-Id: If8612639bffdf91cbcab3387b0603bf5dffef1f5
2021-06-25 16:41:05 +01:00
Jiyong Park
9477c26f6c Add platform_apis property to APEX module type
The property is used to allow non-updatable APEXes to use platform APIs
(e.g. symbols marked as "# platform-only").

Bug: 191637950
Test: m com.android.virt com.android.compos
Merged-In: Id2410b4e38a78ec2146a42298840954381a7c472
Change-Id: Id2410b4e38a78ec2146a42298840954381a7c472
(cherry picked from commit fb63625a7f)
2021-06-24 11:28:33 +09:00
Treehugger Robot
348f20be0e Merge "AIDEGen: Collect dependencies info from APEX module." 2021-06-21 07:17:28 +00:00
Nikita Ioffe
f932613c3f Merge "Set default value of generate_hashtree property to true" 2021-06-18 13:14:22 +00:00
satayev
b98371cc13 Add classpath fragment property to skip proto generation.
This must always be true for updatable apexes, but is not necessary
for non-updatable apexes like com.android.i18n.

In a follow up this will be used to figure out whether apex boot jars
should be bundled into platform_bootclasspath.

Bug: 191127295
Test: atest CtsClasspathsTestCases derive_classpath_test
Change-Id: Ib7dc5b057cb24955222fb97f3ff9da079f30ed77
2021-06-17 12:59:33 +01:00
Paul Duffin
6717d88f46 Create APEX variant for prebuilt_apex/apex_set
Previously, the prebuilt_apex/apex_set did not have an apex specific
variant created which meant that they depended upon the platform
variant of the modules it depended upon. This change creates an
apex variant for them just as is done for the apex module type which
causes it to depend upon the apex specific variant of the modules it
depends upon.

Test: m droid
Bug: 179354495
Change-Id: I7d6f3609c267b3e90b90b9befe7d76f351a0c2bd
2021-06-16 18:54:19 +01:00
Nikita Ioffe
e261ae64e2 Set default value of generate_hashtree property to true
Test: presubmit
Test: m
Bug: 190621617
Change-Id: I546730f168632c5d5d41c686853bde93a8ba7911
2021-06-16 18:15:03 +01:00
bralee
b0c1f0c45a AIDEGen: Collect dependencies info from APEX module.
Add apex dependencies.

Bug: 188004865
Test: 1. aidegen frameworks/base/apex/appsearch frameworks/base/apex/appsearch/testing   frameworks/base/services/tests/servicestests   frameworks/base/core/tests/coretests   cts/tests/appsearch   external/icing   vendor/google_testing/integration/tests/scenarios
      2. The dependency graph shows it is workable to provide the result to let IDE's [Project structure] to include the service-appsearch as source directory.
Change-Id: Ifd1548f0a86c1c73f8279e8ecea67756aeb1281e
2021-06-15 08:35:13 +08:00
Nikita Ioffe
9d9960f240 Rename test_only_no_hashtree to generate_hashtree
In order not to introduce any changes in the system image default value
of generate_hashtree is currently set to false. Once all the APEXes that
shouldn't have hashtree enable (i.e. non-updatable ones) explicitly set
`generate_hashtree: false` default value will be switched to true.

Test: checked that are no diffs between ${OUT}/system/apex without this
  change and with this change.
Test: m
Bug: 190621617
Change-Id: I5a3144f8f6ea7dee25678af43e2b07a8c85daa9e
Merged-In: I5a3144f8f6ea7dee25678af43e2b07a8c85daa9e
(cherry picked from commit da6dc31389)
2021-06-10 16:54:53 +01:00
Martin Stjernholm
cb3ff1e412 Move logic to disable flattening in unbundled app mode from Soong to
make.

Avoids TARGET_FLATTEN_APEX lying about it, so we can trust it in
scripts, e.g. in art/build/apex/runtests.sh.

Test: banchan com.android.art; art/build/apex/runtests.sh
Bug: 179900989
Change-Id: I5361957c448e61634689968424450420479f6111
2021-05-25 00:28:27 +01:00
Martin Stjernholm
df298b3bed Fix indexing bug when config.Arches() returns an empty array.
This happens when com.android.art.host is not disabled in
prebuilts/build-tools/build-prebuilts.sh.

Test: m nothing
Test: env OUT_DIR=out prebuilts/build-tools/build-prebuilts.sh
  on the aosp-build-tools branch
Bug: 188922057
Change-Id: Ib21d17bb627e4cf7faee320febe65e1bb4566d4e
2021-05-21 21:07:26 +01:00
Paul Duffin
bf4e6ec8a7 Merge "Store dex jar paths in bootclasspath_fragment's apex content info" 2021-05-20 12:46:48 +00:00
Paul Duffin
1a8010a241 Store dex jar paths in bootclasspath_fragment's apex content info
Previously, the DexBootJarPathForContentModule(module) simply called
directly through to the module to retrieve the dex jar path. This
change changes it so the bootclasspath_fragment retrieves the dex
jars from the module and stores them in the info structure for this
method to retrieve directly.

This makes it easier for the bootclasspath_fragment to stop retrieving
hidden API encoded dex jars from the module and perform the encoding
itself.

Bug: 179354495
Test: m com.android.art com.android.ipsec com.android.os.statsd com.android.conscrypt
      - verify that this does not change the contents of the apex files
Change-Id: Ic79dea080f10f4017f1a75d6d1fb5a3bfe04c2ce
2021-05-19 16:17:30 +01:00
Jiyong Park
78349b56a0 Record the actual APEXes that a module is part of.
Consider this case:

apex {
    name: "com.android.foo",
    native_libs: ["foo"],
}

override_apex {
    name: "com.mycompany.android.foo",
    base: "com.android.foo",
}

cc_library {
    name: "foo",
}

There are two APEXes defined: "com.android.foo" and
"com.mycompany.android.foo" which is a copy of "com.android.foo" with
some properties overridden (e.g. signing keys).

The module "foo" is mutated into two variants by the apex mutator: the
platform variant and the apex variant. The former has the variation name
"" and the later has "apex<min_api_ver>" which usually is "apex10000".

Internally, the apex variant has an alias "com.android.foo".

ApexInfo.InApexVariants() returns only "com.android.foo" when called for
the module "foo".

We can see that the information that "foo" is also part of
"com.mycompany.android.foo" is completely lost. This is causing problem
when we compare the apex membership by their "soong module name", not
the "apex name". In the example above, the two modules have different
soone module names, but have the same apex name: "com.android.foo".

To fix that, this CL introduces a new field `InApexes` to the `ApexInfo`
struct. It has the actual name of the APEXes that the module is part of.
With the example above, `InApexes` is ["com.android.foo",
"com.mycompany.android.foo"].

Bug: 180325915
Test: m nothing
Test: m nothing on non-AOSP targets with ag/13740887 applied.

Change-Id: I4e7a7ac5495d2e622ba92a4358ed967e066c6c2e
2021-05-18 16:05:29 +09:00
Jiyong Park
ab50b0795a Rename InApexes -> InApexVariants
.. in preparation for the upcoming change. This change doesn't alter any
behavior.

InApexes is a misleading name. People expects that it has the list of
soong module names of the APEXes that a module is part of. So, for
example, `core-oj` is a part of both `com.android.art` and
`com.google.android.art`. However, in reality, that's not true. The
field has `com.android.art` only. This is because the two APEXes
(android and Google) have the same apex name which is `com.android.art`.
That apex name is used in various places like the `apex_available` and
allows us to keep using the same name regardless of whether the APEX is
overridden or not.

However, this is causing problems in some cases where the exact list of
soong module names is required. The upcoming change will add a new field
to handle the case and the new field actually will get the name
'InApexes'. So, the existing field is renamed to a less misleading name
`InApexVariants`.

Bug: 180325915
Test: m nothing

Change-Id: I0c73361b452eddb812acd5ebef5dcedaab382436
2021-05-18 16:05:14 +09:00
satayev
333a1732b1 Add systemserverclasspath_fragments property to apex.
- All contents of the fragment are added as java_lib dependencies.
- Generated classpaths.proto is added into etc as required.

Bug: 180105615
Test: m nothing
Change-Id: I8e8e8b019c4ca2909182f205a47deffa946de6da
2021-05-17 21:46:08 +01:00
satayev
14e49130bb Move classpaths.proto related info into a separate provider.
The new info struct can be easily shared with systemserverclasspath
fragments.

Bug: 180105615
Test: m nothing
Change-Id: I9986e64fdf19f4168da63c156de3dc9bcafac8d8
2021-05-17 21:46:04 +01:00
Paul Duffin
5606029007 Order apex files by destination path not source path
The destination path, i.e. the path the file will occupy in the APEX is
more stable and less likely to change than the source path, i.e. the
path of the file in the out or source directories. This makes it easier
to diff changes in the copy commands.

Bug: 179354495
Test: m com.android.art com.android.ipsec com.android.os.statsd com.android.conscrypt
      - verify that this does not change the contents of the apex files
      - verify that this does change the order of copy_commands for the above apexes
Change-Id: Ia9ebe8418c59903cd738ee0fb9262b3f32a4ddbf
2021-05-16 22:28:26 +01:00
Paul Duffin
c75bf688cb Merge "Move markPlatformAvailability mutator before apexMutator" 2021-05-13 09:50:56 +00:00
Paul Duffin
9896b3424b Merge "Allow module types to force creation of a default APEX variant" 2021-05-13 08:07:19 +00:00
Ivan Lozano
d7586b6526 Refactor vendor snapshot to use LinkableInterface.
Refactors the vendor snapshot support to use the LinkableInterface
so that support can be extended to Rust. This CL does not add
vendor snapshot support for Rust; that is left for a follow-on CL.

Bug: 184042776
Test: m nothing
Change-Id: Id0c4970ca00053484a52677d182153cbc454c301
2021-05-12 14:01:10 -04:00
Paul Duffin
28bf7ee2ad Move markPlatformAvailability mutator before apexMutator
This will allow the apexMutator to use the NotAvailableForPlatform()
method initialized by the markPlatformAvailability mutator to help
determine whether to create a platform variant or not.

Bug: 187910671
Test: m droid
Change-Id: I745b4428afc68e7ca4cd2f4cbe810b8df1eebdb7
2021-05-12 16:41:35 +01:00
Paul Duffin
b5769c15a3 Allow module types to force creation of a default APEX variant
Bug: 187910671
Test: m droid
Change-Id: I797d4ab60d15b526744fe6e4df1b55c8b75b0310
2021-05-12 16:16:51 +01:00