The build actions will be generated by the top-level apex.
Test: go test ./apex
Test: lunch cf_x86_64_phone-next-userdebug (uses mainline prebuilts)
Test: verified no diff in file_list.txt
Bug: 368337090
Change-Id: I170cf4beec18b54cd950560ea0991bc6f1455d76
Since the deapex actions have been moved to the top-level apex, this
internal module is no longer neessary.
Test: go test ./apex
Test: lunch cf_x86_64_phone-next-userdebug (uses mainline prebuilts)
Test: verified no diff in file_list.txt
Bug: 368337090
Change-Id: I53695e028a36a689fd1b026f5783de7d8567be76
The build ations will be created by the top-level apex.
Details
1. In GenerateAndroidBuildActions, do a graph walk to determine if the
apex has exported deps.
2. If there are exported deps, call the newly introduced `deapex`
function. This registers the build rules and returns a DeapexerInfo
object. This was previously provided by the internal deapexer
dependency.
3. Update `dexpreoptSystemServerJars and `provideApexExportsInfo` to use
the DeapexerInfo object from (2).
A lot of unit tests that relied on the legacy mechanism of deapexing
have been updated.
Test: go test ./apex
Test: lunch cf_x86_64_phone-next-userdebug (uses mainline prebuilts)
Test: verified no diff in file_list.txt
Bug: 368337090
Change-Id: I0edb681beccac4d2a9ceb73f9a506c081a8a96e0
`exported_java_libs` provide information to soong about which .jars in a
prebuilt .apex need to be deapexed. This property is not used today,
since the only use case for deapexing is for dexpreopt, and dexpreopt is
done using bootjars (encapsulated using
`exported_bootclasspath_fragments`) and systemserver jars (encapsulated
using `exported_systemserverclasspath_fragments`)
Test: go test ./apex
Change-Id: I4ceec74aa11d8e32ccfb0bab29a4489cf8b3fc3d
Many TopDown mutators can be easily converted to BottomUp mutators,
which are easier to handle for incremental and partial analysis.
Bug: 367784740
Test: all soong tests pass
Test: no change to build.ninja
Flag: EXEMPT refactor
Change-Id: I82955e844ed0eb6680854678c0744ac5398eb7ba
This will eventually allow us to build devices by skipping `katiBuild`
and moving straight to `katiPackaging`.
Implementation details
- Replace ctx.PackageFile with ctx.InstallFile. These makes these rules
available to both make and soong packaging systems. This will be done
inside d.dexpreopt function
- Add InstallDepNeeded on the dependency tags in this chain
(apex --> sscp_fragment --> (java_library|java_sdk_library)). This is
necessary for the soong packaging system.
- Remove `LOCAL_SOONG_INSTALLED_MODULE=""`. A non-empty string ensures
that kati honors the installation rules generated by soong.
Bug: 355509400
Test: lunch aosp_cf_x86_64_phone-trunk_staging-userdebug (mainline source)
Test: lunch cf_x86_64_only_phone-ap3a-userdebug (mainline prebuilts)
(no diff in system_intermediates/file_list.txt in both cases)
Change-Id: I190a919e9480e7d9981a6b9b96a50d9c33ceea02
If a prebuilt APEX is compressed, it's installed without decompression
even when PRODUCT_COMPRESSED_APEX is false.
This change decompresses the prebuilt/compressed APEX.
Bug: 356533488
Test: OVERRIDE_PRODUCT_COMPRESSED_APEX=false m
Change-Id: Ifdde074e383837df92d172bc483bfca9a97cb6f8
https://r.android.com/2901309 added support for handling installation
rules for multiple prebuilts. One of the implementation details there
was to change LOCAL_MODULE name to the source apex name in the generated
.mk files.
This causes issues in partner builds where the google singed apex
com.google.android.foo gets renamed to com.android.foo in
out/soong/Android-$PRODUCT.mk. If the partner builds does not include
the aosp apex in its PRODUCT_PACAKGES, then the google signed apex gets
elided from packaging.
This CL undoes that change. After this CL, the name of the module as it
appears in Android.bp will match the LOCAL_MODULE name emitted for
product packaging.
Test: in build/soong, go test ./apex
Bug: 355682304
Change-Id: Icadbb75609174cff5326f52baf7d53e901f9f1c0
In the context of incremental soong, the output files
inter-module-communication will be through OutputFilesProvider.
The OutputFileProducer interface will be deprecated.
Test: CI
Bug: 339477385
Change-Id: I5cb79a2e39f7fdd791762bcde5a5bfb325985274
Soong generates AndroidMk modules and Make installs the required
dexpreopt files for APEX bundles. This dependency is not tracked by
the soong and missing from the soong filesystem.
Call PackageFile for the dexpreopt files of APEX bundles to install
the files in the soong-built system image.
Bug: 346439786
Test: lunch aosp_cf_x86_64_phone-trunk_staging-userdebug \
&& m aosp_cf_system_x86_64
Change-Id: I6af4afe5b3183c89bf687ac779007b87e1d7e948
This is a followup to https://r.android.com/3073624 that moved
validation of bcp jars to the top-level prebuilt apex. It is possible
that there can be multiple prebuilt variants of a mainline module and
each prebuilt might have a skew in bcp jars. This CL limits the check to
only the selected apex prebuilt.
Test: go test ./apex
Bug: 328578801
Change-Id: If225a1af6004b6584b86ec442f99672f0d1f8314
During trunk stable development, it is possible that a new boot jar is
added to the source apex, but not available in prebuilt apex yet.
Thefore we need to conditionally check the `contents` of the respective
bootclasspath_fragment modules. https://r.android.com/2822393 did this
using isActiveModule. This function relies on soong config namespaces to
"hide" the source bootclasspath_fragment module when
<module>.source_build is false.
Soong config namespaces for mainline source vs prebuilt selection is no
longer in use, so this CL replaces `isActiveModule`. The CL cleaves the
implementation
1. For source builds, the check will run in the context of the
bootclasspath_fragment module. `disableSourceApexVariant` will be
used to skip the check if prebuilts are selected.
2. For prebuilt builds, the prebuilt bootclasspath_fragment module sets
a provider, and the top-level apex will do the check.
(2) is necessary to handle the edge case of multiple versioned mainline
prebuilts and possible skew in apex boot jars across them. In case of
prebuilt builds, the unselected mainline prebuilts will have
HideFromMake called on them, so the check will only run on the apex that
is actually used during the build.
Bug: 328578801
Test: go test ./apex
Change-Id: I6eec603397eea926f3a481b79ca938245064d809
Merged-In: I6eec603397eea926f3a481b79ca938245064d809
android.ModuleBase already calls aconfigUpdateAndroidBuildActions()
that is the same with CollectDependencyAconfigFiles(). Remove the
CollectDependencyAconfigFiles() to avoid duplication with
aconfigUpdateAndroidBuildActions().
To make the aconfig information available in
GenerateAndroidBuildActions() of all modules, call
aconfigUpdateAndroidBuildActions() before calling
GenerateAndroidBuildActions() of each module.
Also, we don't need SetAconfigFileMkEntries(), which is a duplicate
of aconfigUpdateAndroidMkData()
Bug: 335363964
Test: diff `adb shell printflags` before and after the change.
Change-Id: I52808e442e9fed7db1eae7b7c5ed0b1c5ba74f5d
This is a followup to aosp/2999198 and adds information about apps.
Each app will have an entry in this file with the following properties
- Name, mandatory
- Is_prebuilt, mandatory
- Prebuilt_info_file_path, optional
Implementation details
- Move prebuiltInfoProvider out of build/soong/apex to
build/soong/android. This allows build/soong/java to use it.
- Introduce a new `prebuilt_info` prop to `android_app_set` and
`android_app_import`
- All app module types will set a prebuiltInfoProvider in
GenerateAndroidBuildActions, including the source app module types
Test: m nothing --no-skip-soong-tests
Test: m out/soong/prebuilt_info.json
Test: ls -l out/soong/prebuilt_info.json --human-readable
-rw------- 1 spandandas primarygroup 317K Mar 11 23:46 out/soong/prebuilt_info.json
Test: #modified trunk_staging.locally to select prebuilts of some
mainline apps. Spot-checked that `is_prebuilt` and
`prebuilt_info_file_path` get populated appropriately
Bug: 327480225
Change-Id: I5078e0ec26c9568194550909962b90111a5223f7
out/soong/prebuilt_info.json will contain information about whether
source or prebuilt of an apex was used. If prebuilt is used, it will
print the path to its prebuilt_info file. This file will be
used to pick a matching set of MTS test prebuilts.
Each apex will have an entry in this file with the following properties
- Name, mandatory
- Is_prebuilt, mandatory
- Prebuilt_info_file_path, optional
Implementation details
- Introduce a new `prebuilt_info` prop to `apex_set` and `prebuilt_apex`
- All apex module types will set a prebuiltInfoProvider in
GenerateAndroidBuildActions, including the source apex module types
- Create a `apex_prebuiltinfo_singleton` that visits all apex modules.
It uses `IsHideFromMake` to filter out the unselected variants of a
specific apex. This new singleton will create prebuilt_info.json
- Dist prebuilt_info.json for droidcore
Test: m nothing --no-skip-soong-tests
Test: m out/soong/prebuilt_info.json
Test: ls -l out/soong/prebuilt_info.json --human-readable
-rw------- 1 spandandas primarygroup 25K Mar 11 23:46 out/soong/prebuilt_info.json
Test: #modified trunk_staging.locally to select prebuilts of some
mainline modules. Spot-checked that `is_prebuilt` and
`prebuilt_info_file_path` get populated appropriately
Bug: 327480225
Change-Id: I65c73010142b034ad1d2b3d05ef493be034e8d74
There are a couple of instances in apex/ and java/ that rely on the naming
convention that the jars exported by prebuilt apexes follow the name of
the java_sdk_library_import, minus the prebuilt_ prefix. With muliple
module sdk prebuilts in trunk stable, this naming convention might not
be valid. Use `source_module_name` instead.
```
prebuilt_sscp_fragment {name: "", contents: ["service-foo.v2"]}
java_import {name: "service-foo.v2", source_module_name: "service-foo"},
```
We should use service-foo and not service-foo.v2 because
1. The prebuilt apex contains service-foo.dex
2. PRODUCT_*JARS will contain service-foo and not service-foo.v2
For clarity, this CL does not drop the current mechanism where prebuilt bcp
fragments create a dependency edge to prebuilt java or java_sdk_library
imports. There is still some discussion around whether we can remove
that completely (b/320711431). If we were to do that, then the
Android.bp files will become
```
prebuilt_sscp_fragment {name: "", contents: ["service-foo"]}
```
Bug: 322175508
Test: Added a unit test
Test: In internal, lunch cf_x86_64_only_phone-next-userdebug && m
nothing # .ninja files identical
Test: In internal, created a parallel set of v2 prebuilts of
java_sdk_library_import and prebuilt_bcp_fragments for adservices && m
nothing # build passes
Change-Id: Ia899d75e826fa1a559368d706eaa65835f748d40
For prebuilts, the dexpreopt rules of system server jars are now
generated from the context of the top-level prebuilt apex and not in the
context of the shim java_import modules. Since
`dex_preopt.profile_guided` property is defined in java_import, this
needs to be bubbled up to the top-level apex. This will be done using
deapxerInfo. If profile_guided of a transitive java_import is true, the deapexed .prof file will be
set as dexreopter.inputProfilePathOnHost before invoking
dexpreopter.dexpreopt. This ensures that only that java_import undergoes
profile guided dexpreopt, and not every other transitive java_import
Test: go test ./apex -run TestPrebuiltStandaloneSystemserverclasspathFragmentContents
Test: lunch cf_x86_64_only_phone-next-userdebug && m
$ANDROID_PRODUCT_OUT/system/framework/oat/x86_64/apex@com.android.art@javalib@service-art.jar@classes.odex
Test: du -sh
$ANDROID_PRODUCT_OUT/system/framework/oat/x86_64/apex@com.android.art@javalib@service-art.jar@classes.odex
24K
Bug: 308790457
Change-Id: Ibf46ecb400b3f126b243fc8d27b08d9a1aa4cc97
For prebuilts, java.dexpreopt gets called twice
1. in the context of the top-level prebuilt apex
2. in the context of the java_library shim
Only the artifacts of (1) get installed. The artifacts generated by 2)
are unused. However the args used to generate the artifacts of (1) are
incorrect. It uses moduleName(ctx) to do special-handling of apex system
server jars. This special-handling does not happen in (1), so although
dexpreopt artficats get generated, they get generated with the wrong
args.
To fix this, add an additional parameter in java.dexpreopt to pass the
libraryName explicitly.
Details
- Delete moduleName function. This was used to determine the java
library name, which is no longer safe
- Add a libraryName parameter to java.dexpreopt
- Most module types will use j.Name() as libraryName
- prebuilt_apex and apex_set will iterate over its `contents` and use
each element as libraryName when invoking java.dexpreopt
With the correct args passed from (1), we can drop j.dexpreopt from (2)
completely. Dropping (2) also breaks profile guided dexpreopt tests.
These currently operate on (2). They will be moved to (1) in the next CL
of this stack.
Test: presubmits
Test: lunch cf_x86_64_auto-trunk_staging-userdebug && m nothing (this
was a postsubmit failure with aosp/2923733)
Test: art_standalone_dexpreopt_tests on next https://android-build.corp.google.com/builds/abtd/run/L86000030001579256
Test: art-gtest on git_master-art-host https://android-build.corp.google.com/builds/abtd/run/L07800030001550262
Bug: 308790457
Bug: 322255144
Change-Id: I8eb604c82f1fa5289d3cd1a20084d56e4d7485e3
This provider contains information about the contents of an apex and is
set during the post deps phase of top-level source and prebuilt apexes.
java.dexpreoptDisabled needs to distinguish between platform and apex
variants of a library to prevent creating duplicate ninja rules. It does
so by looking at ApexInfoProvider. This provider gets set on the ctx of
the java_library being dexpreopted.
In case of prebuilts, these rules are being moved to the top-level
prebuilt apexes. Move ApexInfoBundleProvider from apex to android so
that java.dexpreoptDisabled can consume this provider without running
into a golang circular dependency. (java.dexpreoptDisabled will use this
provider in the next CL of this stack).
Test: go build ./android ./apex
Change-Id: I665abd9d0591eeb17b273cc822beb20690e6af09
(This was missed in aosp/2876756, which copied only the bcp jars)
This hardcoded location is used by dex2oat to compile the dexpreopt
artifacts. The copy rules are currently generated by java_(sdk)_library
for source builds, and their prebuilt counterparts in prebuilt builds.
After this change, the copy rule will be bifurcated between source and
prebuilt builds
1. For source builds, it will come from java_(sdk)_library
2. For prebuilt builds, it will come from the top-level prebuilt apex.
Since there can be multiple prebuilt apexes in trunk stable,
HideFromMake will be used to determine which deapexed jar to copy.
The bifurfaction is expected to be temporary. It is needed for now since
the `apex_contributions` which will be used for source vs prebuilt
selection have not been populated completely.
Test: Added a unit test
Test: Presubmits
Test: git_master-art-host:
art-gtest (https://android-build.corp.google.com/builds/abtd/run/L40800030001459791)
Test: git_main:art_standalone_tests
(https://android-build.corp.google.com/builds/abtd/run/L09000030001463855)
Bug: 308790457
Change-Id: I3105d3b3a7e5c41cb601d07806f4ea483a61b50a
Every module belonging to a single mainline module family will be
hidden from make, except the one which has been flagged using
apex_contributions
Details
- Introduce a new `source_apex_name` property to prebuilt_apex and
override_apex. This property will be used to identify the source
equivalent of a prebuilt soong apex module.
- Create an N-ary tree from source to prebuilt(s). The tree wil be
rooted at the source module.
- In a subsequent mutator, visit every node in the tree(s). Query
apex_contributions and store the handle of the node which is "active"
(if any)
- In the same mutator, do another pass over the tree. Invoke
`HideFromMake` on every node which is not "active". The two-pass
approach is needed PrebuiltSelectionInfoProvider does not know about
the inter source-prebuilt dependency, this dependency can only be
known by doing a graph walk of the N-ary tree.
Some tangential implementation details
- Each prebuilt apex has an internal deapxer module that is responsible
for generating the deapex ninja rules. The name of this internal
module uses the BaseModuleName (without the prebuilt_ prefix). Since
we can have multiple prebuilt soong modules in trunk stable, change
this to follow the name of the prebuilt module in order to avoid name
collisions. Update existing unit tests accordingly
Bug: 316179314
Test: go test ./apex -run TestInstallationRulesForMultipleApexPrebuilts
Test: m nothing --no-skip-soong-tests
Test: presubmits
Change-Id: I58aa99d5e6a9859954614e6db9a8e9e2e581642d
hiddenapi and dexpreopt require boot and system server jars from apexes.
When building with prebuilts, this comes via
java_import/java_sdk_library_import, which acts as a hook for
prebuilt_apex/apex_set. If we have multiple apexes in the tree, this
hook becomes 1:many. This CL prepares dex_bootjars to select the right
deapexerd .jar files when mutliple prebuilts exist.
Implementation details
- Update prebuilt module types (prebuilt_apex/apex_set) and source
apexes to set a map of
library name to dex jar path on host.
- dex_bootjars will access the path of the .dex jar on host via the
provider. These then
copied/installed to the right locations.
This CL does not drop the old mechanism to get the dex file (i.e. by
creating a dep on java_library). Once all mainline
modules have been flagged using apex_contributions, the old mechanism
will be dropped
Bug: 308790457
Test: git_master-art-host:art-gtest https://android-build.corp.google.com/builds/abtd/run/L21500030000926533
Test: git_main:art_standalone_dexpreopt_tests https://android-build.corp.google.com/builds/abtd/run/L99000030000891212
Test: Added a unit test that checks that the right .jar is selected
when multiple prebuilts exists
Change-Id: I6ef94135b9303a35135810930af4b641df13a583
Generating boot image requires a .prof file provided by the ART apex.
When building with prebuilts, this comes via the
prebuilt_bootclasspath_fragment module, which acts as a shim for
prebuilt_apex/apex_set. If we have multiple prebuilt apexes in the tree,
this shim becomes 1:many. This CL prepares dex_bootjars to select the
right .prof file when multiple prebuilts exist.
Implementation details
- Update deps mutator of dex_bootjars to create a dep on
all_apex_contributions. The latter contains information about which
apex is selected in a specific release configuration. dex_bootjars
will create a dependency on the selected apex in a postdeps phase
mutator.
- All apex module types (apex, prebuilt_apex and apex_set) will set a
provider that contains info about the location of the .prof file on
host
- dex_bootjars will access the provider of the selected apex to get the
location of the .prof file
This CL does not drop the old mechanism to get the .prof file (i.e. by
creating a dep on {prebuilt_}bootclasspath_fragment). Once all mainline
modules have been flagged using apex_contributions, the old mechanism
will be dropped
Bug: 308790457
Test: Added a unit test that checks that the right .prof is selected
when multiple prebuilts exists
Change-Id: I40fdb21416c46bed32f6ff187ce5153711ec2c69
FindDeapexerProviderForModule raises an exception if multiple apexes in
the tree has an export dep on the java module. In prepartation to
support multiple prebuilts, move this error check out of
FindDeapexerProviderForModule and into rdeps. i.e. raise an exception
only if an rdep calls DexJarBuildPath
- This should be a no-op for now.
- In the short-term future, a java import module will be allowed to have
multiple deapexers. An error will be raised if anyone actually tries
to depend on the dexjar
- In the long-term future, this function will be removed. All processing
will be done at the prebuilt apex level and not at the prebuilt java
library level
Since this check now happens in the moduleCtx of rdeps, add some
additional props to unit tests to ensure that it does not exit early on
unrelated validation checks (e.g. hidden_api prop is not set)
Test: go test ./apex ./java
Bug: 308790457
Change-Id: I3323d993c1ea8f43305834cae8e65b6fe41dfefd
dexpreopt of apex system server from prebuilts involves three soong
modules
1. prebuilt_apex / apex_set
2. an internal deapexer module created by the prebuilt apex
3. java_import/java_sdk_library
(3) acts as a shim for the deapexer to set the dexjar extracted from the
prebuilt apex. This methodolody requires a 1:1 correspondence across the
three modules
This breaks down when we have multiple versions of the same prebuilt
apex in the tree. In preparation for this, move the dexpreopt
processing from (3) to (1). Each prebuilt_apex will create the necessary
rules for dexpreopting the jars deapexed from itself. In the future,
apex_contributions will be used to pick which service-foo.{odex|.vdex} to
install depending on which prebuilt apex is selected.
Implementation details
- Embed dexpreopter in prebuiltApex structs so that this module type can
register the dexpreopt rules. Since a single apex can have multiple
system server jars, this also requires creating an additional scope in
dexpreopt.go to prevent name collisions
- Add the dexpreopt modules as required in initApexFilesForAndroidMk
- Add the depreopt modules to androidMk in AndroidMkEntries. Drop the
equivalent from java_import and java_sdk_library_import
Bug: 308790457
Test: existing soong unit tests
Test: lunch cf_x86_64_phone-next-userdebug && m out/target/product/vsoc_x86_64/system/apex/com.google.android.adservices.apex
Test: Verified that the above command installs
/out/target/product/vsoc_x86_64/system/framework/oat/x86_64/apex@com.android.adservices@javalib@service-adservices.jar@classes.{odex|vdex} and the equivalent files of service-sdksandbox
Test: presubmits
Change-Id: I01cea8956d2857fb864b415e73d3d2686d069b5e
Convert all of the callers of SetProvider to use the type-safe
android.SetProvider API.
Bug: 316410648
Test: builds
Change-Id: If58f4b5355264ddab2045bc3591a4eac19cd58fc
Installed files should only depend on other installed files, change
the deps arguments of the ctx.Install* methods from Paths to
InstallPaths.
Bug: 311428265
Test: builds
Change-Id: I1ebef60a943bdbe907744cc43aa985371ac56d32
Merged-In: I1ebef60a943bdbe907744cc43aa985371ac56d32
Instead of listing all apexes in the source tree, now each apex emits
its own fragment for apexkeys.txt, which is pointed by
LOCAL_APEX_KEYS_FILE. Makefile collects apexkeys.txt from installed apex
files. This is to avoid listing unrelated apexes (not installed,
testdata, unexported namespaces, etc.)
Bug: 304914238
Test: m apexkeys.txt
Test: m blueprint-tests
Change-Id: Iefbe6e486cb418955584ad1a282455307e90be95
Since we no longer support "zip" APEX, we don't need "host" support.
For example, we don't need go/python binary support.
Bug: 279835185
Test: m
Change-Id: I6e8d2b205e42662f31866dc9ac7507524effd144
We want to make FULL_SYSTEMIMAGE_DEPS more accurate so that we can
remove the need for `m installclean`.
Currently, the compat symlinks are installed by adding them as
dependencies of other installed files. Doing this doesn't end up
propagaing the compat symlinks all the way to FULL_SYSTEMIMAGE_DEPS.
Add them to LOCAL_SOONG_INSTALL_SYMLINKS, which will cause them to
be propagated to FULL_SYSTEMIMAGE_DEPS.
Bug: 205632228
Test: Full build with a change to delete files not in FULL_SYSTEMIMAGE_DEPS from the staging dir
Change-Id: Idc7fc0a0dbf7c8ac3f0628c16413cd1df69a7821
Revert submission 2718295-colefaust_track_apex_compat_symlinks
Reason for revert: To validate if this change is causing the build breakage.
Reverted changes: /q/submissionid:2718295-colefaust_track_apex_compat_symlinks
Change-Id: I1896530219cf984efdf9ca7f4b14d8784367f242
We want to make FULL_SYSTEMIMAGE_DEPS more accurate so that we can
remove the need for `m installclean`.
Currently, the compat symlinks are installed by adding them as
dependencies of other installed files. Doing this doesn't end up
propagaing the compat symlinks all the way to FULL_SYSTEMIMAGE_DEPS.
Add them to LOCAL_SOONG_INSTALL_SYMLINKS, which will cause them to
be propagated to FULL_SYSTEMIMAGE_DEPS.
Bug: 205632228
Test: Full build with a change to delete files not in FULL_SYSTEMIMAGE_DEPS from the staging dir
Change-Id: I0600ccf8d863d5e29e044c82293ece730fc98d43
We don't need this anymore because we are going to compile ART jars and
framework jars together.
Bug: 280776428
Test: m
Change-Id: I070157530449a1bb5779e25984c367df3dde7b36
