Previously, when building from prebuilts boot no rules were created to
produce the boot image files for the host, i.e. the OS on which the
build was running. That caused problems with checkbuilds. No rules were
produced as there was no host variant of a prebuilt apex to provide
them.
This change restructures the code to allow the prebuilt bootclasspath
fragment to build the host variants of the files from the dex files
provided by the prebuilt APEX. The generated files will not be the same
as they would be if built from source as there is no boot image profile
to use but it should be sufficient to satisfy the checkbuild target and
allow for host side testing.
Bug: 192575099
Test: m SOONG_CONFIG_art_module_source_build=false droid dist checkbuild
Change-Id: I6af00f19bb71aa18dd462f5eac6aa38e3e721023
Previously, boot image generation used GetCachedGlobalSoongConfig(ctx)
in order to get access to the GlobalSoongConfig. That required that
some other part of the code had called GetGlobalSoongConfig(ctx) to
initialize the cached value. That was left over from when the boot
image generation was done in a singleton which could not call
GetGlobalSoongConfig(ctx) directly. That is no longer true.
This change switches the uses of GetCachedGlobalSoongConfig(ctx) to
GetGlobalSoongConfig(ctx) and removes the now unnecessary call to
GetGlobalSoongConfig(ctx) from outside the functions.
Bug: 192575099
Test: m nothing
Change-Id: I34b7b1526d072d8b09fda7caa96e381695888e16
Previously, the boot zip file, containing all the boot image files for
all the supported architectures, was only created from source. It was
not created when building from a prebuilt_bootclasspath_fragment. That
lead to build failures when building from ART prebuilts.
This change pulls the boot zip file creation out so that it can be done
for both source and prebuilt bootclasspath_fragment modules as well as
for the platform_bootclasspath module.
Bug: 192575099
Test: m out/target/product/generic_arm64/boot.zip
m SOONG_CONFIG_art_module_source_build=false out/target/product/generic_arm64/boot.zip
- Compare the output of the first command from before the change
with the output from them both after and confirm that when the
ART prebuilts are up to date with the source that there are no
differences.
Change-Id: Ie7dd5e2ca4a865d06fd9ebf87320cf68c4d05bc3
Sometimes the stem property is set to change both the installed file
name and the name use to filter the configured module list, e.g. when
adding a test library to replace the standard library, e.g. test_foo
instead of foo.
Sometimes it is used just to change the installed file name.
This change uses both to filter the updatable boot jars and not just
the stem.
Bug: 180105615
Test: m nothing
Change-Id: I6c459fc3597b1e4f062bc9a4e52843305b538c5f
Previously, the stub dex jars for each HiddenAPIScope was created by
merging the stub dex jars provided by each module for that scope. Then
the widest stub dex jars were chosen. So, if module A provided public,
system and test stub dex jars and module B provided only public then
the stub dex jars for each scope would be:
* public -> A,B
* system -> A
* test -> A
So, the widest API scope for which there are stub dex jars is "test"
and so the widest stub dex jars would just come from module A and not
module B. So, when "hiddenapi list" is run for module C which depends
on modules A and B it only gets given stub dex jars for module A which
means that it cannot resolve all the types that C may use which can
lead to incorrect flags being generated.
This change does not merge the stub dex jars from each module together
and instead keeps them separate by module. The widest stub dex jars
list is constructed by asking each module in turn for their widest stub
dex jars. e.g. Given the above example we would have:
Module A:
* public
* system
* test <- widest
Module B:
* public <- widest
So, the widest stub dex jars will be A's test and B's public stub dex
jars.
Bug: 179354495
Test: m out/soong/hiddenapi-flags.csv
- make sure that this does not change the file.
Change-Id: Ib137825ebffe94b2bf220732bae6077f7b7ac6db
Previously, hidden API processing could only be done by those
bootclasspath_fragment modules that either did not depend on any other
fragments (e.g. art-bootclasspath-fragment) or only depended on APIs
provided by other fragments (e.g. i18n-bootclasspath-fragment). That
meant that modules like com.android.os.statsd-bootclasspath-fragment
that depended on APIs provided by parts of the platform which are not
yet part of another bootclasspath_fragment could not perform hidden
API processing.
This change adds support for a bootclasspath_fragment to specify the
additional stubs needed to perform hidden API processing. It adds a new
additional_stubs property that can be used to specify the additional
stub libraries.
Most bootclasspath_fragments that need to use the property will need
access to the APIs provided by the android-non-updatable.* libraries.
Rather than have each fragment explicitly specify the correct module
for each scope it treats "android-non-updatable" as if it was a
java_sdk_library that can provide different jars for each scope.
Soong will handle mapping that to the correct android-non-updatable.*
module.
Bug: 179354495
Test: m out/soong/hiddenapi/hiddenapi-flags.csv \
out/soong/hiddenapi/hiddenapi-index.csv \
out/soong/hiddenapi/hiddenapi-stub-flags.txt \
out/soong/hiddenapi/hiddenapi-unsupported.csv
- make sure that this change does not change the contents.
m TARGET_BUILD_APPS=Calendar nothing
Change-Id: Ia8b79830ed0e6d42100de03d76b0c51b7f6c8ade
Previously, the hidden API processing used SdkKind to identify the API
scopes, e.g. public, system, etc. that are of interest for hidden API
processing. Unfortunately, there is a mismatch between the SdkKind and
what hidden API processing needs. e.g. SdkKind includes values that are
not used by hidden API processing and hidden API processing needs
additional API scope specific information not provided by SdkKind. The
apiScope struct used in sdk_library.go is also not a suitable
representation for similar reasons.
This change adds the HiddenAPIScope (following a similar approach as
apiScope) that rectifies that and uses it as a replacement for SdkKind
in most parts of the hidden API processing. The SdkKind is still used
for retrieving information from java_sdk_library[_import] modules.
Follow up changes will extend the HiddenAPIScope with more information.
Bug: 179354495
Test: m out/soong/hiddenapi/hiddenapi-flags.csv
- make sure that this change has no effect on the generated flags.
Change-Id: I97968f58535121652852b8d25217aa288afd2bfd
Adds support for printing a PropertySet that has a property whose value
is an array of structs.
Bug: 179354495
Test: m nothing
m conscrypt-module-sdk
- check generated Android.bp file
Change-Id: I71be04188465610bcbea4d3c9a5e8204171a1eeb
Any apex classpath fragment that doesn't generate its own classpaths
proto, must still propagate it's boot jars for the platform classpath
fragment to include for complete CLASSPATH variables on device.
Bug: 191127295
Test: atest CtsClasspathsTestCases derive_classpath_test
Change-Id: I93687f69006741fcd66eb6e04891a4b4bbcc3b47
Uses the apex relative path to the file as the identifier that is used
to obtain the path to the corresponding file extracted from the apex.
That is instead of a special constructed string id.
Bug: 177892522
Test: m nothing
Change-Id: I5dc77c8fb272bac289b8891d1eac801e541af1f5
Removes the special case in createDeapexerModuleIfNeeded for handling
java libraries and just get the Import and SdkLibraryImport module
types to implement RequiredFilesFromPrebuiltApex instead.
Bug: 177892522
Test: m nothing
Change-Id: I5cc341b5b4168b8eb196f72273a00d498de6856f
Previously, the prebuilt art-bootclasspath-fragment did not provide any
boot image files. That meant it was impossible to build any module that
requires access to those files from prebuilts, e.g. any module that
needs to be dexpreopt-ed.
This change enables that module to retrieve those files from the
prebuilt_apex.
Bug: 177892522
Bug: 189298093
Test: m nothing
m droid SOONG_CONFIG_art_module_source_build=false SKIP_BOOT_JARS_CHECK=true
- the previous command does not work but this change does fix one
of the issues reported.
Change-Id: I1d4d9545172d79282918130df6b9aa55471bffc1
Previously, the boot dex jars were only copied to the predefined
locations used by the build from the source art-bootclasspath-fragment
if it was preferred, otherwise no files were copied. That caused build
failures when attempting to build with ART prebuilts. This change
copies the files from prebuilts too.
Bug: 177892522
Bug: 189298093
Test: m nothing
m droid SOONG_CONFIG_art_module_source_build=false SKIP_BOOT_JARS_CHECK=true
- the previous command does not work but this change does fix one
of the issues reported.
Change-Id: I35b37355170546daf6ecac2134d1ca9a73d0e3bc
Previously, the rules to perform hidden API encoding were generated
separately to the rules to perform hidden API flag generation. This
change combines them within the (renamed) produceHiddenAPIOutput()
method and makes the paths to the encoded dex files an output of the
generateHiddenAPIBuildActions method alongside the paths to the
generated flag files.
As encoded dex jars are now an output of the produceHiddenAPIOutput()
method which is implemented for both prebuilts and source
bootclasspath_fragment modules that necessitated the prebuilt also
providing paths to encoded dex files. That in turn required updates to
some of the tests to provide dex files from prebuilt_apex modules.
Similarly, as the produceHiddenAPIOutput() method may not be called
for some bootclasspath_fragment modules as they do not yet provide all
the information needed to perform hidden API encoding then it is
necessary to extract the encoded dex files produced by the modules
themselves. That also required a few changes to tests that did not
previously provide dex files.
Bug: 177892522
Test: m com.android.art
- check that this change does not change the contents of the apex
file, i.e. is byte-for-byte identical.
Change-Id: I60996a34d06ed1c87ed244ab3509621999ad86ec
Add apex dependencies.
Bug: 188004865
Test: 1. aidegen frameworks/base/apex/appsearch frameworks/base/apex/appsearch/testing frameworks/base/services/tests/servicestests frameworks/base/core/tests/coretests cts/tests/appsearch external/icing vendor/google_testing/integration/tests/scenarios
2. The dependency graph shows it is workable to provide the result to let IDE's [Project structure] to include the service-appsearch as source directory.
Change-Id: Ifd1548f0a86c1c73f8279e8ecea67756aeb1281e
Previously, an []android.Module was converted to an []hiddenAPIModule
that was then used to retrieve boot dex jars. That was ok when
obtaining the dex jar files from source modules for
bootclasspath_fragment but does not work well for other use cases as
it would require doing that conversion in multiple places.
This change pushes the use of hiddenAPIModule down to the methods that
retrieve information from it which makes the methods more flexible and
easier to reuse.
Bug: 177892522
Test: m nothing
Change-Id: Ib84aaf03d8f5a63b48232036fe4589646fc23352
Previously, copyBootJarsToPredefinedLocations relied on all its
parameters having the same length and the same order. That made it
quite fragile as changes to one of the parameters without corresponding
changes to the other would cause failures. It also combined the
retrieval of the boot dex jars from the modules, handling of missing
boot dex jar files and the generation of the rules to copy the files.
This change separates the retrieval of boot dex jars and handling of
missing files from the copying of those files while at the same time
making the function less fragile by replacing the three ordered
parameters with two maps that shared common keys.
Bug: 179354495
Test: m nothing
Change-Id: Idbcd24a7e8af89f7895a20aeddc58502dcbaad03
This will export some hidden api related types and fields so they can
be used from outside the java package. This is needed to allow a follow
up change to move the TestPlatformBootclasspath_Fragments from the java
to the apex package.
Bug: 179354495
Test: m nothing
Change-Id: Ib69eea9d79cc83b8e3fc29919a29f071e1ec17b5
As long as we have boot jars defined in make, we should try to keep
global.UpdatableBootJars as a source of truth when it comes to classpath
configuration. However, for testing purposes there is a need to add
java libraries to bootclasspath that aren't present in
PRODUCT_UPDATABLE_BOOT_JARS variable.
Ideally, we would unconditionally add all contents to the config for
test apexes, however, I don't fully understand apex merging to support
it.
Bug: 180105615
Test: atest sdkextensions_classpaths_e2e_tests
Change-Id: I552f93289c27b3713e75dcda1faed909b0bfcef2
* changes:
Treat java libraries in classpath fragments as directly in apex
Make CopyDirectlyInAnyApex match the documentation
Remove unused cc.copyDirectlyInAnyApexDependencyTag
Coverage is applied to java libraries that are directly in an apex.
Mark java libraries that are in an apex through a bootclasspath_fragment
or a systemserverclasspath_fragment as directly in the apex by
implementing CopyDirectlyInAnyApexTag on the dependency tags used for
their contents.
Bug: 183759446
Test: TestApexJavaCoverage
Change-Id: I0116f5f415083b5194000988cb257454ef115200
For testing purposes, a boot jar may be provided by a test java_library
that has a different content name, but sets "stem" property to match
the original java_library.
Given that Stem() returns either the property value or module name,
it is safe to replace all content names by their stems.
Bug: 180105615
Test: atest CtsClasspathsTestCases sdkextensions_e2e_tests
Merged-In: Ic519ffa0c5b616abddf15b41c934421dfac2e78a
Change-Id: Ic519ffa0c5b616abddf15b41c934421dfac2e78a
Previously, due to legacy reasons, the property validation did not
require a contents property and allowed the image_name to be either
"art" or "boot". Those reasons no longer apply and so this change
requires a contents property and only allows the image_name to be set
to "art" if specified.
Bug: 177892522
Test: m nothing
Change-Id: I8855d6e5365ef0b55490e90e7b6c0081cf070ee5
Previously, the generation of the rules to create the boot image files
was separate from the code to provide those files to the APEX which
meant that it was possible for the APEX to try and use files that had
no rules to create them.
This changes avoids that by only exporting the files once the rules
have been created.
This necessitated a few changes to tests that were relying on the
previous behavior. Including removing completely the test that used an
image_name: "boot" as that is no longer a valid configuration name as
its functionality has been replaced by platform_bootclasspath.
A follow up change will make the validation of the properties of
bootclasspath_fragment stricter to prevent "boot" being used as the
image_name.
Bug: 177892522
Bug: 188680624
Test: lunch qemu_trusty_arm64-userdebug
m droid dist
- verify that before this change it fails and after this change
it works.
m com.android.art
- verify that this change does not change the APEX contents with
either qemu_trusty_arm64-userdebug or aosp_arm64-userdebug
Change-Id: I0497a151eb0731cbe6a1a7e7bbbb1e4dda75898f
Previously, a bootclasspath_fragment that depended on classes provided
by another bootclasspath_fragment did not support hidden API processing
as it would not supply information about those dependencies.
This change adds support for that as follows. Each fragment:
1. Exports the transitive sets of stub dex jars for each of the public,
system, test and core_platform APIs (where relevant).
2. Adds dependencies onto its dependent fragments.
3. Retrieves the API stubs dex jars from its dependent fragments and
passes them to the "hiddenapi list" tool which will use them to
resolve dependencies but will not output them to the generated
flags.
Once the flags are generated the existing encoding functionality
encodes the flags into the dex files of the bootclasspath_fragment's
content modules which are then packaged into the apex.
Bug: 179354495
Test: m com.android.sdkext
- verify that this does not change the contents of the apex files
Change-Id: I3e82a6dbb437f1e417e5d7e25aeb212e378603d0
Previously, the bootclasspathApiInfo was only used for tests and follow
up changes will need to provide the stub dex jars. This change moves
the stubJarsByKind into HiddenAPIInfo and removes bootclasspathApiInfo
and the corresponding provider.
Bug: 179354495
Test: m nothing
Change-Id: I5459c56de561c053ed671dc9d5cb3ee4820c0ee8
Change 70cfdff3da changed the hidden API
flags in com.android.i18n as it stopped the i18n-bootclasspath-fragment
from making the hidden API flag files available for use by
platform-bootclasspath.
This change fixes that by exporting the flag files even if hidden API
flag generation is skipped.
Bug: 179354495
Test: m com.android.i18 out/soong/hiddenapi/hiddenapi-flags.csv
- make sure that the flags in
packages/modules/RuntimeI18n/apex/hiddenapi/hiddenapi-max-target-o-low-priority.txt
are reflected in the core-icu4j dex files in the apex.
Change-Id: I9b5c7c74bd996ab447bc0e0452da5fd49191a35d
This reflects that it has expanded from its initial purpose to include
more than just flag files. It is exported for use in tests in other
packages.
Bug: 179354495
Test: m nothing
Change-Id: I9f780b20e18ce3a774e4aa04a276463070a64c34
Encapsulating the information needed by hidden API processing in a
struct makes it easy to add additional information in future and allows
the code to populate that struct from various different sources to be
grouped together.
Bug: 179354495
Test: m com.android.art com.android.ipsec com.android.os.statsd com.android.conscrypt
- verify that this does not change the contents of the apex files
Change-Id: I53805737dff36a3ae87aca5aad51cf46ae1361fe
HiddenAPIFlagOutput encapsulates the paths to the files produced by the
hidden API flag generation of a single bootclasspath_fragment. It is
returned from hidden API flag generation and is embedded within the
hiddenAPIFlagFileInfo so they can be passed to other modules.
Unlike the fields it replaces in hiddenAPIFlagFileInfo the fields in
HiddenAPIFlagOutput are of type Path not Paths which makes it easier to
use.
Bug: 179354495
Test: m com.android.art com.android.ipsec com.android.os.statsd com.android.conscrypt
- verify that this does not change the contents of the apex files
Change-Id: I7373ba1001cac3a75eb54a23e62fa52f5013ee7f
The hiddenAPIFlagFileInfo was being used for both the input and output
of bootclasspath_fragment and platform_bootclasspath and also to pass
information around to various hidden API rule methods. Supporting
multiple different uses in this way made it hard to reason about.
This change creates a separate structure for use by the
platform_bootclasspath. Follow up changes will split out other
functionality into separate types.
Bug: 179354495
Test: m com.android.art com.android.ipsec com.android.os.statsd com.android.conscrypt
- verify that this does not change the contents of the apex files
Change-Id: Ia5c5f65ae5645486c42819c669a8601588217f88
Previously, the apex content info was populated with hidden API encoded
dex jars retrieved directly from the java module. This change retrieves
the unencoded dex jars from the java module, encodes them and then
stores the result in the apex content info.
Bug: 179354495
Test: m com.android.art com.android.ipsec com.android.os.statsd com.android.conscrypt
- verify that this does not change the contents of the apex files
Change-Id: Ib1b6eb8b62ac50e03b9e0d07c877ca70bb6f6d25
To avoid duplicates on *CLASSPATH environ variables at runtime, remove
split entries from platform-*classpath, i.e. all updatable jars that
have their own classpath fragments should not appear in the
platform-*classpath's classpaths.proto config.
Bug: 180105615
Test: m && launch_cvd; atest CtsClasspathsTestCases
Change-Id: Id2759ab8e106cc183e695bf3509a6ab60ab0ef2a
Merged-In: Id2759ab8e106cc183e695bf3509a6ab60ab0ef2a
Previously, the DexBootJarPathForContentModule(module) simply called
directly through to the module to retrieve the dex jar path. This
change changes it so the bootclasspath_fragment retrieves the dex
jars from the module and stores them in the info structure for this
method to retrieve directly.
This makes it easier for the bootclasspath_fragment to stop retrieving
hidden API encoded dex jars from the module and perform the encoding
itself.
Bug: 179354495
Test: m com.android.art com.android.ipsec com.android.os.statsd com.android.conscrypt
- verify that this does not change the contents of the apex files
Change-Id: Ic79dea080f10f4017f1a75d6d1fb5a3bfe04c2ce
The new info struct can be easily shared with systemserverclasspath
fragments.
Bug: 180105615
Test: m nothing
Change-Id: I9986e64fdf19f4168da63c156de3dc9bcafac8d8
Hidden API processing of a bootclasspath_fragment requires the fragment
provides information, such as dependencies on other fragments and flag
files to override the default flags. Failing to do so will cause hidden
API generation to either fail or to generate different flags to that
generated by the hidden API processing done by platform_bootclasspath
which will cause the build to fail.
Previously, this was handled by only performing hidden API processing
for those modules that provide stub libs and relied on there only being
bootclasspath_fragments defined for ART (which already supports hidden
API processing), and Conscrypt and I18n neither of which provide stubs.
Unfortunately, that can no longer be relied upon due to a couple of
recent changes:
1. A java_sdk_library in a bootclasspath_fragment's content property is
automatically treated a stub library. That avoids duplication for
most bootclasspath_fragments that provide the implementation and
stub libraries through the same java_sdk_library. It does not affect
either ART, conscrypt or i18n as they all define the implementation
separately to the stubs.
2. bootclasspath_fragment modules have been defined for a number of
android modules as they are needed for reasons other than hidden API
processing.
In combination this meant that rules to perform hidden API processing
were being created but they were not currently being used which is
good because they fail.
However, adding the fragment to the platform-bootclasspath will cause
those rules to be used as the platform_bootclasspath module performs a
consistency check on the hidden API flags generated by each of the
fragments to ensure that they are consistent with those it generates
itself.
The dynamic bootclasspath work will need to add fragments to the
platform_bootclasspath and without this change that would be blocked
until all fragments had been switched to generating hidden API flags
properly.
This change adds a new fragments property to the bootclasspath and
disables hidden API processing for everything other than ART and tests
if the fragments property or stub libs is empty.
Bug: 179354495
Test: - Before making this change.
- Add com.android.os.statds-bootclasspath-fragment to platform-bootclasspath.fragments
m out/soong/hiddenapi/hiddenapi-flags.csv
- hidden API processing fails in statsd as it cannot find java.lang.Object.
- After making this change, run the above command again and it should pass.
Change-Id: Ifbb362f8fcfb2c06595fbd5ae39421b536e329ef
A previous change treated this as an error in order to try and detect
issues with misconfigured java_sdk_library modules and/or
bootclasspath_fragment modules but unfortunately this is not always an
error, e.g. when migrating a library that was a shared library to the
bootclasspath.
This change stops treating that as an error.
Bug: 179354495
Test: m nothing
Change-Id: I4a833ab5f4caf86c6cd340090fc65d2c2f141512
Previously, the monolithic hidden API files, e.g. hiddenapi-index.csv
file, were generated in two steps. First, each module created its own
files using the information in its class jars. Then, the monolithic
files were created by merging those module specific files into a larger
file.
This change switches to generating the monolithic files directly from
the class jar files and bypassing the intermediate files.
In order to ensure that this change did not change the monolithic files
it is necessary for the hiddenapi-metadata.csv to go through a
reformatting step. Hopefully, this will be able to be removed in a
follow up change.
Bug: 179354495
Test: verified that the monolithic out/soong/hiddenapi/... files are
unchanged by this change
Change-Id: I5a78e747516014b7c0f402a4b4431b14be6a84b2
A java_sdk_library specified in the bootclasspath_fragment contents
must be providing APIs for the bootclasspath_fragment and so must be
treated as if they were specified in the stub_libs. This avoids having
to specify them in both contents and stub_libs.
Bug: 179354495
Test: m nothing
Change-Id: I535065ee1a79b439e2676f35e06a75d4626adcaf
This change adds support for generating the hidden API flags for the
contents of a bootclasspath_fragment. Currently, it will only work for
the art-bootclasspath-fragment as it has no support for creating
dependencies between bootclasspath_fragment modules which will be
needed for handling any other bootclasspath_fragment.
The hidden API flag generation added by this change is completely
separate to the normal hidden API processing and is not as yet encoded
in dex jars so will have no effect on the runtime.
The generated files are provided for use by other modules and copied
into the sdk snapshot. That is needed to allow the build to verify that
the hidden API flags generated by the individual bootclasspath_fragment
modules are consistent with the flags generated for the whole
bootclasspath, whether building from source or prebuilts.
Bug: 179354495
Test: m art-module-sdk
m out/soong/.intermediates/art/build/boot/art-bootclasspath-fragment/android_common_apex10000/modular-hiddenapi/all-flags.csv
m out/soong/hiddenapi/hiddenapi-flags.csv
- test that the former file is a subset of the latter and that
where they overlap they are identical.
Change-Id: Ie27303e2960953db1b7abe95510e3bca4411b09a
A java_library specified in a bootclasspath_fragment's contents
property will be automatically added to the sdk containing that
bootclasspath_fragment. Previously, if that was attempted with a
java_sdk_library it would be added to the sdk as if it was a normal
java_boot_libs which would prevent the sdk from containing the
API specific artifact such as current.txt files and stub libraries
and sources.
This change fixes that and adds a java_sdk_library as a java_sdk_libs
module.
Bug: 177892522
Test: m nothing
Change-Id: Ided57b846ce5b8940c7e898c786fd77602582ea2
Moves the building of boot images from the dexpreopt_bootjars singleton
to the bootclasspath_fragment and platform_bootclasspath.
The art boot image is generated by the art-bootclasspath-fragment
module and the framework boot image by the platform-bootclasspath
module.
This does temporarly duplicate the generation of an identical boot
profile for each image. As part of the work to modularize the boot
image profile each image will have its own custom default boot profile.
Bug: 177892522
Bug: 186455808
Test: m droid and TreeHugger
Change-Id: I23cf05ec7648749b21c7cf6fcba282b46649a981
