This relands aosp/2478277. The original CL was submitted as part of a
stack that broke tm and udc since those branches contains soong modules
that have min_sdk_version of type (kind+level).
Test: go test ./sdk
Bug: 208456999
Change-Id: Ie5d7f2597092a51c6e74e7b3cab2f81c02906d5e
Previously, the sdk snapshot code used SdkAware instead of Module to
ensure that all its members had implemented SdkAware. However, it never
used any of the methods provided by SdkAware, or if it did it no longer
does. So, this change replaces usages of SdkAware with Module in
preparation for deleting it completely.
Bug: 260237150
Test: m nothing
Change-Id: Ia89e02394f27b2da776f0cf0f0bc86835a03433a
The following environment variables are no longer used when generating
sdk snapshots:
* SOONG_SDK_SNAPSHOT_PREFER - we can control the modules that are
preferred at build time now.
* SOONG_SDK_SNAPSHOT_USE_SOURCE_CONFIG_VAR - this is done as a
post-processing step by the mainline_module_sdks.sh script to avoid
having to regenerate the ninja files for every module.
* SOONG_SDK_SNAPSHOT_VERSION - support for this was removed a long time
ago but was still referenced from dumpvars.go.
Bug: 259095197
Test: m nothing
# Searched for uses of these variables but could not find any
# outside the build.
Change-Id: Id2b32b29ff6f09faffea43292f549ba13acd956d
The sdk snapshot must not be including implementation code for boot
libraries, the implementation is provided by dex jars within the
corresponding APEX. However, the snapshot does need a module for each
boot library so that the build can seamlessly access the dex files from
the APEX.
A java_library boot library (like core-oj) is represented in the
snapshot by a java_import module which requires a jar file to be
provided, otherwise it is disabled. However, that is provided purely
to keep Soong happy and should never be used.
Previously, the snapshot would contain an empty file for the jar. As
an empty file is an invalid jar any tool (like compiler) that tried
to consume it would fail which was the correct behavior. Unfortunately,
the error message that was produced was not very helpful, it was just
some variant on `invalid file` which lead to a lot of bugs being
raised.
This change replaces that empty file with a reference to the output
from a genrule which runs a script which produces a more useful error
message, with information on how to fix the issue, and fails the build.
It also adds a Name() method to the SdkMemberProperties type as that is
needed in AddInternalModule() to construct the name of the additional
module.
Tested as follows:
In AOSP/master make the following changes:
1. Temporarily set visibility on core-oj and core-libart to
//visibility:public.
2. Run packages/modules/common/build/mainline_modules_sdks.py to create
the snapshots.
For each of the S, T and latest snapshots I did the following in the
s-aml-prebuilt-test, t-aml-prebuilt-test and aosp/master branches:
1. Created an Android.bp file containing the following:
java_library {
name: "broken",
static_libs: [
"prebuilt_core-libart",
"prebuilt_core-oj",
],
}
2. Fix the visibility issues and run `m broken` where it fails with an
invalid file.
3. Delete the contents of the prebuilts/module_sdk/art/current/sdk
directory.
4. Unpack the relevant version of the art-module-sdk snapshot into the
directory.
5. Run `m broken` where it fails with the helpful message.
6. Test the instructions on how to use the ninja -t path tool to
identify the cause of the problem and fix it.
Bug: 257969510
Test: See above.
Change-Id: I125bde2d7202afff84c97daebcef37e21c548a3a
Fields tagged with `sdk:"keep"` will keep the value even if it would
normally be cleared because it was common across a number of structs.
This will allow a module type to specify the same value across all
structs populated directly from variants and have it be copied into all
common property structs without clearing it from the variant specific
structs.
Bug: 248258460
Test: m nothing
Change-Id: I147d946b11fd8766a7d875d9206e8f5034f585d6
"ignore" is a better description of the effect that this tag has on the
optimization that is done as part of the sdk snapshot generation.
"keep" implies that something special is done with the field during the
optimization process but actually the optimization process completely
ignores the property, just as it does with unexported fields.
Bug: 248258460
Test: m nothing
Change-Id: Idfdfd87f927d641d4d301cbbde72b463c89155bf
When an sdk snapshot is targeted at release X then it cannot include
bootclasspath fragment libraries which are not present in that build as
otherwise it causes build failures. It should also not include any
unsupported libraries, i.e. libraries that cannot work on that release.
This change causes sdk snapshot to exclude libraries that have a
min_sdk_version > target build release
It also ensures that hidden API flags do not include any information
from excluded libraries.
Bug: 240406019
Test: BUILD_NUMBER=fixed packages/modules/common/build/mainline_modules_sdks.sh
# Ran the previous command with and without this change to make
# sure that this change excludes framework-connectivity-t library from the
# tethering sdk snapshot for S, including from the hidden API flag files.
Change-Id: I57969b85a12e9e5a3fc76c055b260cec5d5f7d7f
Previously, the build releases only included named releases so did not
have a way to represent the latest build release. This adds the current
build release (named after the current API level) to represent that.
Bug: 240406019
Test: m nothing
Change-Id: Ib8336da716b447448b23bc9684ce3be1ab78648a
The order of the files in the sdk snapshot was stable but depended on
how the sdk was built. If the snapshot was created directly then the
files have one order, if it was created by merging in files from other
zips then it has a different order.
This change ensures files are in alphabetical order no matter how the
snapshot zip is constructed.
Bug: 232401814
Test: m media-module-sdk
# Check the contents of the following zips to ensure that they are sorted
# after this change.
# out/soong/.intermediates/packages/modules/Media/apex/media-module-sdk/common_os/media-module-sdk-current.unmerged.zip
# out/soong/mainline-sdks/media-module-sdk-current.zip
Change-Id: Ie97e0119c07a1f34a2b1d3ea6895f0e76cd195a8
While enabling prebuilts in T we hit b/229932396 which was caused by
some parts of the build depending on the prebuilt updatable-media jar
which used to be a full implementation jar but which is now an invalid
jar as the snapshot must not be including implementation details. We
fixed the issue in T but we are hitting the same problem in S with the
M-2022-07.
That is the first train in which the prebuilt updatable-media module
provides an invalid jar, prior to that it was always providing an
implementation jar. This change tweaks the sdk snapshot generation
code to use an implementation jar for updatable-media in the S
snapshot to avoid partners having to cherry pick changes similar to
those needed to fix b/229932396 in T.
Bug: 239121291
Test: packages/modules/common/build/mainline_modules_sdks.sh
# Check that S media snapshot includes implementation jar.
# Check that S art snapshot includes invalid jar.
# Check that T media snapshot includes invalid jar.
Change-Id: Ib49484d00a60b4ed7f8268e04f9c10a3498edb56
Specifying an apex in the apexes propety will cause all the
*classpath_fragments that are contents of the APEX to be automatically
added as members of the sdk and appear in the snapshot.
The purpose of this change is to dedup the APEX and sdk definitions and
try and avoid some of the issues that we have been finding while
attempting to build against the prebuilts.
Two tests, one each for bootclasspath_fragment and
systemserverclasspath_fragment, have been refactored to compare the
output when adding the *fragment to the sdk directly of via the APEX.
That ensures switching to use the APEX will not change the sdk snapshot
unless it was previously missing a *fragment.
There was also a slight difference in where the hidden API flags were
copied from. That should have no impact on the output as the flags are
identical.
The sdk snapshot generation needed some tweaks to avoid generating a
prebuilt for the APEX.
Bug: 232401814
Test: m nothing
Change-Id: I7aaf16a3a0ab4bebf97765d1484215cc008dc4b8
Previously, when targeting the S release the generated sdk snapshot
would contain prebuilt_systemserverclasspath_fragment modules even
though they were only added in T.
This allows SdkMemberTypes to specify the set of target build releases
they support and ignores them when targeting an unsupported target
build release.
Test: m nothing
packages/modules/common/build/mainline_modules_sdks.sh
# Check that the for-S-build snapshots do not include SSCPFs.
Bug: 237718221
Change-Id: I2df08c2fcebf9b866695d691572a9d3783758b17
Previously, the code for selecting specific versions of sdk snapshots
was removed (along with the uses_sdks property). That makes versioned
snapshots useless so this change removes all the code and tests that
generated those versioned snapshots.
Bug: 232546567
Test: m nothing
packages/modules/common/build/mainline_modules_sdks.sh
Change-Id: Ib6d1b72bc8399fbb39075494ae37da92f4b28d03
Currently, if the same library is specified for multiple of native_libs,
native_shared_libs, and native_static_libs for different arch/oses,
there can be a few errors:
1. specifying a .so file as `srcs` within a cc_prebuilt_library rather
than being specified only for shared
2. the final type of prebuilt library is dependent on the arch/os
This change introduces:
* an ability for a member type to override the type for specified
properties
* checks for a library being used with incompatible member types
* basing linkage nesting on the member type in addition to variants
This will ensure that the correct library type is used, regardless of
the order of iteration over oses/arches, and support nesting linkages
where necessary but only one linkage variant exists.
Test: soong tests
Test: CI
Change-Id: I81dee013b09b99c34ca6c18f9cfcc12ee56d33d1
Previously the SDK info file only contained basic common information
about each member. This change adds support for each member to
provide custom information to add to the info file.
It uses that mechanism to add the following:
* "dist_stem"
* "scopes" object containing:
* for each scope a:
"<scope>" object containing:
* "current_api" - the path within the snapshot for the API's .txt
file.
* "removed_api" - the path within the snapshot for the removed
API's .txt file.
* "latest_api" - the path within the build to the latest finalized
API .txt file.
* "latest_removed_api" - the path within the build to the latest
finalized removed API .txt file.
In order to access the latest API files it was necessary to add and
resolve dependencies on the module that makes them available. In order
to do that safely the code for creating the names of the modules was
refactored to avoid duplicating the name creation logic.
Bug: 204763318
Test: m nothing
Change-Id: Ica68abbd2b2c7c2b2b7877b502f96cc89f06fd68
Some build scripts need to know information about the contents of an
sdk, such as what APIs it provides (via java_sdk_library). Rather than
duplicate that information in the scripts or attempt to access that
information (where available) by looking at the contents of the
snapshot this change generates a JSON file that sits alongside the
snapshot itself.
The info file can be generated without generating the snapshot zip file
but whenever a snapshot zip file is generated the info is generated
too. The info file sits alongside the zip file in out/mainline-sdks.
Bug: 204763318
Test: m art-module-sdk
m dist
Change-Id: I289530bb21693dc6443826c24c17c9b5d85d2d8b
This change adds the build_number.txt file into the sdk snapshot to
make it easier to debug issues arising in partner builds caused by the
snapshots.
The file is added to the root of the snapshot and is called:
snapshot-creation-build-number.txt
Bug: 201295663
Test: m nothing
packages/modules/common/build/mainline_modules_sdks.sh
# Check the contents of various snapshots.
Change-Id: Ib3ac0cea5c9315cad0b9a3c25d88759097e37163
This was reverted unnecessarily along with the change that actually
broke the build.
Previously, the sdk snapshot was assumed to be generated for the
current build system. This change adds support for the
SOONG_SDK_SNAPSHOT_TARGET_BUILD_RELEASE environment variable which can
be used to select the target build release in which the generated
snapshot will be used.
At the moment this will only affect the properties that are output but
if/when it becomes necessary it can also be used to control more
aspects of the snapshots such as supported member types or even
members.
This change does not modify any properties that are build release
specific. That will come in following changes. However, it does add a
test that targets build release S which defines a baseline for
properties that will be affected in following changes. That baseline
makes it easier to see the effect of those follow up changes.
Bug: 197842263
Test: m nothing
Change-Id: If4b452237f105382550d2842c8010249afbc7432
Previously, the sdk snapshot was assumed to be generated for the
current build system. This change adds support for the
SOONG_SDK_SNAPSHOT_TARGET_BUILD_RELEASE environment variable which can
be used to select the target build release in which the generated
snapshot will be used.
At the moment this will only affect the properties that are output but
if/when it becomes necessary it can also be used to control more
aspects of the snapshots such as supported member types or even
members.
This change does not modify any properties that are build release
specific. That will come in following changes. However, it does add a
test that targets build release S which defines a baseline for
properties that will be affected in following changes. That baseline
makes it easier to see the effect of those follow up changes.
Bug: 197842263
Test: m nothing
Change-Id: I5cfde5e9e86af522b0d5030d301e116b0233fd62
Previously, the archTypeSpecificInfo included an array of
*linkTypeSpecificInfo. This change replaces that array with an array of
*imageVariantSpecificInfo which themselves contain an array of
*linkTypeSpecificInfo.
That allows the sdk snapshot to handle image variants correctly, i.e.
collate their properties, optimize their properties and then detect if
there are any image variant specific properties for any image variant
other than the CoreImageVariant ("") and report it as an error.
The latter case is treated as an error because while Soong needs to
handle image specific variants there is currently no requirement to
handle generating a prebuilt with image specific properties. A follow
up change will test the error handling.
Image specific variants are needed because the "jni_headers"
cc_library_headers module provides a number of image variants (e.g.
recovery) that are used outside the ART module. Therefore, the sdk
snapshot needs to do the same.
At the moment image variants like the recovery variant are supported by
copying the property that creates the variant (e.g. recovery_available)
through to the prebuilt but that is not safe for a couple of reasons:
1. It ignores any differences between the recovery variant and the
other variants which could cause compatibility issues in modules
that build against the prebuilts.
2. It marks modules in the snapshot with recovery_available even when
they do not need it.
This change will allow follow up changes to address both those issues.
Bug: 195754365
Test: m nothing
Change-Id: I1c187d814f44b2cb7607cd43a6b215134be0faad
Previously, the snapshot code ignored native bridge related arch
specific variants and only used the non native bridge variants. This
change enabled it to track native bridge variants too.
There is currently no support for creating native bridge specific
prebuilts so this ensures that any attempt to create one will fail
during generation of the snapshot with a clear message.
At the moment that error message cannot be tested by creating an sdk as
there is no way to add a dependency from the sdk onto a native bridge
variant. A follow up change will add the capability to test this
functionality through an sdk.
Bug: 195754365
Test: m nothing
Change-Id: I0494fdba9cc2ff013ea990d12d6a03a0a444bd5c
Currently, every sdk member of a specific module type has to be treated
in the same way as every other sdk member of that type. e.g. it is not
possible for an sdk member to use different variants to other members
of the same type.
Adding a new member type for each different way to treat the members is
not scalable as if there were N different ways treat a member then it
would require 2^N types for all the possible combinations.
This adds a new traits mechanism that allows the behavior of member
types to be customized per sdk member. Each member type can specify a
list of supported traits and customize its behavior based on which
traits are required for each member. A trait can be supported by
multiple different member types.
Bug: 195754365
Test: m nothing
Change-Id: I165ac80d208c0402d2a9ffa8085bba29562c19b7
Makes it adhere to the go standard practice of prefixing documentation
comments with the name of the type/func/method.
Bug: 195754365
Test: m nothing
Change-Id: Idf3fe827edc9b6d67d12a99a4b27539ac938ea95
Previously, some of the sdk member type related types and variables
included the words "member type" whereas others only used the word
"member" and did not include "type". This changes makes them all
include the word "type" to differentiate them from other types related
to sdk members.
Bug: 195754365
Test: m nothing
Change-Id: I913daaa1d436fd5dc857c2af0ffdee6471dcc6ed
This reverts commit 22ff0aaf51.
The workaround to support hidden API generation with prebuilts by
passing the full implementation jars through to the SDK snapshot is no
longer needed as the monolithic hidden API flag generation uses the
prebuilt flag files that are already part of the snapshot.
This change reverts that previous workaround.
Bug: 192868581
Test: - Update the prebuilts
DIST_DIR=$PWD/dist TARGET_BUILD_VARIANT=userdebug art/build/build-art-module.sh --skip-apex
packages/modules/ArtPrebuilt/update-art-module-prebuilts.py --local-dist=dist --skip-cls --skip-apex
- Build hidden API flags to make sure it does not fail
Change-Id: Idb3fde6f7dcb171677316d8794a4af91ede1f7e0
Allows the builder of an sdk snapshot to specify a Soong config
variable that can be used to control whether the snapshot's prebuilts
are used or not.
Bug: 193523070
Test: m nothing
Change-Id: Ib09500ba61befc1202dff61dc06847c730ba9253
This reverts commit d0b99cea0a.
Reason for revert: Not needed anymore. We will simple exclude the
gernated Android.bp files from prebuilt drops instead, it keeps
things simpler.
Change-Id: I5b01b2e01197f9fceb4630c8ade95f0d1d0f33f4
This is needed because:
- the SDK prebuilt drop is being automated, including the Android.bp
- We want to control the prefer flag on a per-module level
It augments the existing SOONG_SDK_SNAPSHOT_PREFER which now overrides
this new flag when set to true.
Bug: 188427719
Test: m nothing
Change-Id: Ieb5ab9fab53a34c615345b7a9d19cadf713eec26
Previously, hidden API processing could only be done by those
bootclasspath_fragment modules that either did not depend on any other
fragments (e.g. art-bootclasspath-fragment) or only depended on APIs
provided by other fragments (e.g. i18n-bootclasspath-fragment). That
meant that modules like com.android.os.statsd-bootclasspath-fragment
that depended on APIs provided by parts of the platform which are not
yet part of another bootclasspath_fragment could not perform hidden
API processing.
This change adds support for a bootclasspath_fragment to specify the
additional stubs needed to perform hidden API processing. It adds a new
additional_stubs property that can be used to specify the additional
stub libraries.
Most bootclasspath_fragments that need to use the property will need
access to the APIs provided by the android-non-updatable.* libraries.
Rather than have each fragment explicitly specify the correct module
for each scope it treats "android-non-updatable" as if it was a
java_sdk_library that can provide different jars for each scope.
Soong will handle mapping that to the correct android-non-updatable.*
module.
Bug: 179354495
Test: m out/soong/hiddenapi/hiddenapi-flags.csv \
out/soong/hiddenapi/hiddenapi-index.csv \
out/soong/hiddenapi/hiddenapi-stub-flags.txt \
out/soong/hiddenapi/hiddenapi-unsupported.csv
- make sure that this change does not change the contents.
m TARGET_BUILD_APPS=Calendar nothing
Change-Id: Ia8b79830ed0e6d42100de03d76b0c51b7f6c8ade
Previously, an sdk snapshot could contain the following:
* A java_sdk_library_import module, e.g. "foo" which creates component
modules "foo.stubs", etc.
* A corresponding versioned module, e.g. "sdk_foo@current" which
created component modules "sdk_foo@current.stubs", etc.
* An internal (to the sdk snapshot) java_import for one of "foo"'s
components, e.g. "sdk_foo.stubs"
* A corresponding versioned module, e.g. "sdk_foo.stubs@current".
That causes a few problems:
1. The "foo.stubs" is duplicated.
2. The names of the components created by the versioned
java_sdk_library_import are invalid, as they append the component's
suffix to the version and not the name before the version.
The latter causes problems when building against prebuilts and fixing
that causes the generated snapshot to be invalid because it contains
duplicate definitions of the "sdk_foo.stubs@current" module. One
explicitly in the Android.bp file and one created by the
"sdk_foo@current" module.
Removing the duplicates from the snapshot causes errors as the name
generated by the snapshot for the component module, i.e.
"sdk_foo.stubs@current" does not match the name generated by the
"sdk_foo@current", i.e. "sdk_foo@current.stubs".
This change fixes them together.
Bug: 179354495
Test: m nothing
Merged-In: I515f235fe21755b5275af12366e96c24c94c0273
Change-Id: I515f235fe21755b5275af12366e96c24c94c0273
(cherry picked from commit a1aa7387f7)
Adds support for printing a PropertySet that has a property whose value
is an array of structs.
Bug: 179354495
Test: m nothing
m conscrypt-module-sdk
- check generated Android.bp file
Change-Id: I71be04188465610bcbea4d3c9a5e8204171a1eeb
Previously, all of the supported types were handled explicitly using
type switches. This change switches to use reflection instead to allow
it to work with unknown types such as custom structures, support for
which will be added in a follow up change.
Bug: 179354495
Test: m nothing
Change-Id: I197f3f1dbce6d80c0127c99bbe6f836d422a8db9
SOONG_SDK_SNAPSHOT_VERSION=current will generate unversioned and
versioned prebuilts and a versioned snapshot module. This is the
default behavior. The zip file containing the generated snapshot will
be <sdk name>-current.zip.
SOONG_SDK_SNAPSHOT_VERSION=unversioned will generate unversioned
prebuilts only and the zip file containing the generated snapshot will
be <sdk name>.zip.
SOONG_SDK_SNAPSHOT_VERSION=<number> will generate versioned prebuilts
and a versioned snapshot module only. The zip file containing the
generated snapshot will be <sdk name>-<number>.zip.
Bug: 157884619
Test: m nothing
m SOONG_SDK_SNAPSHOT_VERSION=current art-module-sdk
- check that the generated Android.bp file has not changed
from the default.
m SOONG_SDK_SNAPSHOT_VERSION=none art-module-sdk
- check that the generated Android.bp file does not contain
versioned modules.
m SOONG_SDK_SNAPSHOT_VERSION=2 art-module-sdk
- check that the generated Android.bp file only contains
version 2 of each module.
Change-Id: I087e9d7d3ad110508a3d6a39bca50cbb46b3ce82
This ensures that the LSC license tool does not attempt to add a
package module to an sdk snapshot Android.bp.
Bug: 181569894
Test: m nothing
Change-Id: I37b66e0df56c5b9ec255f66cb4f2a066ea96a738
Adds initial support for adding license modules to the sdk, along with
any referenced license text files. There is a number of minor
improvements to be made but the core of the support is there and it
works for ART sdks.
Basically, this change will automatically add license modules
referenced from any sdk member module as an internal sdk member. An
internal module has an sdk snapshot specific name that will not
conflict with the source module and which cannot be referenced from
outside the sdk snapshot.
Bug: 181569894
Test: m art-module-sdk art-module-host-exports art-module-test-exports
- diff output before and after this change
made sure that every prebuilt had a licenses field
and that the license modules were added
Change-Id: I0c5ccabf58f4ef487e42ef8e61a5b2a74c0e81af
By default the generated snapshot has prefer: false. Building it with
SOONG_SDK_SNAPSHOT_PREFER=true will force it to generate prefer: true.
Bug: 157884619
Test: m nothing
m SOONG_SDK_SNAPSHOT_PREFER=true art-module-sdk
- check the generated Android.bp file to make sure it contains
prefer: true
Change-Id: Ied297b32d9bd4822a140fa99016f38e234c50f64
Previously, only those SdkMemberTypes which had specific need to
automatically add some of their dependencies as sdk members would cause
the sdk to visit their transitive dependencies. However, as any module
can have dependencies on license modules and license modules need to be
included in the sdk then it needs to visit transitive dependencies of
all members.
So, this change removes the support for allowing an SdkMemberType to
control whether its transitive dependencies are visited and just visits
them all.
This does not have any effect on sdk snapshots as in order for a
dependency to be added to an sdk it needs to be added with a tag that
implements SdkMemberTypeDependencyTag and the only tags that implement
this are used by SdkMemberTypes that had enabled transitive members.
Bug: 181569894
Test: m art-module-sdk art-module-host-exports art-module-test-exports
- verify that this change has no effect on the generated snapshots
Change-Id: If0293af0237aa7e39335e5b8383a41c023ff5853
