Commit Graph

3984 Commits

Author SHA1 Message Date
Paul Duffin
975e757232 Merge "Combine hidden API encoding with flag generation" 2021-06-17 16:03:22 +00:00
Paul Duffin
e521881bd4 Combine hidden API encoding with flag generation
Previously, the rules to perform hidden API encoding were generated
separately to the rules to perform hidden API flag generation. This
change combines them within the (renamed) produceHiddenAPIOutput()
method and makes the paths to the encoded dex files an output of the
generateHiddenAPIBuildActions method alongside the paths to the
generated flag files.

As encoded dex jars are now an output of the produceHiddenAPIOutput()
method which is implemented for both prebuilts and source
bootclasspath_fragment modules that necessitated the prebuilt also
providing paths to encoded dex files. That in turn required updates to
some of the tests to provide dex files from prebuilt_apex modules.

Similarly, as the produceHiddenAPIOutput() method may not be called
for some bootclasspath_fragment modules as they do not yet provide all
the information needed to perform hidden API encoding then it is
necessary to extract the encoded dex files produced by the modules
themselves. That also required a few changes to tests that did not
previously provide dex files.

Bug: 177892522
Test: m com.android.art
      - check that this change does not change the contents of the apex
        file, i.e. is byte-for-byte identical.
Change-Id: I60996a34d06ed1c87ed244ab3509621999ad86ec
2021-06-17 10:20:33 +01:00
satayev
142ed27e32 Rename ClasspathFragmentToConfiguredJarList methods to configuredJars.
They are internal implementation details of individual fragments.

Bug: 191127295
Test: m
Change-Id: Ib350135f5f6720741492f362f7b3203e5107772e
2021-06-16 10:13:55 +01:00
Treehugger Robot
768692bc69 Merge "Add a build flag to always enable errorprone per-target" 2021-06-15 00:32:41 +00:00
Cole Faust
75fffb14b8 Add a build flag to always enable errorprone per-target
Currently, errorprone is only run if the RUN_ERROR_PRONE
enviornment variable is true. Add a flag that individual
modules can use to always enable errorprone.

In a followup cl, I plan to add another flag that will
force all errorprone checks to be errors, so that modules
can be confident that they're not ignoring any errorprone
checks.

Bug: 190944875
Test: New unit test and manually
Change-Id: Iab0c81642ed22a736add054147829e91a891d179
2021-06-14 15:31:32 -07:00
Paul Duffin
dd5993f6d4 Avoid passing around []hiddenAPIModule
Previously, an []android.Module was converted to an []hiddenAPIModule
that was then used to retrieve boot dex jars. That was ok when
obtaining the dex jar files from source modules for
bootclasspath_fragment but does not work well for other use cases as
it would require doing that conversion in multiple places.

This change pushes the use of hiddenAPIModule down to the methods that
retrieve information from it which makes the methods more flexible and
easier to reuse.

Bug: 177892522
Test: m nothing
Change-Id: Ib84aaf03d8f5a63b48232036fe4589646fc23352
2021-06-14 17:23:52 +01:00
Paul Duffin
cc1bd8364a Merge "Make copyBootJarsToPredefinedLocations simpler and less fragile" 2021-06-14 08:20:50 +00:00
Paul Duffin
a11b8707f1 Merge "Ensure primary boot image files are created before they are used" 2021-06-14 08:01:27 +00:00
Paul Duffin
2eddb50fd5 Merge "Move TestPlatformBootclasspath_Fragments to apex package" 2021-06-11 04:40:04 +00:00
Paul Duffin
5e1073b7cd Merge "Export hidden api related types and fields" 2021-06-11 04:39:05 +00:00
Colin Cross
7b4bda29fc Merge "Always propagate some environment variables to RBE" 2021-06-10 15:57:22 +00:00
Paul Duffin
5f148ca7cf Make copyBootJarsToPredefinedLocations simpler and less fragile
Previously, copyBootJarsToPredefinedLocations relied on all its
parameters having the same length and the same order. That made it
quite fragile as changes to one of the parameters without corresponding
changes to the other would cause failures. It also combined the
retrieval of the boot dex jars from the modules, handling of missing
boot dex jar files and the generation of the rules to copy the files.

This change separates the retrieval of boot dex jars and handling of
missing files from the copying of those files while at the same time
making the function less fragile by replacing the three ordered
parameters with two maps that shared common keys.

Bug: 179354495
Test: m nothing
Change-Id: Idbcd24a7e8af89f7895a20aeddc58502dcbaad03
2021-06-10 14:28:10 +01:00
Paul Duffin
bff50e2b80 Ensure primary boot image files are created before they are used
Previously, when building a framework boot image variant it added an
implicit dependency onto the first file in the primary boot image
variant to ensure that the primary boot image variant files that the
dex2oat command needs have been created. That works when generating
from source as in that case all the files for a boot image variant are
created by a single command. However, it does not work for prebuilts as
each prebuilt file will be copied into the required location by
separate copy commands.

This change adds all the files that the dex2oat command uses implicitly
when building an extension boot image as implicit dependencies.

Bug: 177892522
Test: m SOONG_CONFIG_art_module_source_build=false droid
      - the previous command only works in combination with a number of
        other build changes.
Change-Id: I183748fd17f8f3003890675b8c6bb9fcab331443
2021-06-10 13:40:33 +01:00
Paul Duffin
ffa8375f07 Move TestPlatformBootclasspath_Fragments to apex package
This test checks that fragments which are referenced from a
platform_bootclasspath module contribute their hidden API flags to those
used by platform_bootclasspath module. Previously, it was unrealistic
because the bootclasspath_fragment does not belong in an APEX.

This change moves the test from the java package to the apex package to
allow it to be modified to make the bootclasspath_fragment part of an
apex.

Bug: 179354495
Test: m nothing
Change-Id: Icb57f2e1eaea4b14aab5f47f3af7d05ea0555816
2021-06-10 12:36:43 +01:00
Paul Duffin
524c82c01a Export hidden api related types and fields
This will export some hidden api related types and fields so they can
be used from outside the java package. This is needed to allow a follow
up change to move the TestPlatformBootclasspath_Fragments from the java
to the apex package.

Bug: 179354495
Test: m nothing
Change-Id: Ib69eea9d79cc83b8e3fc29919a29f071e1ec17b5
2021-06-10 12:36:43 +01:00
Colin Cross
95fad7a56e Always propagate some environment variables to RBE
Always propagate LANG, LC_MESSAGES and PYTHONDONTWRITEBYTECODE
to RBE to get more consistent behavior between local actions and
RBE.

Bug: 182415460
Bug: 190593001
Test: treehugger
Change-Id: I726e6f02fd3ef77e158baf6fde77ffb7247a1375
2021-06-09 15:05:40 -07:00
Paul Duffin
84c1cdf31f Maintain header order in merge_csv
Previously, if the --header property was not specified then merge_csv
would use a header constructed by sorting all the fields in the input
files. That required that any use of merge_csv which did not already
have headers in the required order would have to explicitly specify the
headers. That made it harder to use merge_csv as a generic tool as each
invocation needed to be aware of what headers were exported in the
output.

This change causes merge_csv to simply use the headers in the order in
which they are encountered in the input files. That removes the need to
specify the --header option when generating the index files.

Bug: 179354495
Test: m out/soong/hiddenapi/hiddenapi-index.csv out/soong/hiddenapi/hiddenapi-unsupported.csv
      - make sure that they are not changed by this change.
Change-Id: I420b7d07aea85af6372cd7580a8be5e2cc82a513
2021-06-09 14:02:03 +01:00
Spandan Das
05f1a0a795 Merge "Port module_partition logic for RRO from Make to Soong" 2021-06-08 20:41:45 +00:00
Spandan Das
5d1b929f21 Port module_partition logic for RRO from Make to Soong
The default partition for RRO is "product/" in Make, but it was
"system/" in Soong. This CL ports the logic from Make to Soong

To implement this, a new function PathForModuleInPartitionInstall is
created that enables callers to provide the relevant partition

Bug: 158407753
Test: from build/soong, ran go test ./java
Change-Id: I05b02eae7fe57189aaad5109c26cccc5823518ef
2021-06-08 17:53:31 +00:00
Colin Cross
9eb611a3cc Merge changes I162b0f68,I1689b670,Id6d997ee,I9aae2a16
* changes:
  Don't use unsafe_ignore_missing_latest_api in TestJavaSdkLibraryDist
  Remove core_lib property from java_sdk_library
  Ignore owner property when computing java_sdk_library dist subdirectory
  Make the default java_sdk_library dist_group "unknown"
2021-06-08 17:02:59 +00:00
satayev
5831f70dab Merge "Add exemption for framework-sdkextensions-classpaths boot jar." 2021-06-08 13:45:56 +00:00
Anton Hansson
556e8149cb Remove sourcepath argument from metalava invocations
This is no longer required to find package.html files, so remove it.

Bug: 153703940
Test: m checkapi
Change-Id: I80ec219cb9ef8922094336a5d45652c8e59aa113
2021-06-07 16:04:46 +01:00
Anton Hansson
edd61643be Stop filtering html files from droidstubs input
Metalava now expects html files to be specified explicitly. This
filtering was originally added because jdiff could not cope with html
inputs, but jdiff support has been removed in r.android.com/1455335.

Bug: 153703940
Test: m checkapi
Change-Id: I34164b633bab2892e3a1c0fd0257fcf98d537655
2021-06-07 16:02:53 +01:00
Anton Hansson
70336cd2fa Make soong create sourcepath dirs
Sourcepath is set to dir of the Android.bp, but that dir does not
necessarily have any inputs for the metalava invocation defined in that
Android.bp. This leads to problems when metalava tries to read a
non-existent dir, so create it first.

Bug: 153703940
Test: m sdk dist
Change-Id: I8c2ca45c8f066cabf3247b9652a7094cefc047a9
Merged-In: I8c2ca45c8f066cabf3247b9652a7094cefc047a9
(cherry picked from commit 7eff3a8fc3)
2021-06-07 14:34:39 +01:00
satayev
1b75a3ce62 Add exemption for framework-sdkextensions-classpaths boot jar.
As long as we have boot jars defined in make, we should try to keep
global.UpdatableBootJars as a source of truth when it comes to classpath
configuration. However, for testing purposes there is a need to add
java libraries to bootclasspath that aren't present in
PRODUCT_UPDATABLE_BOOT_JARS variable.

Ideally, we would unconditionally add all contents to the config for
test apexes, however, I don't fully understand apex merging to support
it.

Bug: 180105615
Test: atest sdkextensions_classpaths_e2e_tests
Change-Id: I552f93289c27b3713e75dcda1faed909b0bfcef2
2021-06-04 19:42:54 +01:00
Colin Cross
67c17aede8 Don't use unsafe_ignore_missing_latest_api in TestJavaSdkLibraryDist
Use FixtureWithLastReleaseApis to create the necessary files instead
of disabling the check with unsafe_ignore_missing_latest_api.

Bug: 186723288
Test: TestJavaSdkLibraryDist
Change-Id: I162b0f68eec596274f8d98dca9d3f0500ab13f5d
2021-06-02 13:02:51 -07:00
Colin Cross
f0eace9eed Remove core_lib property from java_sdk_library
Its not used, remove it.

Bug: 186723288
Test: TestJavaSdkLibraryDist
Change-Id: I1689b670a8ae6a614e5e4ec5e79cb5e283b2e277
2021-06-02 13:02:23 -07:00
Colin Cross
59b92bfdb3 Ignore owner property when computing java_sdk_library dist subdirectory
The owner property is no longer used, the dist subdirectory is determined
by the dist_group property.

Bug: 186723288
Test: TestJavaSdkLibraryDist
Change-Id: Id6d997eef05f6511070677974219674f248cb754
2021-06-02 13:02:01 -07:00
Colin Cross
3dd662509d Make the default java_sdk_library dist_group "unknown"
Change the default dist_group from "android" to "unknown" to prevent
accidentally including java_sdk_library stubs that do not set
dist_group or owner in the public SDK.

Bug: 186723288
Test: TestJavaSdkLibraryDist
Change-Id: I9aae2a16254ac1a8d444acfa63bc571d1ef4b045
2021-06-02 12:59:21 -07:00
Colin Cross
c92738a24b Merge changes I9e62c703,I4c2ae2a2,I899c2946
* changes:
  Support dist_group property instead of owner for setting sdk dist subdirectory
  Add test for java_sdk_library dist properties
  Move java_sdk_library tests to sdk_library_test.go
2021-06-02 14:33:47 +00:00
Yuntao Xu
a8f4e62b8d Merge "add the LOCAL_DISABLE_TEST_CONFIG variable" 2021-06-01 22:11:59 +00:00
Colin Cross
986b69aa51 Support dist_group property instead of owner for setting sdk dist subdirectory
Reusing the owner property is confusing, especially when the property is
required on every java_sdk_library module.  Create a new dist_group property
to use instead.

Bug: 186723288
Test: TestJavaSdkLibraryDist
Change-Id: I9e62c703a95d6b63cafa60bffb1b37ba85388593
2021-06-01 13:40:44 -07:00
Colin Cross
30c491b885 Add test for java_sdk_library dist properties
Add tests that cover the owner, core_lib and dist_stem properties.

Bug: 186723288
Test: TestJavaSdkLibraryDist
Change-Id: I4c2ae2a23dcd9a668c46ed5cc23b1d7dcfae798a
2021-06-01 13:40:17 -07:00
Colin Cross
1d2b6b3b2e Move java_sdk_library tests to sdk_library_test.go
Seperate the java_sdk_library tests into their own file.

Bug: 186723288
Test: go test ./java/...
Change-Id: I899c2946cb2234dc595a4281e64bbb239b89bda8
2021-06-01 13:40:17 -07:00
Yuntao Xu
7a31855228 add the LOCAL_DISABLE_TEST_CONFIG variable
Introduce and set the variable LOCAL_DISABLE_TEST_CONFIG varaible to be
true for the `android_test_helper_app` type modules.

Bug: 188927912
Test: rum 'm module-name' (`android_test_helper_app` type module)
Test: TreeHugger
Change-Id: I5db1029f31d8afe1ca5e9fab95e39d751971058d
2021-05-27 17:53:46 -07:00
Anton Hansson
5ab4286e84 Remove support for removed_dex_api_filename
Nothing uses this argument anymore, so simplify the droidstubs code by
removing the argument.

Bug: 189426360
Test: m
Merged-In: I0e4b84d73a954b3819db1149be3da287cf604eec
Change-Id: I0e4b84d73a954b3819db1149be3da287cf604eec
2021-05-27 12:50:44 +01:00
Martin Stjernholm
664b2cf89a Merge "Drop "prebuilt_" prefixes from names registered in ApexInfo.InApexXxx." 2021-05-27 09:24:55 +00:00
Colin Cross
4ab2dd1afe Merge changes I0116f5f4,I950c9b54,I967f5c42
* changes:
  Treat java libraries in classpath fragments as directly in apex
  Make CopyDirectlyInAnyApex match the documentation
  Remove unused cc.copyDirectlyInAnyApexDependencyTag
2021-05-27 00:56:52 +00:00
Colin Cross
c33e5216f1 Treat java libraries in classpath fragments as directly in apex
Coverage is applied to java libraries that are directly in an apex.
Mark java libraries that are in an apex through a bootclasspath_fragment
or a systemserverclasspath_fragment as directly in the apex by
implementing CopyDirectlyInAnyApexTag on the dependency tags used for
their contents.

Bug: 183759446
Test: TestApexJavaCoverage
Change-Id: I0116f5f415083b5194000988cb257454ef115200
2021-05-26 10:07:44 -07:00
satayev
1c564cc9c9 Use stem when filtering boot jars.
For testing purposes, a boot jar may be provided by a test java_library
that has a different content name, but sets "stem" property to match
the original java_library.

Given that Stem() returns either the property value or module name,
it is safe to replace all content names by their stems.

Bug: 180105615
Test: atest CtsClasspathsTestCases sdkextensions_e2e_tests
Merged-In: Ic519ffa0c5b616abddf15b41c934421dfac2e78a
Change-Id: Ic519ffa0c5b616abddf15b41c934421dfac2e78a
2021-05-26 17:13:43 +01:00
Martin Stjernholm
be10503d3a Drop "prebuilt_" prefixes from names registered in ApexInfo.InApexXxx.
Neither InApexVariants nor InApexModules should have them. This allows
us to get rid of InApexVariantByBaseName as well.

Test: m nothing
Test: m nothing SOONG_CONFIG_art_module_source_build=false
Bug: 180325915
Change-Id: Icbe4e025ce1a4c8dd258ff95d326ca2f27905188
2021-05-26 17:05:17 +01:00
satayev
70975ac815 Merge "Rename generate proto config file to match classpath type." 2021-05-25 14:17:04 +00:00
satayev
8a7bbb5b79 Merge "Populate individual systemserverclasspath_fragments' proto configs." 2021-05-25 11:38:40 +00:00
Treehugger Robot
c2d8fa0515 Merge "Clarify how metalava @SuppressLint works" 2021-05-25 00:21:39 +00:00
Aurimas Liutikas
b23b745e19 Clarify how metalava @SuppressLint works
Bug: 186157121
Change-Id: Ia3316b80dcf16e7fb2a41983732e0be8d0657497
Test: None
2021-05-24 18:01:17 +00:00
Paul Duffin
8018e50ddb Tighten bootclasspath_fragment property validation
Previously, due to legacy reasons, the property validation did not
require a contents property and allowed the image_name to be either
"art" or "boot". Those reasons no longer apply and so this change
requires a contents property and only allows the image_name to be set
to "art" if specified.

Bug: 177892522
Test: m nothing
Change-Id: I8855d6e5365ef0b55490e90e7b6c0081cf070ee5
2021-05-24 16:22:47 +01:00
Paul Duffin
58e0e769c9 Only export boot image files to APEX if they are actually created
Previously, the generation of the rules to create the boot image files
was separate from the code to provide those files to the APEX which
meant that it was possible for the APEX to try and use files that had
no rules to create them.

This changes avoids that by only exporting the files once the rules
have been created.

This necessitated a few changes to tests that were relying on the
previous behavior. Including removing completely the test that used an
image_name: "boot" as that is no longer a valid configuration name as
its functionality has been replaced by platform_bootclasspath.

A follow up change will make the validation of the properties of
bootclasspath_fragment stricter to prevent "boot" being used as the
image_name.

Bug: 177892522
Bug: 188680624
Test: lunch qemu_trusty_arm64-userdebug
      m droid dist
      - verify that before this change it fails and after this change
        it works.
      m com.android.art
      - verify that this change does not change the APEX contents with
        either qemu_trusty_arm64-userdebug or aosp_arm64-userdebug
Change-Id: I0497a151eb0731cbe6a1a7e7bbbb1e4dda75898f
2021-05-24 16:22:43 +01:00
Paul Duffin
32cf58a8fc Support removed API members in modular hidden API processing
Previously, the hidden API flags generated for a bootclasspath_fragment
did not include removed API members. That was because it did not supply
a file containing the dex signatures of the removed API members.

The monolithic hidden API processing uses combined-removed-dex which is
the output of a genrule that takes as input the *removed.txt files from
all the APIs and uses metalava to construct the dex signatures file.
This change does the equivalent for the *removed.txt files for the APIs
provided by a bootclasspath_fragment and then passes them to the rule
that generates the final all-flags.csv.

Bug: 179354495
Test: - Update packages/modules/RuntimeI18N to enable hidden API
        processing.
      m out/soong/hiddenapi/hiddenapi-flags.csv
      - Before this change that fails as the flags generated for the
        i18n-bootclasspath-fragment differ from the monolithic flags.
	After this change that passes.
      - Verify that this change does not change any of the monolithic
        hidden API files.
      m com.android.i18n
      - Verify that the apex file before and after this change are byte
        for byte identical.
Change-Id: I6a21edb8a5231666e3f35b2c99a8687f36dd98fd
2021-05-24 15:05:21 +01:00
Paul Duffin
f1b358cb57 Support hidden API processing for fragments with dependencies
Previously, a bootclasspath_fragment that depended on classes provided
by another bootclasspath_fragment did not support hidden API processing
as it would not supply information about those dependencies.

This change adds support for that as follows. Each fragment:
1. Exports the transitive sets of stub dex jars for each of the public,
   system, test and core_platform APIs (where relevant).
2. Adds dependencies onto its dependent fragments.
3. Retrieves the API stubs dex jars from its dependent fragments and
   passes them to the "hiddenapi list" tool which will use them to
   resolve dependencies but will not output them to the generated
   flags.

Once the flags are generated the existing encoding functionality
encodes the flags into the dex files of the bootclasspath_fragment's
content modules which are then packaged into the apex.

Bug: 179354495
Test: m com.android.sdkext
      - verify that this does not change the contents of the apex files
Change-Id: I3e82a6dbb437f1e417e5d7e25aeb212e378603d0
2021-05-24 14:56:10 +01:00
satayev
703c67a0a9 Populate individual systemserverclasspath_fragments' proto configs.
Bug: 180105615
Test: m && launch_cvd; atest CtsClasspathsTestCases
Change-Id: Ic0ae897e9baca75dcc022d84656496563645e74e
2021-05-24 13:41:20 +01:00