Commit Graph

3402 Commits

Author SHA1 Message Date
Paul Duffin
9d67ca6ab7 Allow dex jars from prebuilt_apex to be used by hiddenapi
Invokes hiddenAPIExtractInformation() on the dex jar provided by the
deapexer (on behalf of prebuilt_apex) so that hiddenAPI can extract the
information it needs, if anything, from the dex file (and accompanying
classes implementation file).

The dex file provided by deapexer has already had the hiddenapi
information encoded into it so it does not need to do that again.

This change adds the primary parameter to hiddenAPIExtractInformation()
and checks it (and also the hiddenAPI.active property) before it does
anything. That ensures that it behaves correctly when called directly
as well as when called from hiddenAPIExtractAndEncode().

Bug: 178361284
Test: m droid
      Verified that hiddenapi files (both aggregated ones and for the
      individual modules) are not affected by this change.
      Also verified that the hiddenapi files created when using the
      prebuilts (using SOONG_CONFIG_art_module_source_build=false) are
      the same as when using the source. There is a slight difference
      in the order but otherwise identical.
Change-Id: I7abb63fd310bb94787ab7f4821e5fd283dc03046
2021-02-08 19:10:50 +00:00
Paul Duffin
4fd997bc13 Refactor the hiddenAPI() method for reusability
A follow up change needs to be able to contribute to the information
the hiddenapi process collates without having a dex file encoded. This
change pushes all the functionality related to information gathering
into the hiddenAPIGenerateCSV() method and then renames it and the
hiddenAPI() method to make it clearer what they do.

Bug: 178361284
Test: m droid
      Verified that hiddenapi files (both aggregated ones and for the
      individual modules) are not affected by this change.
Change-Id: I04417720216a0fbadcd88e6185e7de6570af6216
2021-02-08 19:10:50 +00:00
Paul Duffin
4103e92c4b Extract initHiddenAPI() from hiddenapi()
Previously, the hiddenapi() method combined both checks to determine
whether a module contributed to/was modified by the hiddenapi process
and logic to create ninja rules to perform those tasks. This change
separates the former out into a new initHiddenAPI() method.

The main purpose of this is to simplify the process of allowing the
CSV generation to be separated from the encoding. That is required
because when a java_import retrieves its dex file from the apex it
has already been encoded.

The initHiddenAPI() method is only called for java.Library (and
indirectly for java.SdkLibrary) and java.Import modules which means
that the hiddenapi() method does nothing for any other module type.
That is consistent with the previous behaviour because while the
hiddenapi() method is called for other module types they fail because
the boot image jars only support java_library, java_sdk_library,
and java_import at the moment. While it will need to support
java_sdk_library_import once any of the libraries that are currently
provided as java_sdk_library modules switches to providing prebuilts
that is outside the scope of this work.

Bug: 178361284
Test: m droid
      Verified that hiddenapi files (both aggregated ones and for the
      individual modules) are not affected by this change.
Change-Id: Iaa91e0a8e2bffec03296dd894e9662556f4464c0
2021-02-08 19:10:50 +00:00
Paul Duffin
219b177279 Remove duplicate entries from hiddenapi files
Prevents the hiddenapi files index and metadata files from including
duplicate entries.

Bug: 178361284
Test: m droid
      Compared hiddenapi files before and after change
      Verified that the only differences were due to the removal of duplicates
      Used sort -u to remove duplicates from before file and compared with after file
Change-Id: I0b9e420aec42cdcf4584099a7ffd2ed9c3715d6d
2021-02-07 10:58:24 +00:00
Paul Duffin
01289a2407 Add test for hiddenAPI index file generation
The index file and the metadata file both currently include duplicate
entries due to them including both the <x> and <x>.impl libraries
created by java_sdk_library in their inputs, plus including both source
and prebuilt versions of the same named module. This change adds a test
to illustrate that behavior. A follow up change will correct the problem
and update the test accordingly.

This change only adds a test for the index file because the metadata
file depends on files from frameworks/base which makes it difficult to
test. Bug 177317659 will fix that and allow the metadata file
generation to be tested too.

Bug: 178361284
Test: m nothing
Change-Id: I33921d7267c9f4bb42726343d73f8a396d536aaa
2021-02-07 10:58:24 +00:00
Paul Duffin
254aee4843 Merge changes I9e94662d,I4b4e0dc8
* changes:
  Export implementation class jars for java_boot_libs
  Improve module filtering in hiddenapi stubFlagsRule()
2021-02-07 10:52:01 +00:00
Liz Kammer
b7eab01167 Merge "bp2build: convert paths/module refs to Bazel label" 2021-02-05 13:39:08 +00:00
Paul Duffin
22ff0aaf51 Export implementation class jars for java_boot_libs
Hiddenapi processing currently requires access to the class
implementation jars for libraries on the bootclasspath which means that
they need to be provided as part of the prebuilts. This change modifies
the java_boot_libs property on the sdk to make those files available.

Modularization of the hiddenapi processing will hopefully remove the
need for these to be exported so this should be temporary.

Bug: 178361284
Test: m art-module-sdk
      check generated snapshot zip contains implementation jars
Change-Id: I9e94662dddb0ddb85a477ae6d27e533085147e88
2021-02-05 13:35:25 +00:00
Paul Duffin
dd63d6d7bd Improve module filtering in hiddenapi stubFlagsRule()
Previously, it ignored any module that was not a platform apex variant
however that required every module that was referenced from the boot
jars to have a platform variant which is not the case when building
from prebuilts.

This change switches it to explicitly check that the variant is for
either the appropriate apex or the platform depending on what is
configured in the BootJars or UpdatableBootJars.

It partially duplicates some logic from the getBootImageJar() function.
It intentionally does not refactor the getBootImageJar() to allow for
reuse because coupling the hiddenapi and dexpreopt logic would make
refactoring either of them more difficult. Any duplicated code will be
deduped once they have both been refactored.

Bug: 178361284
Test: m droid
Change-Id: I4b4e0dc8ee40c1ba1713d7ef72df13d175e84af6
2021-02-05 13:33:25 +00:00
Ulyana Trafimovich
9b91082452 Merge "Don't write nonexistent DEX file paths to dexpreopt.config files." 2021-02-05 11:10:03 +00:00
Paul Duffin
b8887127ba Merge "Detect empty apex in ConfiguredJarList" 2021-02-05 09:30:15 +00:00
Liz Kammer
356f7d45c1 bp2build: convert paths/module refs to Bazel label
This currently expands all globs, still need to support converting glob
syntax.

Test: go build_conversion_test
Test: GENERATE_BAZEL_FILES=true m nothing
Test: m nothing
Bug: 165114590
Change-Id: If7b26e8e663d17566fad9614ca87a8da1f095284
2021-02-04 13:45:56 -05:00
Jiyong Park
cdd9b8392d data dependency of java_test_host can be arch-specific
This allows java_test_host to have arch-specific data files. Currently,
VirtualizationHostTestCases needs this to have arch-specific kernels as
its data.

Bug: N/A
Test: m VirtualizationHostTestCases
Change-Id: Ib5680ef48b613955ad92c9bc212184ac75900d3e
2021-02-05 00:29:47 +09:00
Ulya Trafimovich
c0f64799da Don't write nonexistent DEX file paths to dexpreopt.config files.
A path to an nonexistent file caused dex2oat to write an empty path in
the class loader context stored inside of the OAT/ODEX file, which
resulted in class loader context mistmatch and a failure to load
dexpreopt code on device.

An example of a faulty class loader context for GoogleDialer, can be
seen either with oatdump or by grepping logcat for class loader context
mimatch messages (the first empty PCL[] is expected, note the second
empty PCL[] in the subcontext for shared libraries):

  PCL[]{PCL[]#PCL[/system/framework/org.apache.http.legacy.jar*22205474]

This CL removes the nonexistent path from the generated dexpreopt.config
file, which works in conjunction with a change in makefiles that stops
using the removed config field. This is possible because makefiles can
access DEX files using a module name and "intermediates-dir-for" macro.

Bug: 132357300
Test: lunch cf_x86_64_phone-userdebug && m && launch_cvd \
      adb wait-for-device && adb root && adb logcat \
      | grep -E 'ClassLoaderContext [a-z ]+ mismatch'
      # empty grep output, no errors
Change-Id: I8d16f77c6f76ad6234cc4114ad3545cbe572ef13
2021-02-04 12:14:51 +00:00
Paul Duffin
9c3ac96f1f Detect empty apex in ConfiguredJarList
Previously, ConfiguredJarList would accept an empty apex name,
e.g. ":jar" which makes no sense as every apex has to have a non-empty
name. This change makes an empty apex invalid.

In order to improve the test coverage of the TestConfiguredJarList test
this change also changes the implementation of
CreateTestConfiguredJarList([]string) to marshal the supplied strings
into a json list and then unmarshal into a ConfiguredJarList which more
closely matches how it is used at runtime.

Bug: 178361284
Test: m nothing
Change-Id: I7dfec6b4cc1923aa99746e976da0393922ef0791
2021-02-04 10:07:41 +00:00
Paul Duffin
e1f9b35d92 Merge "Switch hiddenapi to use OutputPath instead of ModuleOutPath" 2021-02-03 10:02:36 +00:00
Tobias Thierer
dadc62cdb4 Merge "go/Android.bp: Clarify sdk_version documentation." 2021-02-02 18:19:38 +00:00
Paul Duffin
612e610063 Switch hiddenapi to use OutputPath instead of ModuleOutPath
In order to allow the hiddenapi code which expects its inputs to be
ModuleOutPath to consume the output from the deapexer module type (used
by prebuilt_apex) this change converts the hiddenapi from using
ModuleOutPath to use OutputPath instead.

This is part of a general cleanup to remove ModuleOutPath that is
separated out in order to avoid having the hiddenapi work dependent
upon and possibly delayed by issues with that general cleanup.

Bug: 178361284
Bug: 179124768
Test: m droid
Change-Id: I890f775bf5e33528dbaa62b95fa4198185ff8bf8
2021-02-02 14:42:05 +00:00
Paul Duffin
8705ba06bf Merge "Move creation of paths into hiddenAPIGenerateCSV" 2021-02-01 17:28:20 +00:00
Paul Duffin
aded43c808 Merge "Fix some minor issues with boot_image" 2021-02-01 10:21:34 +00:00
Paul Duffin
4a838dfb3f Merge "Add boot_images to apex" 2021-02-01 10:21:21 +00:00
Ulyana Trafimovich
a4fce3b488 Merge "Write module dexpreopt.config for Make." 2021-02-01 10:16:57 +00:00
Paul Duffin
5bbfef8718 Fix some minor issues with boot_image
Mistakenly used HostAndDeviceDefault (which builds host and device
variants by default) instead of HostAndDeviceSupported which only
builds a device variant by default.

Moved the test definitions of art and framework boot images from being
defined in all Java related tests into the test where they belong. This
is needed in order to improve the ART and framework boot image specific
testing.

Bug: 177892522
Test: m droid
Change-Id: I16771f09bd789033e18c58ae6dd4b6b9e865d831
2021-01-30 12:57:26 +00:00
Paul Duffin
a1d6025a49 Add boot_images to apex
Previously, the apex module had to hard code behavior specific to the
art apex module in order to include the art boot image. This change
adds support to the apex module to allow the boot images to be
specified per apex.

In combination with a change to add the "art-boot-image" to the ART
apex this allows the custom code for handling the art boot image in
apex to be removed.

That custom apex code also included the logic to ensure that the
GlobalSoongConfig was initialized for use by the dex_bootjars
singleton. That logic has been moved from the APEX to the boot_image
module. That ensures that it will be run if and only if a boot_image
module is present in the checked out repos. So, limited manifest
checkouts which do not contain the art or frameworks/base repos (which
is where the boot_image modules are defined) will not attempt to run
this logic, which would fail because dex2oat would not be present.

Bug: 177892522
Test: m droid
Change-Id: I02d25fbef6e864e31eb5e0f4eb50358c79486db0
2021-01-30 12:45:07 +00:00
Paul Duffin
34982f1096 Move creation of paths into hiddenAPIGenerateCSV
A minor refactoring to simplify the generation of the CSV files.

Test: m nothing
Bug: 178361284
Change-Id: If79d21c7cebc6643a404973d3e0e5f174d7b0bb1
2021-01-29 13:22:31 +00:00
Paul Duffin
ff774a04ad Add documentation for the different files created by hiddenapi
Test: m nothing
Bug: 178361284
Change-Id: Id55646b9d4b7bc1acdb4ed6a6dd4456b746eb54c
2021-01-29 13:19:51 +00:00
Martin Stjernholm
c4e17317d1 Merge "Fix boot jar handling when both source and prebuilt APEXes and modules are present." 2021-01-29 13:15:02 +00:00
Tobias Thierer
1b3e949d14 go/Android.bp: Clarify sdk_version documentation.
The documentation was unclear because there are at least three
different ways (codename, version, API level) to refer to each
version, see https://source.android.com/setup/start/build-numbers
It is further complicated by the fact that this value is taken
as a String even though the API level it refers to is normally
an integer. To disambiguate, this CL adds an example.

Test: Treehugger only.
Change-Id: I40c13104a87d16c84c2098f36f7f63447d3dda4d
2021-01-28 21:55:37 +00:00
Martin Stjernholm
1dc0d6d7f2 Fix boot jar handling when both source and prebuilt APEXes and modules
are present.

1) The boot jar to APEX mapping is maintained by the base names for
both of them. When building with prebuilt modules and APEXes, that
means we need to take care to compare them without regard to any
"prebuilt_" prefixes.

2) VisitAllModules can visit disabled modules and both source and
prebuilt modules, so they need some conditions to skip modules that
aren't applicable for boot jars.

Test: `m droid`
Test: `m droid SOONG_CONFIG_art_module_source_build=false`
  with fresh ART Module prebuilts in place
Bug: 171061220
Change-Id: Iced269d29127bc8b8f9b3171adb60a97d115628b
2021-01-28 20:09:24 +00:00
Anton Hansson
b0d0710f67 Merge "Fix prebuilt_apis creating of "latest" incompatibilities" 2021-01-28 16:48:16 +00:00
Ulya Trafimovich
76b0852a48 Write module dexpreopt.config for Make.
This is needed for Java libraries that are <uses-library> dependencies
of Java libraries and apps defined as Make modules. Each dexpreopted
module in Make generates a dexpreopt.config file, which incorporates
information from its dependencies' dexpreopt.config files. For
dependencies that are Make modules their dexpreopt.config files are
generated by Make, and for Soong modules they are generated by Soong.
Since Soong doesn't know which libraries are used by Make, it generates
build rules for a superset of the necessary libraries.

Bug: 132357300
Test: lunch aosp_cf_x86_phone-userdebug && m
Change-Id: I325b1037658736ee3c02450b08c00eca1a175962
2021-01-28 06:29:13 +00:00
Anton Hansson
20ce41d0b9 Fix prebuilt_apis creating of "latest" incompatibilities
Incompatibilities are a bit special in that we only ever want to use
the "latest" if it is for a non-finalized SDK.

Add a new argument, next_api_dir, and use only those files to create
"latest incompatibilities". Create empty incompatibilities for all
modules not in next_api_dir.

Bug: 178171189
Test: m
Change-Id: I727a52e3a45a5e607c6176d481b40fbf495c6eb0
2021-01-26 18:08:19 +00:00
Paul Duffin
3451e1600f Add boot_image module type
Adds a boot_image module type. Follow on changes will add instances of
boot_image to represent the ART and framework boot images in the art
and frameworks/base repositories respectively.

For the moment they retrieve their associated boot image configuration
from the map returned by genBootImageConfigs() and leave the actual
work of creating the ninja rules to create the boot images to the
dex_bootjars singleton.

Bug: 177892522
Test: m droid
Change-Id: Ib69701de0d24d996bb8e8be7a20b941be907390e
2021-01-26 13:00:07 +00:00
Paul Duffin
4c183c7581 Merge "Make dexpreopt tools available for all java tests" 2021-01-26 12:08:30 +00:00
Paul Duffin
635aa0834f Make dexpreopt tools available for all java tests
Bug: 177892522
Test: m nothing
Change-Id: I594bddae692ef76dc8c3da504934d0151b13d4c9
2021-01-25 19:15:19 +00:00
Anton Hansson
370fd0b050 Minor cleanup in prebuilt_apis
Use ints instead of strings for version comparison.

Bug: 178171189
Test: m
Change-Id: I53d71b138a64b902b3f00adafec5d354630a5e35
2021-01-25 10:58:41 +00:00
Paul Duffin
98325717b1 Merge "Switch dex_bootjars to a singleton module" 2021-01-22 17:42:54 +00:00
Paul Duffin
8479ea296d Merge "Allow createGlobalSoongConfig() to be used from tests" 2021-01-22 17:42:00 +00:00
Paul Duffin
c44075b710 Merge "Add java.RegisterRequiredBuildComponentsForTest function" 2021-01-22 17:41:02 +00:00
Anton Hansson
4de3c42c73 Merge "Add a better error message for missing api files" 2021-01-22 10:31:15 +00:00
Paul Duffin
1ab6186e91 Switch dex_bootjars to a singleton module
The main difference between a singleton and a singleton module that
affects this change is that the latter requires a module to be defined
in order for the singleton to be instantiated. This change defines the
module in both the tests and the standard build.

Bug: 177892522
Test: m droid
Change-Id: Id6669835cf3a2ab7359fa53047eb9042992a9362
2021-01-21 16:49:34 +00:00
Paul Duffin
9f04524673 Allow createGlobalSoongConfig() to be used from tests
Previously, the createGlobalSoongConfig() function was explicitly
prevented from being used in tests because it would fail. However, it
turns out that is no longer the case and it does now work.

That allows the following changes to be made:
* Tests no longer need to use GlobalSoongConfigForTests() to
  prepopulate the cache.
* GlobalSoongConfigForTests() is only needed in the dexpreopt
  package.

Bug: 177892522
Test: m nothing
Change-Id: Ifcbb1a44254c5d2d10c1d02ab23227488d1d1ed1
2021-01-21 15:44:30 +00:00
Paul Duffin
c059c8c9a0 Add java.RegisterRequiredBuildComponentsForTest function
Insulate tests that exercise code in the java package from having to
register the build components provided by the java package by providing
a single function that registers them all. This follows the pattern
currently used in the cc and rust packages.

This change is in preparation for switching the dex_bootjars singleton
from a singleton, which does not require a module definition in order
to be instantiated, to a singleton module which does. That will require
adding a module definition into java.GatherRequiredDepsForTest() and
this change ensures that the required components will have been
registered in every test.

Bug: 177892522
Test: m nothing
Change-Id: I6475db8240894947dd07c89a940a3e4f201aa598
2021-01-21 11:47:32 +00:00
Anton Hansson
e77fccc959 Add a better error message for missing api files
Add some instructions for what to do.

Bug: 177806245
Test: m nothing (with java_sdk_library with no tracking files)
Change-Id: I286018f4d9105d5ade1e76cddb8670d0d097d77d
2021-01-20 19:21:16 +00:00
Nicolas Geoffray
324234bd00 Merge "dexpreopt: Use the dirty-image-objects path from frameworks/base." 2021-01-20 18:13:20 +00:00
Ulyana Trafimovich
0caabb6e27 Merge "Support building boot images with disabled dexpreopt." 2021-01-20 17:46:02 +00:00
Bill Peckham
1c8e5e817e Merge changes from topic "prebuilt-hiddenapi"
* changes:
  Supply LOCAL_SOONG_DEX_JAR for java_import with compile_dex.
  Enable an apex to include a java_import.
  Enable prebuilt hiddenapi CSV files.
2021-01-20 17:43:48 +00:00
Nicolas Geoffray
1086e604b9 dexpreopt: Use the dirty-image-objects path from frameworks/base.
Test: m
Bug: 176171716
Change-Id: Ifbc08ae03e2b1bec7161948970bd197386e3b305
2021-01-20 14:31:32 +00:00
Ulya Trafimovich
a4a1c4ee98 Support building boot images with disabled dexpreopt.
Bug: 176171716
Test: build_mainline_modules.sh and ensure that the ART boot image is
    built: `find out -type f -name '*.art'` is nonempty for every arch.
Change-Id: Ibc45581eef2b205c750a30709780cf659ba7cfa1
2021-01-20 14:03:07 +00:00
JaeMan Park
cdbbb1ea68 Merge "Make TestJavaSdkLibraryEnforce faster" 2021-01-20 00:00:54 +00:00