Commit Graph

3755 Commits

Author SHA1 Message Date
Paul Duffin
3e7fcc3fe7 Extract common stub flags code
The stubFlagsRule does three separate tasks:
1. It computes the set of modules that provide the stubs.
2. It scans all the modules to find the stub modules and retrieves the
   paths to their dex files.
3. It constructs the ninja rule.

Of those three tasks, 1 and 3 will be same for the
platform_bootclasspath. Instead of searching all the modules for the
ones that provide the stubs it will simply add dependencies onto the
stub modules and retrieve the dex file paths from them.

This change extracts tasks 1 and 3 into separate methods for reuse. It
also parameterizes the generation of the ninja rule.

Bug: 179354495
Test: verified that the monolithic out/soong/hiddenapi/... files are
      unchanged by this change
Change-Id: I893845dbddc4b001dfd44d0e0b1c8a31b7f3f89f
2021-04-22 14:23:08 +01:00
Rupert Shuttleworth
dac451715b Merge "Add various test files as testSrcs, so that they run on CI." 2021-04-22 03:08:21 +00:00
Paul Duffin
d3c1513df9 Add tempPathForRestat to improve consistency
Previously, there were two approaches used to construct the temporary
path needed by the commitChangeForRestat method and neither was
suitable for general use. One was:
    android.PathForOutput(ctx, outputPath.Rel()+".tmp")
The other was:
    combinedAidl.ReplaceExtension(ctx, "aidl.tmp")

The first approach would not work if the supplied path had been created
by PathForModuleOut() or similar as it would drop the module directory.
That could lead to the same path being used for multiple rules.

The second approach would not work for files with a different
extension.

The tempPathForRestat combines the two approaches so it can be used
generally with any WritablePath.

Bug: 179354495
Test: verified that the ninja rules that used these files were not
      changed by these changes.
Change-Id: I4439dea0a823512c281eeb1366522fb49dceb4e3
2021-04-21 23:26:57 +01:00
Paul Duffin
eaa432d470 Merge "Skip initHiddenAPI check of prebuilt's boot dex jar for APEX modules" 2021-04-21 17:14:56 +00:00
Rupert Shuttleworth
b7e30760af Add various test files as testSrcs, so that they run on CI.
Test: go test
Test: TH (now that these are hooked up to TH)

Change-Id: Ia199794ef1a64074f3f9159f8c160ba3e45fd181
2021-04-21 11:23:57 -04:00
Paul Duffin
894546d301 Skip initHiddenAPI check of prebuilt's boot dex jar for APEX modules
There is a check in initHiddenAPI that makes sure that if the source
module has been replaced by a prebuilt the prebuilt provides a boot dex
jar. The check was added to try and detect an issue with hidden API
processing early at build time rather than at runtime.

That check relies on the source module having a PrebuiltDepTag
dependency on the prebuilt module that is preferred and will be used by
hidden API processing. That is true for modules that are not in an APEX
but doesn't work for modules that are in an APEX.

The issue is that an APEX variant of a source module depends on the
non-APEX variant of the corresponding prebuilt module. However, that
variant of the prebuilt is not the one that will be used by hidden API
processing; it will use the APEX variant of the prebuilt module which
is the one that has access to the boot dex jar.

That results in the initHiddenAPI check giving a false negative as it
states that the boot dex jar is not available when it may be.

This change simply skips the check is the source module is an APEX
variant. This is a temporary work around as the hidden API processing
is being modularized for each APEX at which point initHiddenAPI will
either be removed entirely or just skipped for APEXes altogether.

Bug: 179354495
Bug: 185828824
Test: m nothing
Change-Id: I2dc3654c9aa476541855b3f0f243a181ddf8d723
2021-04-21 12:53:58 +01:00
Paul Duffin
cc17bfe7de Rename propertyAccessor to propertyValueReader
Bug: 179354495
Test: m nothing
Change-Id: I956e93fdbe43e09b5460337d2eb72084440f14be
2021-04-21 10:14:02 +01:00
Jaewoong Jung
cbb3825d86 Merge changes Ibdeb2e5a,I520a5af4,I0db32bec,Id3ab0f3b
* changes:
  Add lint_project_xml_test.py
  Rename lint-project-xml.py to remove dashes.
  Lint baseline file check in lint-project-xml
  Extract getBaselineFilepath method.
2021-04-20 22:59:38 +00:00
Jaewoong Jung
5a4202575e Rename lint-project-xml.py to remove dashes.
So that a future test can import it without a dirty syntax.

Test: m lint-check
Bug: 182349282
Change-Id: I520a5af49543801ab2a8ee888ff235876546dc74
2021-04-20 07:06:03 -07:00
Paul Duffin
42030eafff Merge "platform_bootclasspath: aggregate hidden API flag files from fragments" 2021-04-20 11:02:23 +00:00
Paul Duffin
cb5d20887b Merge "java_sdk_library: Make dex stub jars available for hiddenapi" 2021-04-20 09:48:53 +00:00
Colin Cross
f8d9c499d4 Reland: Add jni_libs property to java tests
Add jni_libs property to java tests and treat it as test data that
should be copied to the lib or lib64 directory in the test directory.

This relands I3a118b933ab30dcd731c6dc2708da9bc63ab5520 with fixes for
the test on mac.

Fixes: 176593487
Test: java_test.go
Change-Id: I2f8c46643cff7a7ae6dc4d4dbad58f0396e45f09
2021-04-19 10:57:55 -07:00
Jaewoong Jung
302c5b8d80 Extract getBaselineFilepath method.
Test: m nothing
Bug: 182349282
Change-Id: Id3ab0f3b7d398af9dcfd66ee3c0bda64d999178d
2021-04-19 08:54:36 -07:00
Colin Cross
b5fa2646d8 Merge changes Iefe133ce,I893f3dd0,I604a11c9
* changes:
  Fix metalava api baseline update command
  sbox: print failing command line before output
  sbox: best-effort copy output files on failure
2021-04-19 15:34:57 +00:00
Colin Cross
de1b891690 Merge "Revert "Add jni_libs property to java tests"" 2021-04-17 05:17:45 +00:00
Colin Cross
26616d4cfd Revert "Add jni_libs property to java tests"
This reverts commit 246164a055.

Reason for revert: broke the mac build
Bug: 176593487

Change-Id: Id0242e37aa40d84d291bf7b8ee46dfe98c4d5905
2021-04-17 05:17:03 +00:00
Treehugger Robot
aef14db78a Merge changes from topic "tvts-test-host-jni"
* changes:
  Add jni_libs property to java tests
  Add Target to cc.SharedLibraryInfo
2021-04-17 04:35:24 +00:00
Colin Cross
246164a055 Add jni_libs property to java tests
Add jni_libs property to java tests and treat it as test data that
should be copied to the lib or lib64 directory in the test directory.

Fixes: 176593487
Test: java_test.go
Change-Id: I3a118b933ab30dcd731c6dc2708da9bc63ab5520
2021-04-16 21:59:23 +00:00
Colin Cross
63eeda027c Fix metalava api baseline update command
metalava is run inside sbox with a modified $PWD, so putting $PWD in
the output message results in an incorrect path.  It was also always
incorrect when the output directory was an absolute path.  Add a
cd $ANDROID_BUILD_TOP to the command line and use relative paths
instead.

Bug: 185516277
Test: m out/soong/.intermediates/frameworks/base/system-api-stubs-docs-non-updatable/android_common/metalava/api_lint.timestamp with lint error
Change-Id: Iefe133cea4c3a604ecd2b0ea20f4ba14ae13b425
2021-04-16 14:55:50 -07:00
Paul Duffin
1267d875b6 java_sdk_library: Make dex stub jars available for hiddenapi
The hidden API processing needs access to dex jars for the API stubs in
order to determine which dex members are part of an API surface. The
dex stubs used for the monolithic file are provided by normal
java_library modules for legacy reasons. However, the APEXes that
contribute to the bootclasspath, and so need to perform hidden API
processing, typically provide stubs created by a java_sdk_library.

This change adds support to java_sdk_library/_import to make the dex
stub jars available when requested, that involves:
1. Adding compile_dex property to java_sdk_library_import and
   propagating it down the the java_import modules for the stubs. That
   is already handled for java_sdk_library.
2. Propagating the java_sdk_library compile_dex property to the
   java_sdk_library_import in the generated snapshot.
3. Refactoring and wiring to make the dex stubs jar available to other
   parts of Soong.

Bug: 179354495
Test: m nothing
Change-Id: I5895d4f2ba0b684870862b9429b2364865e4afc6
2021-04-16 18:48:20 +01:00
Jeongik Cha
06d6373bbe Merge "Made ManifestPath OptionalPath instead of Path" 2021-04-16 03:40:51 +00:00
Paul Duffin
9b381ef2b8 platform_bootclasspath: aggregate hidden API flag files from fragments
Aggregates hidden API flag files from the bootclasspath_fragments which
will allow the hidden API flag files in frameworks/base/boot/hiddenapi
to be modularized and moved to the appropriate repo.

Bug: 177892522
Test: verified that the out/soong/hiddenapi/... files are unchanged
      by this change
      also verified that changes to the fragment provided files do
      affect the monolithic files.
Change-Id: Ifce14c9ef24c58c7ab1085475d85b61cfbfefecd
2021-04-16 00:29:53 +01:00
Treehugger Robot
107f92efbe Merge "Handle boot jars on /system_ext correctly." 2021-04-15 16:30:03 +00:00
satayev
76f88384c4 Merge "Generate classpaths.proto config for *CLASSPATH variables." 2021-04-15 16:07:13 +00:00
Paul Duffin
d9d1102b35 Merge "Rename hidden API types ..Augmentation.. to ..FlagFile.." 2021-04-15 15:29:18 +00:00
Paul Duffin
bca75b1f77 Merge "Generalize hiddenAPIAugmentationInfo to make it easier to use" 2021-04-15 15:29:07 +00:00
Ulya Trafimovich
e736e13a8d Handle boot jars on /system_ext correctly.
When Soong looks for boot jars among all modules, it applies certain
constraints to each module that looks like a boot jar (e.g. that it
comes from the right apex or platform). Previously these constraints did
not handle boot jars on /system_ext correctly (they were handled like
apex jars, while they should be handled like platform jars).

Bug: 154976937
Test: m nothing (the modified Soong test would fail previously)
Change-Id: I0746a2fd276ab5ef0400340c5b61cf26c2570e5a
2021-04-15 15:07:06 +01:00
Pedro Loureiro
53d69eb028 Merge "Introduce NewApi lint checks" 2021-04-15 11:04:47 +00:00
Paul Duffin
4616977948 Rename hidden API types ..Augmentation.. to ..FlagFile..
Augmentation was too abstract this makes it clearer.

Bug: 177892522
Test: m nothinge
Change-Id: I60ad005e68d9e15b01bcb46bc6a9b7f84d8bfd43
2021-04-15 10:45:39 +01:00
Paul Duffin
e3dc6608cb Generalize hiddenAPIAugmentationInfo to make it easier to use
Previously, each category of flag file had its own property in the
hiddenAPIAugmentationInfo struct that required a lot of repetition
to use. This change moves the flag file specific handling into a
new hiddenAPIFlagFileCategory struct which allows use of the
hiddenAPIAugmentationInfo struct to be parameterized.

Bug: 177892522
Test: verified that the out/soong/hiddenapi/... files are unchanged
      by this change
Change-Id: I4413134c0c9382139bef3813f847e453f426692c
2021-04-15 10:45:39 +01:00
Jeongik Cha
33a3a8182f Made ManifestPath OptionalPath instead of Path
because it is actually an optional value.

Bug: 158843648
Test: m nothing
Change-Id: If323eacf6a7aa83dcf5ce4825ad8cabeeaa97ac9
2021-04-15 09:12:49 +09:00
Paul Duffin
85dee5d8fb Move hidden API metadata file rule to platform_bootclasspath
This change moves the monolithic hidden API index file creation rule
from the hiddenAPISingleton into the platform_bootclasspath.

Bug: 179354495
Test: verified that the out/soong/hiddenapi/... files are unchanged
      by this change
Change-Id: Ib25be3618e30a83dc2929a18062eb58eefdabefb
2021-04-14 18:54:55 +01:00
Paul Duffin
22c743170d Move handling of prebuilt hiddenapi-index.csv to hiddenapi singleton
This change moves the handling of the prebuilt hiddenapi-index.csv file
from the hiddenAPIIndexSingleton and removes that as it no longer
needed.

Bug: 179354495
Test: verified that the out/soong/hiddenapi/... files are unchanged
      by this change
Change-Id: Iab1b39a57f28caa855c48538aa2230795f2c3271
2021-04-14 18:54:45 +01:00
Paul Duffin
00b2bfdea5 Move hidden API index file rule to platform_bootclasspath
This change moves the monolithic hidden API index file creation rule
from the hiddenAPIIndexSingleton into the platform_bootclasspath. It
also moves the corresponding test from java/hiddenapi_singleton_test.go
to java/platform_bootclasspath_test.go.

Bug: 179354495
Test: verified that the out/soong/hiddenapi/... files are unchanged
      by this change
Change-Id: Ia295d0f7ae9b51ea816f16921aa42339ed91704e
2021-04-14 18:54:29 +01:00
Paul Duffin
08abf7274a Merge "Ensure boot jar modules have been initialized properly for hidden API" 2021-04-14 15:53:15 +00:00
Artur Satayev
97259dc625 Generate classpaths.proto config for *CLASSPATH variables.
Instead of embedding "raw" values from PRODUCT_*_CLASSPATH variables
into /etc/classpath, encode them into classpaths.proto binary format.

Existing platform_bootclasspath{} module is used to generate the whole
monolithic config to begin with. Later, the config will be split
among individual bootclasspath_fragment and
systemserverclasspath_fragment modules.

Bug: 180105615
Test: m && launch_cvd
Change-Id: Ia66f521f8976ff78a62ecf91131d26db21064de7
2021-04-14 15:03:23 +01:00
Jiyong Park
f398995a64 Merge "SdkSpec is fully using ApiLevel" 2021-04-14 13:27:05 +00:00
Anton Hansson
e9d2f9fcda Merge "Remove special case code for obsolete files" 2021-04-14 09:22:51 +00:00
Paul Duffin
1ba246732d Ensure boot jar modules have been initialized properly for hidden API
Checks to make sure that every module that is part of the platform
bootclasspath has been initialized property for hidden API processing.

Bug: 177892522
Test: verified that the out/soong/hiddenapi/... files are unchanged
      by this change
Change-Id: Ic4368963f2011784b537b8aebf5ef97ea22b2db5
2021-04-14 09:08:02 +01:00
Paul Duffin
8f146b99e6 prebuilt_apex created ApexInfo must not include prebuilt_ prefix
As part of the work to modularize the hiddenAPI processing the
generation of the monolithic hidden API index file needs to be moved
to the platform_bootclasspath module type. Doing that broke the
TestBootDexJarsFromSourcesAndPrebuilts tests which checks the inputs to
the rule that creates that file. Fixing that required added a
platform_bootclasspath module to the test fixture for those tests which
highlighted an issue with the prebuilt_apex module.

Previously, when the prebuilt_apex created apex variants it would use
its own name as the apex variant name, even when that name included the
prebuilt_ prefix. That broke the platform_bootclasspath logic as it was
looking for apex variants for "myapex" but the only ones available were
"prebuilt_myapex".

This change ensures that it always uses the unprefixed name and fixes
the TestNoUpdatableJarsInBootImage to match. This also adds some
improved error reporting in platform_bootclasspath which helped debug
this problem.

Bug: 177892522
Test: m nothing
Change-Id: I3e88b5cec767f77dcc0e94b3ae38b499d07eadf0
2021-04-13 19:15:27 +01:00
Anton Hansson
fa5e6b50aa Remove special case code for obsolete files
incompatibilities.txt doesn't exist anymore, only
"<module>-incompatibilities.txt". Clean up the associated logic that
mapped incompatibilities.txt to android-incompatibilities.txt.

Bug: 178171189
Test: m
Change-Id: I61eeb5dcc82770fa2af356bad11b7205551d6d83
2021-04-13 19:09:48 +01:00
Pedro Loureiro
f4a88b1633 Introduce NewApi lint checks
We are enabling NewApi lint check where the min sdk != compile sdk.
At the same time, we are introducing baseline files for existing
projects that fail this check in order to keep the build running.

At the very least we stop introducing new problems and teams might
realise of risks in their projects they were not aware of.

Bug: 150847901

Test: m lint-check
Change-Id: Icfa5eb98cc6b6708149f0c52fac8fc1440d9c3b0
Merged-In: Icfa5eb98cc6b6708149f0c52fac8fc1440d9c3b0
2021-04-13 15:19:02 +00:00
Paul Duffin
60264a0b99 Improve realism of boot jar tests
Boot jars, updatable boot jars and art apex jars are part of two
separate but related configuration objects, the main Config struct
(actually the nested productVariables struct) and the dexpreopt
specific GlobalConfig. The fields in both are initialized from the same
data in the make config files but handled separately.

Previously each test that used one of the configuration objects would
generally just initialize the one it used. That would make the test
sensitive to the specific configuration object that was used. A
refactoring that change the code from using one configuration object to
the other would cause the test to fail.

Also, some tests would inadvertently create invalid configurations by
setting ArtApexJars without also setting BootJars. While the ability to
create invalid configurations is useful (and there are some tests that
exist to verify the behavior in that case) most tests should not be
using them.

This change simplifies the configuration of the tests and improves
their realism by:
1. Providing a new FixtureConfigureBootJars method that takes a set of
   boot jars and sets ArtApexJars, and BootJars in the
   dexpreopt.GlobalConfig and BootJars in the product variables too.
2. Providing a new FixtureConfigureUpdatableBootJars method that takes
   a set of boot jars and sets UpdatableBootJars in both the
   dexpreopt.GlobalConfig and productVariables.
3. Migrating existing tests to use these new methods.

Some tests still use the dexpreopt.FixtureSet...Jars() methods directly,
generally to create invalid configurations.

Bug: 177892522
Test: m nothing
Change-Id: I4d8f0b9762cfcc7ae6383bef08563d7c3fa13955
2021-04-13 14:48:34 +01:00
Paul Duffin
0b659866e8 Support UNSAFE_DISABLE_HIDDENAPI_FLAGS in platform_bootclasspath
Previously attempting to use m UNSAFE_DISABLE_HIDDENAPI_FLAGS would
cause a build failure.

It is necessary when UNSAFE_DISABLE_HIDDENAPI_FLAGS=true to create
empty files for the monolithic files so that those usages which are not
protected by the build flag, e.g. in dists or CTS tests like
CtsHiddenApiBlocklistTestApiTestCases do not cause ninja failures due
to reliance on a file that does not exist and has no rule to create it.

Bug: 177892522
Test: m nothing
      m UNSAFE_DISABLE_HIDDENAPI_FLAGS=true CtsHiddenApiBlocklistTestApiTestCases
Change-Id: I05d0b3680dbc0b5b7c7c0d792550faf848829647
2021-04-13 14:19:51 +01:00
Paul Duffin
90b8ad38e0 Remove noop code from generateHiddenAPIBuildActions
Change https://r.android.com/1673266 introduced some code that does
nothing. This change removes that.

Bug: 177892522
Test: m nothing
Change-Id: I349b17c7fe7e2f68a807dab43a4300da7fe0ad4d
2021-04-13 14:18:34 +01:00
Paul Duffin
99e4fc13bf Merge "Remove unused hiddenapi_flags module type" 2021-04-13 09:57:31 +00:00
Paul Duffin
db173510d3 Merge "Export monolithic hidden API files from platform_bootclasspath" 2021-04-13 09:57:12 +00:00
Artur Satayev
eabf2c175f Introduce classpath_fragment.go.
A skeleton of a classpath fragment's interface and base to be used by
any modules that provide entries for *CLASSPATH variables at runtime.

In follow up changes, this would be extended to generate
packages/modules/SdkExtensions/proto/classpaths.proto configs for such
modules to be bundled into system and individual apex binaries.

Bug: 180105615
Test: m
Change-Id: I2df550862e97222c5650c4d0480c90231fd78ef0
2021-04-13 08:58:18 +01:00
Paul Duffin
082afbb656 Remove unused hiddenapi_flags module type
Bug: 177892522
Test: m nothing
Change-Id: I604a86a25bbc722ec718b0b0272d0481218d3cd9
2021-04-13 00:23:55 +01:00
Paul Duffin
6a766453fd Export monolithic hidden API files from platform_bootclasspath
Makes the monolithic hidden API files accessible from the
platform_bootclasspath so they can be output to the dist build target
and used by other modules, e.g. by doing something like this:
  java_resources: [
    ":platform-bootclasspath{hiddenapi-flags.csv}",
  ],

It makes the paths relative to the out/soong/hiddenapi directory rather
than the out/soong directory to make them easier to use in the
java_resources property without changing the structure of the APK.
Without that attempting to use them in a java_resources property will
result in them being copied to a hiddenapi/ within the APK instead of
being used at the top level as existing APKs like
CtsHiddenApiBlocklistTestApiTestCases expect.

Bug: 177892522
Test: m nothing
Change-Id: I829412fc7d25411e0c2e0713d0d219a18f4af2ee
2021-04-13 00:23:55 +01:00