Commit Graph

4371 Commits

Author SHA1 Message Date
satayev
28e5eda261 Merge "Make sure that classpath fragment contents appear in make vars." 2021-08-24 13:29:58 +00:00
Ulya Trafimovich
0b1c70efbc Don't add uses_libs/optional_uses_libs to the manifest_fixer.
These properties specify libraries that cannot be implicitly inferred by
Soong. If these properties are added to Android.bp, this can only be for
the reason that there is a <uses-library> tag in the manifest which is
unknown to the build system. Adding them to the manifest_fixer doesn't
make sense: if they are not in the manifest, they should be removed from
Android.bp as well.

Bug: 132357300
Test: $ lunch aosp_cf_x86_64_phone-userdebug && m && launch_cvd
      $ adb wait-for-device && adb root && adb logcat \
        | grep -E 'ClassLoaderContext [a-z ]+ mismatch'
        # empty grep output, no errors
Change-Id: Ic6eb5268a954ef3be7f06a181ec72af99000c547
2021-08-20 15:54:42 +01:00
Ulyana Trafimovich
abcce6f1c3 Merge "Propagate <uses-library> dependencies from java_library to apps." 2021-08-19 09:14:33 +00:00
Liz Kammer
380dbb9327 Merge "Don't create a new module for bp2build conversion." 2021-08-18 19:28:20 +00:00
Ulya Trafimovich
e4432874a9 Propagate <uses-library> dependencies from java_library to apps.
Automatically add `uses_libs`/`optional_uses_libs` properties specified
in `java_library` modules to the apps that transitively depend on these
libraries.

Note that a library may choose to specify its <uses-library> deps as
either optional or required. If this is not the right choice for an app,
it should be able to override it by specifying its own `uses_libs`/
`optional_uses_libs` properties.

Add Soong tests to cover the new cases.

Bug: 196377222
Test: lunch aosp_cf_x86_64_phone-userdebug && m
Change-Id: I601f3e2830ed9699b78d911966ee70812d32b4f6
2021-08-18 16:57:11 +01:00
Liz Kammer
2ada09a546 Don't create a new module for bp2build conversion.
A performance improvement for bp2build as Blueprint/Soong no longer have
the overhead of additional modules. The creation of these modules
results in:
* traversal of additional modules for each subsequent mutator
* synchronization over a go channel to collect newly created modules:
https://cs.android.com/android/platform/superproject/+/master:build/blueprint/context.go;l=2594,2600;drc=1602226f23181b8c3fbfcaf3358f0297e839d7d3

We avoid both of these by storing the information directly in the
underlying module.

Also as a fringe benefit, removes some necessary boilerplate for
conversion.

For benchmarks, reduces runtime ~1% for 1% converted, ~24% for 100%
converted. See more: go/benchmarks-for-https:-r.android.com-1792714

Test: ran benchmarks/tests in bp2build
Test: build/bazel/ci/bp2build.sh
Change-Id: Ie9273b8cbab5bc6edac1728067ce184382feb211
2021-08-17 15:57:09 -04:00
Ulya Trafimovich
42c7f0d173 Propagate <uses-library> dependencies from android_library to apps.
Automatically add `uses_libs`/`optional_uses_libs` properties specified
in `android_library` modules to the apps that transitively depend on
these libraries.

Note that a library may choose to specify its <uses-library> deps as
either optional or required. If this is not the right choice for an app,
it should be able to override it by specifying its own `uses_libs`/
`optional_uses_libs` properties.

Bug: 196377222
Test: lunch aosp_cf_x86_64_phone-userdebug && m
Change-Id: I1e36c56719c0023dcd25fa6aeb92016471702b9f
2021-08-17 16:58:20 +01:00
satayev
d34eb0c4a6 Make sure that classpath fragment contents appear in make vars.
The source of truth for what jars are in BOOTCLASSPATH and
SYSTEMSERVERCLASSPATH is make; adding a classpath fragment in soong
with a new jar in the contents without having it declared in make
is wrong (excluding test apexes).

Bug: 191369843
Test: m nothing
Change-Id: Ifd313776ee7ad206031244534ed3870126e4f835
2021-08-16 12:00:25 +01:00
Aurimas Liutikas
15c82a10de Merge "Fix the way manifest fixer detects optional <uses-library> entries." 2021-08-13 21:00:35 +00:00
Ulya Trafimovich
fc0f6e34ce Fix the way manifest fixer detects optional <uses-library> entries.
Previously manifest_fixer used a naive way to distiniguish optional libs
from required ones: it checked if a library is on the list of optional
compatibility libraries. This works for compatibility libs, but not for
other libs.

Now we properly track optionality through all stages of the build,
starting with the addition of the library as a dependency (here's where
the `uses_libs`/`optional_uses_libs` distinction kicks in), store it in
dependency tag and propagate to class loader context, and from there to
the manifest_fixer.

The tests have been updated accordingly.

Bug: 196377222
Test: lunch bertha_x86_64-userdebug && m droid dist cts mts
Change-Id: I3631ce59ebe47116ce7a9b3d33a86f636846ef0f
2021-08-13 16:10:42 +01:00
Paul Duffin
1f0cc77010 Merge "Filter blocked entries from modular flag files" 2021-08-11 18:36:37 +00:00
Paul Duffin
280bae6d20 Filter blocked entries from modular flag files
Previously, the sdk snapshot would include all the entries from the
stub-flags.csv and all-flags.csv modular files generated by a single
bootclasspath_fragment. That included implementation details, i.e.
class members that are not part of a stable API or the hidden API which
meant that the sdk snapshots were implementation dependent.

This change removes the implementation details from the modular flag
files, i.e. those entries that are only marked as "blocked". When
comparing the files against the corresponding subset of the monolithic
files it assumes that any entries missing from the modular flag files
are blocked.

Bug: 194063708
Test: atest --host verify_overlaps_test signature_patterns_test
      m out/soong/hiddenapi/hiddenapi-flags.csv
      - manually change files to cause difference in flags to check
        that it detects the differences.
Change-Id: I6b67b2253cf029d6830b58a06ebb0c8fcaa0dd71
2021-08-11 17:29:47 +01:00
satayev
721f42d12a Merge "Fix tests to use correct fixtures for configuring (Apex)BootJars." 2021-08-11 16:13:20 +00:00
Colin Cross
8355c1524f Collect lint reports from apps in apexes
Lint reports were only being collected from jars in apexes, not
apps.

Bug: 188772607
Test: mainline_modules_arm64 build
Change-Id: Ie84819343f8b8ab3f43496bbc9b7876d0d0d8e70
2021-08-10 19:25:59 -07:00
satayev
abcd59731e Fix tests to use correct fixtures for configuring (Apex)BootJars.
This is in preparation to r.android.com/1740313 where setting correct
variables would be enforced (i.e. apex and non-apex boot jars must be in
config.ApexBootJars and config.BootJars correspondingly).

Bug: 191369843
Test: m nothing
Change-Id: Ic86680c1f7af53d229083b2cc58beb3ceccb4b6a
2021-08-10 15:35:52 +01:00
Colin Cross
795a9f99f4 Merge "Reland: Pass jars with resources to R8" 2021-08-06 18:33:23 +00:00
Colin Cross
03349188bc Merge changes Ia0c3826a,I92ad6635
* changes:
  Revert "Time out and dump stacks from R8 processes after 30 minutes"
  Add more debugging to run_with_timeout
2021-08-06 17:47:49 +00:00
Paul Duffin
8d007e9919 Export signature patterns to sdk snapshot
Bug: 194063708
Test: atest --host verify_overlaps_test signature_patterns_test
      m out/soong/hiddenapi/hiddenapi-flags.csv
      - manually change files to cause difference in flags to check
        that it detects the differences.
Change-Id: Ic819def9b0c41e3b22fb98582cd39f6d27080a32
2021-08-06 14:11:14 +01:00
Paul Duffin
67b9d61ac2 Separate creation of signature patterns from overlap checking
Previously, the signatures used to select the subset of the monolithic
flags were simply the signatures read from the modular flags file. This
change moves the creation of the signature list into a separate script
that outputs the signatures to a file and then passes the path through
Soong from the bootclasspath_fragment modules that create it to the
platform_bootclasspath module that uses it to compare the modular
flags against the monolithic flags.

Currently, the signatures are the full signatures but follow up changes
will replace them with patterns (hence the name) that avoids having to
include implementation details in the hidden API flags that are output
as part of a bootclasspath_fragment's snapshot.

This change moves the stub flags related code next to the all flags
related code as they are treated in a similar way.

Bug: 194063708
Test: atest --host verify_overlaps_test signature_patterns_test
      m out/soong/hiddenapi/hiddenapi-flags.csv
      - manually change files to cause difference in flags to check
        that it detects the differences.
Change-Id: I2855bf6d05c91b8a09591664185750361c7e644f
2021-08-06 13:40:34 +01:00
Colin Cross
a79a52c7c4 Reland: Pass jars with resources to R8
R8 will parse proguard files out of resources in injars.  Use the jar
with resources instead of the classes jar so that R8 can see the
proguard files.

R8 fails when an input jar contains dex files, so filter out dex files
from the input jar before passing it to r8.

This relands Ibb870ee9c70470336f542a3b7542dab86716dbf8 with a fix to
move the temporary input jar outside the output directory.  Putting it
in the output directory interacted poorly with RBE in local comparison
mode, which incorrectly deleted the output directory.

Bug: 195558228
Test: m checkbuild
Change-Id: Id4b091c3bd72d4c2904883e8793ec4b9499e4b78
2021-08-05 23:16:38 -07:00
Colin Cross
a832a04db2 Revert "Time out and dump stacks from R8 processes after 30 minutes"
This reverts commit 29c294b2cf.

The deadlock in R8 was fixed, and the timeout causes problems when
RBE is enabled and the R8 action gets stuck in a queue for 30
minutes.

Test: checkbuild
Change-Id: Ia0c3826aea3288e9db3bdbd4955e1e16c0a38fe3
2021-08-05 17:04:06 -07:00
Colin Cross
93fb53f5d0 Merge "Revert "Pass jars with resources to R8"" 2021-08-05 15:25:25 +00:00
Colin Cross
1d73c7316c Revert "Pass jars with resources to R8"
This reverts commit 05ca7c8769.

Reason for revert: b/195644067

Flaky failure with "missing in.jar" error.

Change-Id: Id2d6452de1e43951bc58f0bfb8e20d5ad65ce830
2021-08-05 15:16:11 +00:00
Colin Cross
ac33aac0aa Merge "Pass jars with resources to R8" 2021-08-05 01:41:32 +00:00
Colin Cross
05ca7c8769 Pass jars with resources to R8
R8 will parse proguard files out of resources in injars.  Use the jar
with resources instead of the classes jar so that R8 can see the
proguard files.

R8 fails when an input jar contains dex files, so filter out dex files
from the input jar before passing it to r8.

Bug: 195558228
Test: m checkbuild
Change-Id: Ibb870ee9c70470336f542a3b7542dab86716dbf8
2021-08-04 14:57:45 -07:00
Treehugger Robot
8662a3793b Merge "Update documentation for genrule and sdk_library" 2021-08-04 00:14:31 +00:00
Spandan Das
93e95992a4 Update documentation for genrule and sdk_library
1. Documentation for genrule and sdk_library was missing a period, which
has been added now. For
genrule, Before:
https://android-build.googleplex.com/builds/submitted/7594870/linux/latest/view/genrule.html,
After:
https://x20web.corp.google.com/users/sp/spandandas/soong_docs/genrule.html

2. Documentation for genrule was missing definition of $(locations),
which has been added now

Bug: 194231221
Test: m soong_docs
Change-Id: Ia367f13055c110d92a491a9c1b1af4113f66b296
2021-07-30 00:25:20 +00:00
satayev
0d8c497535 Merge "Rename UpdatableSystemServerJars to ApexSystemServerJars." 2021-07-29 11:23:43 +00:00
satayev
492b17d533 Rename UpdatableSystemServerJars to ApexSystemServerJars.
This is consistent with naming for boot jars, e.g. ApexBootJars.

Bug: 191127295
Test: presubmit
Change-Id: I6e8828d55ac86b7f5260ed7f8d4eca1c0b3dced8
2021-07-28 14:04:49 +01:00
Rupert Shuttleworth
5c4881c8ec Add initial bp2build support for converting android_app_certificate modules.
Test: Added unit test.

Change-Id: Ib5bf1711eca5be772e1bed73663dcb86252e0b4a
2021-07-28 06:51:07 -04:00
Rupert Shuttleworth
6e4950a061 Add some more properties to the bp2build APEX converter (second try)
In particular:

- AndroidManifest
- file_contexts
- key
- certificate
- min_sdk_version
- updatable
- installable
- native_shared_libs
- binaries

Test: Updated unit test

Change-Id: I1c6e8c4b6b24ce487f64e5d37bd594dbb000fe6f
2021-07-27 05:02:50 -04:00
Rupert Shuttleworth
d0e6ad1ab3 Merge "Revert "Add some more properties to the bp2build APEX converter."" 2021-07-27 02:56:56 +00:00
Rupert Shuttleworth
0358478150 Revert "Add some more properties to the bp2build APEX converter."
This reverts commit 69bf4c0d60.

Reason for revert: Possibly broke CI (https://android-build.googleplex.com/builds/quarterdeck?branch=aosp-master-bazel&target=bp2build-incremental&lkgb=7266256&fkbb=7237028), rolling back just in case...

Change-Id: I14ebac4f47576a73285d1f724ae4499036e8e678
2021-07-27 02:37:32 +00:00
Treehugger Robot
57dd3c40e8 Merge "Add some more properties to the bp2build APEX converter." 2021-07-27 02:17:51 +00:00
satayev
4102c0cc47 Merge "Rename UpdatableBootJars to ApexBootJars." 2021-07-26 17:59:36 +00:00
Rupert Shuttleworth
69bf4c0d60 Add some more properties to the bp2build APEX converter.
In particular:

- AndroidManifest
- file_contexts
- key
- certificate
- min_sdk_version
- updatable
- installable
- native_shared_libs
- binaries

Test: Updated unit test

Change-Id: I9ea13f021d63ec0f75006b3b0b23ec8178c7e9da
2021-07-26 07:32:36 -04:00
Treehugger Robot
04f3f07752 Merge "Disable lint checks which do not apply in the platform" 2021-07-24 16:10:31 +00:00
Tor Norbye
b015aecbfe Disable lint checks which do not apply in the platform
Lint is primarily targeted at Android app developers, and some of the
checks reflect that. Some of these checks do not apply to the platform
code, where they can produce false positives.

This CL adds a number of these to an exclude list passed in to lint.
Also reenables a check previously disabled because of a bug which has
since been fixed, HardcodedDebugMode, and downgrades from error to
warning a few more lint checks which have existing violations in the
codebase when lint is upgraded to the latest version.

Test: Existing
Bug: None
Change-Id: Icbfbcfd51516c43c342e0d3cdb6973278110fd35
2021-07-24 14:48:19 +00:00
Colin Cross
78fd15f7d6 Merge "Replace android.BuildOs with Config.BuildOS" 2021-07-23 00:07:01 +00:00
satayev
d604b210c4 Rename UpdatableBootJars to ApexBootJars.
Note that ART apex boot jars and core-icu4j are exceptions here as they
are not part of ApexBootJars. ART apex boot jars are defined in their
own variable, while core-icu4j is treated as a regular non-updatable
boot jar.

Bug: 191127295
Test: atest CtsClasspathsTestCases
Change-Id: I3cea3d82ef521655a1a5ffa8cae2258ab9d08bfc
2021-07-22 17:35:42 +01:00
Paul Duffin
6e9bc01b97 Remove unnecessary dedup logic
The CreateClasspathElements function makes sure that each standalone
library and bootclasspath_fragment on the platform bootclasspath
appear once and once only and each file extracted is unique to the
module so there is no chance that any duplicate files will be used.

Bug: 194063708
Test: m out/soong/hiddenapi/hiddenapi-flags.csv
      - check that this does not change the contents.
Change-Id: I60c02baa4d179293ee0f305375afaa3a8c98e112
2021-07-20 22:02:44 +01:00
Paul Duffin
ed12c132c3 Remove unnecessary handling of nil paths in HiddenAPIOutput
Previously, bootclasspath_fragment modules could produce
HiddenAPIOutput structs containing nil paths. That is no longer
possible as both source and prebuilt bootclasspath_fragment modules
always provide non-nil paths so this change removes the special
handling.

Bug: 194063708
Test: m out/soong/hiddenapi/hiddenapi-flags.csv
      - check that this does not change the contents.
Change-Id: I15337e146c9e694ce9da06ad4845ac267d6da27a
2021-07-20 22:02:44 +01:00
Paul Duffin
d6a072b39d Remove fallback to classes jars from bootclasspath_fragment
Previously, the hidden API processing performed by the
platform_bootclasspath would try and get flag files from the fragments
it references but would fall back to the classes jars. This change
removes that logic as it is no longer needed as all fragments, both
source and prebuilts, provide flag files and the prebuilts no longer
provide valid classes jars.

Bug: 194063708
Test: m out/soong/hiddenapi/hiddenapi-flags.csv
      - check that this does not change the contents.
Change-Id: Ib867a08508f5a0f8858f6baedebbe99b6a825f80
2021-07-20 22:02:44 +01:00
Treehugger Robot
41461f3660 Merge "Require prebuilt_bootclasspath_fragment to provide hidden API files" 2021-07-20 20:03:40 +00:00
Colin Cross
0c66bc615b Replace android.BuildOs with Config.BuildOS
Replace the android.BuildOs constant with Config.BuildOS so that it
can vary based on the product config.

Bug: 190084016
Test: all Soong tests
Change-Id: Ia67f872d8b2ab788747a22e3a9659dc21c9775cd
2021-07-20 12:46:48 -07:00
Paul Duffin
54e4197da1 Require prebuilt_bootclasspath_fragment to provide hidden API files
Previously, the prebuilt_bootclasspath_fragment was not required to
provide hidden API files as not all usages of that module provided
them. That should no longer be the case so this change makes the
prebuilt_bootclasspath_fragment stricter and so now always require
them.

Bug: 194063708
Test: m out/soong/hiddenapi/hiddenapi-flags.csv
      - check that this does not change the contents.
Change-Id: I3f19a1b4246f09a927c0f6b0f41b8678150d6cc2
2021-07-20 19:18:13 +01:00
Ulyana Trafimovich
c2427ea290 Merge "Remove OptionalImplicitSdkLibrary() function (it's not used anymore)." 2021-07-19 10:38:45 +00:00
Treehugger Robot
52cc87959b Merge "Revert "Export implementation class jars for java_boot_libs"" 2021-07-19 09:49:17 +00:00
Treehugger Robot
452366ab0c Merge "Make bootclasspath_fragments always perform hidden API processing" 2021-07-19 09:48:07 +00:00
Paul Duffin
393bffee78 Merge "Avoid permitted_packages related build failures in unbundled builds" 2021-07-16 19:06:20 +00:00