Commit Graph

3511 Commits

Author SHA1 Message Date
Ulya Trafimovich
a8c28e27bc Do not add dependencies of shared SDK libraries to manifest_fixer.
Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Bug: 168686456
Change-Id: Ibd9742684fa6a8f1353ca0e513f7fa814a6ec9fc
2020-11-16 14:58:11 +00:00
Ulya Trafimovich
18554243de Add nested class loader subcontext at the proper hierarchy level.
When adding a subcontext in a class loader context tree, there are two
possible cases: 1) the root of the subcontext is itself a <uses-library>
and should be present as a node in the tree, or 2) the root is not a
<uses-library>, but some of its dependencies are -- in that case they
should be disconnected from the root, and the resulting forrest should
be added at the top-level.

Example:

  1) C is a <uses-library>:

     A
     ├── B
     └── C
         ├── D
         └── E
             └── F

  2) C is not a <uses-library>:

     A
     ├── B
     ├── D
     └── E
         └── F

Before the patch subcontexts for transitive dependencies were added
before the subcontext for the direct dependency (even if it was a
<uses-library>, resulting in case-2 hierarchy when case-1 should have
been used. Previosuly this didn't matter because class loader context
was a flat set of libraries, but now it matters because class loader
context is a tree.

This patch changes the order in which libraries are added, so that
direct dependencies are added before transitive ones. The context adding
method now accepts an "implicit root" parameter, so that when adding
transitive dependencies it can check if the corresponding direct
dependency is a <uses-library> and already present in the context.

Partially constructed class loader context is now propagated top-down
into aapt.buildActions, so that the method can use existing part of the
context to decide where the missing part should be connected.

Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Change-Id: I649aff9e27494306885a4f4fc90226c399636b57
2020-11-16 14:57:05 +00:00
Ulyana Trafimovich
af335a4694 Merge "Rework class loader context implementation." 2020-11-16 14:50:11 +00:00
Colin Cross
cf371cc1f7 Replace android.WriteFile rule with android.WriteFileRule
The android.WriteFile rule takes careful escaping to produce the
right contents.  Wrap it in an android.WriteFileRule that handles
the escaping.

Test: compare all android.WriteFile outputs
Change-Id: If71a5843af47a37ca61714e1a1ebb32d08536c31
2020-11-14 16:26:00 -08:00
Colin Cross
62a0cfd054 Annotate dependency tags for dependencies of installed files
Soong currently assumes that installed files should depend on
installed files of all transitive dependencies, which results
in extra installed file dependencies through genrules, static
libs, etc.

Annotate dependency tags for dependencies for which the
installed files are necessary such as shared libraries
and JNI libraries.

This avoids extra installed files, and is also a first step
towards genrules using their own copy of tools instead of
the installed copy.

Bug: 124313442
Test: m checkbuild
Test: java.TestBinary
Test: cc.TestInstallSharedLibs
Test: deptag_test.go
Change-Id: Ic22603a5c0718b5a21686672a7471f952b4d1017
2020-11-14 16:24:10 -08:00
Treehugger Robot
5789a1b32f Merge "Pass Config to NewTestContext instead of ctx.Register" am: ceed6be00d am: 748ecd135f am: b8316056fe am: 5f3bbae64b
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1480601

Change-Id: Ib69b3a789c2de2c83be8a299eafa0740af306081
2020-11-14 07:18:40 +00:00
Colin Cross
ae8600b507 Pass Config to NewTestContext instead of ctx.Register
Prepare for using Config when adding singletons by passing
Config to NewTestContext and NewContext instead of to ctx.Register.
This will enable a followup change to store SingletonMakeVarsProviders
registered on the Context in the Config, which is necessary to run
multiple tests in parallel without data races.

Test: all soong tests
Change-Id: Id229629a4e42ff4487d317241673837726c075fc
2020-11-12 10:07:49 -08:00
Paul Duffin
c459dc8c02 Merge "Switch boot jars package check to using dex jars" am: 68ecbaf614 am: aa50b67217 am: d54bddbda2 am: cb606c48f2
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1479315

Change-Id: If8e9c34e194fbbab52e33993792ff375bff28e2b
2020-11-11 14:34:30 +00:00
Paul Duffin
68ecbaf614 Merge "Switch boot jars package check to using dex jars" 2020-11-11 12:53:23 +00:00
Mathew Inwood
1ef4ba906a Explicitly tag max-target-o APIs as "lo-prio".
This allows the presubmit checker to explicitly allow certain changes to
these APIs, rather than special casing tha max-target-o list. This in turn
will allow other APIs to be tagged as lo-prio in future to get the same
treatment.

Test: m out/soong/hiddenapi/hiddenapi-flags.csv
Bug: 172993934
Change-Id: Ic7cc0243b2b018c96d420a3f266c4b3a6958c32f
2020-11-11 11:01:07 +00:00
Mathew Inwood
660ed8cb7d Merge "Temporarily move APIs to blocked list." am: 82d5b6726b am: 1917bee275 am: 1c7d02ef58 am: d5afd757e3
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1494179

Change-Id: I44e7a1dbc9a21eafe7614f44e41cc99750c26fe2
2020-11-11 10:46:08 +00:00
Mathew Inwood
82d5b6726b Merge "Temporarily move APIs to blocked list." 2020-11-11 09:13:48 +00:00
Paul Duffin
a975cb25ec Merge "Remove support for droidstubs in sdk/module_exports" am: 99a7622885 am: 14301ccfd1 am: 4fc198029d am: 9f3d5d7d11
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1492857

Change-Id: I1cabd47542a339da717a2d858668f11a4b544ddd
2020-11-10 19:57:18 +00:00
Paul Duffin
2d8e1a7e59 Switch boot jars package check to using dex jars
The switch to use dex jars instead of class jars means that a boot jar
that is defined by a dex_import module will now be checked against the
package_allowed_list.txt so it is possible that it will detect
previously unreported problems.

Test: m check-boot-jars - for failing and passing cases
Bug: 171479578
Bug: 125517186
Change-Id: Ie614898dade0fb43c9418d7afb9138169db6f097
2020-11-10 17:37:43 +00:00
Paul Duffin
99a7622885 Merge "Remove support for droidstubs in sdk/module_exports" 2020-11-10 17:34:00 +00:00
Anton Hansson
45ab2c48f4 Merge "Remove dead code" am: a1b3314286 am: 00d7ec3375 am: ab2ebb04ba am: 989e7e6daa
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1488756

Change-Id: Ided08e856716c584c0e61ccd7c04a4b1b65f565d
2020-11-10 15:35:30 +00:00
Paul Duffin
aa78948155 Merge "Use glob for java_sdk_library_import stub_srcs" am: 37fa489a9e am: 0b578e9241 am: a3f456e3d0 am: 6f1e1ff0a6
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1492856

Change-Id: I4aa7bfcb1c5743013af784a31944a05e143a6134
2020-11-10 15:35:23 +00:00
Anton Hansson
a1b3314286 Merge "Remove dead code" 2020-11-10 14:35:13 +00:00
Paul Duffin
a3cb6cf2e5 Remove support for droidstubs in sdk/module_exports
The droidstubs support in sdk/module_exports was a temporary measure
to work around the fact that some APIs were being defined by direct use
of droidstubs instead of java_sdk_library.

This change removes that support as those APIs have all been switched
from droidstubs to use java_sdk_library so droidstubs support is no
longer needed.

Bug: 168301990
Test: m nothing
Change-Id: I3517bed29b030438a0423a6cb8c248992a988222
2020-11-10 13:55:17 +00:00
Paul Duffin
37fa489a9e Merge "Use glob for java_sdk_library_import stub_srcs" 2020-11-10 13:54:23 +00:00
Mathew Inwood
59093e3769 Temporarily move APIs to blocked list.
Bug: 170729553
Test: m

Change-Id: I62db78941785fd1bd336cfe81c8e9763e08a0eb0
2020-11-10 13:08:58 +00:00
Paul Duffin
7f97957ded Use glob for java_sdk_library_import stub_srcs
Some java_sdk_library modules do not have any stubs for some API
scopes. That results in an empty ".srcjar" being created for them which
ends up not creating a directory for the stubs sources when that
snapshot is unzipped. Previously, that would cause a build failure as
the generated java_sdk_library_import module used the directory, which
did not exist, in its stub_srcs property.

This change switches the stubs_srcs property to use a glob pattern of
"**/*.java" relative to the directory instead of using the directory
directly. When the directory does not exist the glob pattern is
resolved to an empty set of paths and does not break the build.

Bug: 172811712
Test: Add sdkextensions-sdk sdk module (local patch)
      m sdk-extensions (local patch)
      unpack generated snapshot
      m nothing - results in build failure due to missing directory
      Make this change and repeat above except this time the build works.
Change-Id: I691ffbfdc01ba89bbcaf647dcbb7dfebc3c8aec2
2020-11-09 17:42:52 +00:00
Anton Hansson
30a4c9d665 Remove dead code
I removed the last use of this method in r.android.com/1450117.

Bug: 169395887
Test: m nothing
Change-Id: I3bbb600d92dac8d9c3dec0eee14c45cdfd320ce0
2020-11-05 10:49:45 +00:00
Ulya Trafimovich
8cbc5d269b Rework class loader context implementation.
The old representation consisted of a list of libraries (UsesLibraries),
a list of optional libraries (OptionalUsesLibraries) and a mapping from
library name to its build/install paths (LibraryPaths). The separation
into lists and map was necessary because of special handling of
compatibility libraries, which is now unified with normal libraries.

The new representation is a mapping from target SDK version to a tree
structure ClassLoaderContext. Each node of the tree represents a library
and contains library name, build/install paths and a slice of
subcontexts for dependencies. The same library may occur in the tree
multiple times in case it is a dependency of multiple libraries. The
order in which libraries are added matters (the resulting tree shape may
be different).

Test results have to be updated, as the resulting <uses-library> list is
reodered (previously it was a sorted list of map keys, and now it is
formed by a depth-first preorder traversal of the class loader tree).

Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Bug: 168686456
Change-Id: I11be8cd2967f004fd58753d7c5fb99fed179cd63
2020-11-03 15:15:46 +00:00
Jingwen Chen
f27c766051 Merge "Add ctx.ModuleDir and top level module dirs of input sources to JDK9 --patch-module lookup." am: 366c7d3a74 am: bb24d6d16c am: ad37e4010f am: e23a79a717
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1480608

Change-Id: I0e00a1969b710fdfd81c11a12ecccf504d009688
2020-11-03 05:53:23 +00:00
Jingwen Chen
5136a6e5aa Add ctx.ModuleDir and top level module dirs of input sources to JDK9
--patch-module lookup.

javac --patch-module accepts a list of directories and/or jars for JDK9
module patching (see bug for more details). In Bazel-Ninja execution,
Bazel executes the javac action in its own execution root working
directory, unlike Ninja, which works in the Android top level directory.
The Bazel execution root is formed of a symlink forest of top level
directories. This symlink forest is a problem for javac because it
doesn't traverse into symlinks.

To support Bazel executing these javac actions, we explicitly encode the
module directory, and the top level directory of any other source file
inputs into the --patch-module javac flag.

For example, the "core-all" libcore module compiles into `java.base`,
and depends on filegroups outside of `libcore` (`tools`). This CL adds
`tools` to the --patch-module lookup dir, on top of `libcore`.

See java_test.go for more details.

Bug: 150878007
Fixes: 150878007
Test: m
Test: bazel build droid (aosp_flame)
Change-Id: Id95b0a9a675fc75678f7b5e600344b4403f0c518
2020-11-02 17:49:57 -05:00
Paul Duffin
cb15e50b24 Merge "Move boot jars package check from make" am: 1eacd95001 am: d4f99cdebf am: 6a20dd5c29 am: d0c7e75f64
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1478639

Change-Id: Iac8ecf4c8910934d0f6ac11113bc8106f35a31b7
2020-10-30 12:04:31 +00:00
Jingwen Chen
ddc74e50d6 java compilation: refactor and extract javac flag computation into a am: 9cb8d1b37b am: 76d6f9be1c am: 63e2704c6a am: c54138f858
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1480607

Change-Id: I29d3af3cf2372ea3e55f2b627f0a00d04e00eb67
2020-10-30 11:04:46 +00:00
Paul Duffin
1eacd95001 Merge "Move boot jars package check from make" 2020-10-30 10:06:46 +00:00
Jingwen Chen
9cb8d1b37b java compilation: refactor and extract javac flag computation into a
separate function.

This cleans up the internal wiring for a follow-up --patch-module
functionality change.

Bug: 150878007
Test: m
Change-Id: Ie7d9c2b1ad40e055da8a53d632510bef408fc7d2
2020-10-30 00:55:58 -04:00
Ulyana Trafimovich
76d69c3bf2 Merge "Drop "android.hidl.manager" -> "android.hidl.base" dependency from class loader context." am: 42cac12541 am: c8b9f0a679 am: eafca437da am: a42dbeaf29
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1478218

Change-Id: Id8cdd0b682235126cbda3865606c511e2e456eec
2020-10-29 19:17:04 +00:00
Ulyana Trafimovich
42cac12541 Merge "Drop "android.hidl.manager" -> "android.hidl.base" dependency from class loader context." 2020-10-29 17:51:48 +00:00
Ulya Trafimovich
180fecedf0 Drop "android.hidl.manager" -> "android.hidl.base" dependency from class loader context.
This dependency is incorrect and shouldn't exist.
It has been removed in https://r.android.com/1467918

Test: m nothing
Bug: 170710203
Bug: 132357300
Change-Id: If3036437e138b552436f135425e6bd15be043678
2020-10-29 17:47:34 +00:00
Paul Duffin
9a89a2a0ea Move boot jars package check from make
Adds a singleton that traverses the module variants finding the ones
that are in the list (updatable and non-updatable) of boot jars and
add a ninja rule to ensure that they only contain packages from an
allowed list.

Replaces a hack that ignored any prebuilt boot jars supplied as dex
file with an equivalent one to ensure that they are still ignored.
A follow up change that switches to checking dex jars will allow the
hack to be removed.

The boot jars check can be strict or lax. If strict then all the boot
jars listed in the configuration must be found, otherwise it will only
check the ones it finds. It is strict by default unless
TARGET_BUILD_UNBUNDLED=true or ALLOW_MISSING_DEPENDENCIES=true.

Moves the script and data file from build/make.

Test: m check-boot-jars - for failing and passing cases
      SKIP_BOOT_JARS_CHECK=true - no check-boot-jars target created
	  ALLOW_MISSING_DEPENDENCIES=true - not strict
	  TARGET_BUILD_UNBUNDLED=true - not strict
      verified manually that apart from path differences the same
      files (same check sum) were checked in both old make checks and
      the new Soong ones
      EMMA_INSTRUMENT=true EMMA_INSTRUMENT_FRAMEWORK=true m check-boot-jars
Bug: 171479578
Change-Id: I9d81d6650ba64fc0d48d2dab4ba5a3ba8dd03dec
2020-10-29 17:20:06 +00:00
Ulyana Trafimovich
54874ad136 Merge "Move part of logic from construct_context.py to Soong." am: e430ac338a am: bc4e9d5f8a am: a9fa672c56 am: ea8b14a87b
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1453038

Change-Id: I95bbb8a843ba46a0a32949a0157f65061a65bde9
2020-10-29 16:01:08 +00:00
Ulyana Trafimovich
e430ac338a Merge "Move part of logic from construct_context.py to Soong." 2020-10-29 13:53:17 +00:00
Ulyana Trafimovich
3a81ccd8f9 Merge "Add separate dependency tags for compat libs based on SDK version." am: 05b9a65f25 am: 9d5814ff7c am: 7b898a1960 am: ad16c12eba
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1453037

Change-Id: I92ac79581ad5cd29b9e05398d4e9b8ea7345fa2d
2020-10-29 12:04:16 +00:00
Ulyana Trafimovich
9ba949ae48 Merge "Fix handling of "android.test.mock" in class loader context." am: 17da06cfd9 am: 94b74f9559 am: deea2844fd am: a9381bec3f
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1467832

Change-Id: I6f245711fd003a5f2c3f036241053c781c26c1fd
2020-10-29 12:04:06 +00:00
Ulyana Trafimovich
05b9a65f25 Merge "Add separate dependency tags for compat libs based on SDK version." 2020-10-29 10:27:59 +00:00
Ulyana Trafimovich
17da06cfd9 Merge "Fix handling of "android.test.mock" in class loader context." 2020-10-29 10:27:33 +00:00
Ulyana Trafimovich
175d8d6cd2 Merge "Unify handling of compat and normal libs in class loader contexts." am: c5b6f32751 am: 228f63b4c1 am: 4dc62516cb am: 182b45c981
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1450817

Change-Id: Idd32fb1f1cfdff57a152492794fddbb642629190
2020-10-28 18:18:25 +00:00
Ulyana Trafimovich
c5b6f32751 Merge "Unify handling of compat and normal libs in class loader contexts." 2020-10-28 17:05:05 +00:00
Paul Duffin
3527eadd0c Merge "Retry: Make ConfiguredJarList immutable" am: c097474f21 am: 101339a133 am: 104a934353 am: 1e6f10bf33
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1476916

Change-Id: I52505ceea5d1284ac3fdfc9f7d3e47a8d85cf972
2020-10-28 11:29:34 +00:00
Paul Duffin
3b8b84973b Cleanup usages of CreateConfiguredJarList am: e10dfa4e3d am: d34168219d am: f80f9004ea am: 6eb013d805
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1471781

Change-Id: I4dcf51fa2fc7b661240943233dbf6312e8163bf3
2020-10-27 18:19:40 +00:00
Paul Duffin
7d584e9360 Retry: Make ConfiguredJarList immutable
By making the Append and RemoveList methods return a new list instead
of modifying the existing list it makes the ConfiguredJarList usages
easier to reason about and safer to use, especially considering that
they are primarily used in global configuration.

Added some tests for Append/RemoveList to ensure that they work and
do not modify the original or result in newly created lists sharing
storage with the original which would lead to corruption.

Bug: 171756871
Bug: 171479578
Test: m nothing
      EMMA_INSTRUMENT=true EMMA_INSTRUMENT_FRAMEWORK=true m nothing
Change-Id: I541c4686ecdd45c6a0c8b1c93fedf0fcd5952e2b
2020-10-27 17:37:20 +00:00
Paul Duffin
7df37bee7f Switch BootJars/UpdatableBootJars to ConfiguredJarList am: 69d1fb1e39 am: 769b1eab84 am: 3e29458b28 am: 0b34b8cdfb
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1471779

Change-Id: I90e1bafedf0492e8299cad31e7b4a648c3cc3bfa
2020-10-27 17:29:01 +00:00
Ulya Trafimovich
8130c482ab Move part of logic from construct_context.py to Soong.
construct_context.py is a script that runs at Ninja stage and constructs
class loader context arguments for dex2oat. Previously it accepted lists
of library paths and concatenated them into one class loader context
string. The script also handled the dependency of "android.hidl.manager"
on "android.hidl.base" that is not tracked by the build system and must
be handled in a special way.

Now that class loader context representation is going to change from
flat lists to trees, passing paths to individual libraries as script
arguments is no longer possible, because a list of paths cannot
represent a class loader context tree. Passing the trees in a serialized
form is also inconvenient, because the script would have to parse them,
which would complicate it a lot.

Therefore this patch ports all the concatenation and "android.hidl.base"
handling to Soong. It is not possible to port the remaining script to
Soong because Soong has no information about the targetSdkVersiion of
the dexpreopted apps (it is in the manifest and sometimes in an APK, and
has to be extracted at Ninja time).

Test: construct_context_test.py
Test: new subtests in TestUsesLibs
Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Change-Id: Icdb03cf00d1e27e4cff3844b89bfaec4de502dd7
2020-10-27 17:05:30 +00:00
Ulya Trafimovich
b521811d7b Add separate dependency tags for compat libs based on SDK version.
The version in the tag is the SDK version that in which compatibility
library was added as a separate libary. Using distinct tags makes it
possible to differentiate between dependencies for different SDK
versions (this will be needed in subsequent CLs).

Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Change-Id: I72296c05d6649e811ddc701aaeb84f91d1ba66cb
2020-10-27 17:05:30 +00:00
Ulya Trafimovich
46b3d5bd05 Fix handling of "android.test.mock" in class loader context.
Mimick the way PackageManager handles it at runtime: do add it to class
loader context for apps with targetSdkVersion < 30, but only if
"android.test.runner" is used. Previously it was not added at all.

Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Change-Id: I4c06635277ab13e21069b9fa0b46eb6a2547dfdd
2020-10-27 17:05:30 +00:00
Ulya Trafimovich
24813e1d80 Unify handling of compat and normal libs in class loader contexts.
Also, add tests for compatibility libraries in class loader context.

This CL separates special-case handling of compatibility libraries into
a "fixup" step that is done after class loader context is constructed by
Soong. This allows to handle compatibility libraries and normal
libraries uniformly, which should enable further simplification of class
loader context representation (in subsequent CLs).

Currently the only "fixup" step is removal of libraries from conditional
class loader context if they already are in unconditional context. This
check cannot be done at the time when the libraries are added to
conditional context, because the full uncoditional context is not yet
known at that time. Previously construction of unconditional context was
delayed, now it is no longer delayed and handled in the same way as
unconditional context, and the "fixup" does the filtering.

Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Change-Id: Ie71e9fb2d3d529b5317cd84e09ab3c853017c349
2020-10-27 17:05:30 +00:00