Commit Graph

27573 Commits

Author SHA1 Message Date
Ulya Trafimovich
a8c28e27bc Do not add dependencies of shared SDK libraries to manifest_fixer.
Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Bug: 168686456
Change-Id: Ibd9742684fa6a8f1353ca0e513f7fa814a6ec9fc
2020-11-16 14:58:11 +00:00
Ulya Trafimovich
18554243de Add nested class loader subcontext at the proper hierarchy level.
When adding a subcontext in a class loader context tree, there are two
possible cases: 1) the root of the subcontext is itself a <uses-library>
and should be present as a node in the tree, or 2) the root is not a
<uses-library>, but some of its dependencies are -- in that case they
should be disconnected from the root, and the resulting forrest should
be added at the top-level.

Example:

  1) C is a <uses-library>:

     A
     ├── B
     └── C
         ├── D
         └── E
             └── F

  2) C is not a <uses-library>:

     A
     ├── B
     ├── D
     └── E
         └── F

Before the patch subcontexts for transitive dependencies were added
before the subcontext for the direct dependency (even if it was a
<uses-library>, resulting in case-2 hierarchy when case-1 should have
been used. Previosuly this didn't matter because class loader context
was a flat set of libraries, but now it matters because class loader
context is a tree.

This patch changes the order in which libraries are added, so that
direct dependencies are added before transitive ones. The context adding
method now accepts an "implicit root" parameter, so that when adding
transitive dependencies it can check if the corresponding direct
dependency is a <uses-library> and already present in the context.

Partially constructed class loader context is now propagated top-down
into aapt.buildActions, so that the method can use existing part of the
context to decide where the missing part should be connected.

Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Change-Id: I649aff9e27494306885a4f4fc90226c399636b57
2020-11-16 14:57:05 +00:00
Ulya Trafimovich
5e13a7307e Disallow adding nested conditional class loader context.
Nested conditional context doesn't make sense because conditional
context is only needed for compatibility libraries, and those are only
used by apps, and apps are always at the root of class loader context
tree. Therefore, trying to add conditional nested context can only mean
an error somewhere on the way.

Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Change-Id: Iadeaaf89fdb11cb23b107c6fb074b1bc765256be
2020-11-16 12:17:30 +00:00
Ulya Trafimovich
8cbc5d269b Rework class loader context implementation.
The old representation consisted of a list of libraries (UsesLibraries),
a list of optional libraries (OptionalUsesLibraries) and a mapping from
library name to its build/install paths (LibraryPaths). The separation
into lists and map was necessary because of special handling of
compatibility libraries, which is now unified with normal libraries.

The new representation is a mapping from target SDK version to a tree
structure ClassLoaderContext. Each node of the tree represents a library
and contains library name, build/install paths and a slice of
subcontexts for dependencies. The same library may occur in the tree
multiple times in case it is a dependency of multiple libraries. The
order in which libraries are added matters (the resulting tree shape may
be different).

Test results have to be updated, as the resulting <uses-library> list is
reodered (previously it was a sorted list of map keys, and now it is
formed by a depth-first preorder traversal of the class loader tree).

Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Bug: 168686456
Change-Id: I11be8cd2967f004fd58753d7c5fb99fed179cd63
2020-11-03 15:15:46 +00:00
Jingwen Chen
366c7d3a74 Merge "Add ctx.ModuleDir and top level module dirs of input sources to JDK9 --patch-module lookup." 2020-11-03 04:56:13 +00:00
Jingwen Chen
5136a6e5aa Add ctx.ModuleDir and top level module dirs of input sources to JDK9
--patch-module lookup.

javac --patch-module accepts a list of directories and/or jars for JDK9
module patching (see bug for more details). In Bazel-Ninja execution,
Bazel executes the javac action in its own execution root working
directory, unlike Ninja, which works in the Android top level directory.
The Bazel execution root is formed of a symlink forest of top level
directories. This symlink forest is a problem for javac because it
doesn't traverse into symlinks.

To support Bazel executing these javac actions, we explicitly encode the
module directory, and the top level directory of any other source file
inputs into the --patch-module javac flag.

For example, the "core-all" libcore module compiles into `java.base`,
and depends on filegroups outside of `libcore` (`tools`). This CL adds
`tools` to the --patch-module lookup dir, on top of `libcore`.

See java_test.go for more details.

Bug: 150878007
Fixes: 150878007
Test: m
Test: bazel build droid (aosp_flame)
Change-Id: Id95b0a9a675fc75678f7b5e600344b4403f0c518
2020-11-02 17:49:57 -05:00
Treehugger Robot
d3e294d8ac Merge "Add --skip-soong-tests argument to soong_ui and use it in multiproduct_kati" 2020-11-02 21:31:31 +00:00
Treehugger Robot
20cf20ba1c Merge changes I0289d866,Ie7363524
* changes:
  Move stubs related methods out of LinkableInterface
  Don't create stubs variants of static libraries
2020-11-02 21:31:11 +00:00
Patrice Arruda
e83dea5aac Merge "Capture the build command that executed the build system." 2020-10-30 16:25:34 +00:00
Ulyana Trafimovich
9b567d555a Merge "Add unit tests for class loader context." 2020-10-30 10:47:37 +00:00
Ulyana Trafimovich
f1a2e55ac9 Merge "Add ModuleInstallPathContextForTesting." 2020-10-30 10:47:21 +00:00
Paul Duffin
1eacd95001 Merge "Move boot jars package check from make" 2020-10-30 10:06:46 +00:00
Jingwen Chen
9cb8d1b37b java compilation: refactor and extract javac flag computation into a
separate function.

This cleans up the internal wiring for a follow-up --patch-module
functionality change.

Bug: 150878007
Test: m
Change-Id: Ie7d9c2b1ad40e055da8a53d632510bef408fc7d2
2020-10-30 00:55:58 -04:00
Zach Johnson
6e64c2fe65 Merge "Add system/bt to Rust allowed paths." 2020-10-29 23:20:30 +00:00
Nikita Ioffe
e3fbadcfee Merge "Update apex/allowed_deps.txt" 2020-10-29 21:46:54 +00:00
Colin Cross
00a8a3f746 Add --skip-soong-tests argument to soong_ui and use it in multiproduct_kati
There is no need for multiproduct_kati to run the tests for
every product, they don't vary by product config. --skip-soong-tests
can also be used for local development to run soong_build even if
the tests don't pass.

Bug: 156428456
Test: m --skip-soong-tests nothing
Change-Id: I9c00e3d1b6e51d17bb290339c3f124d4d1c9e69f
2020-10-29 14:22:04 -07:00
Colin Cross
31076b3185 Move stubs related methods out of LinkableInterface
The stubs methods are very specific to cc for now, move them out
of LinkableInterface so they are not shared with rust.  Instead,
create a cc.Module.library field that contains the libraryInterface
to simplify calling libraryInterface methods on cc modules.

Test: all Soong tests
Test: no change to Soong outputs
Change-Id: I0289d866ce1f7a765631fe3101a62b1b4988ba1c
2020-10-29 14:17:54 -07:00
Colin Cross
a717db7304 Don't create stubs variants of static libraries
Statically linking stub implementations doesn't make sense, don't
create stubs variants of static libraries.

Bug: 170784825
Test: all soong tests
Change-Id: Ie73635244516edf6da884e3a7a275971a9bd7839
2020-10-29 14:17:37 -07:00
Martin Stjernholm
3a456b606e Merge "Disable sanitizer flags and output files for sanitizer-enabled SDK members." 2020-10-29 20:12:25 +00:00
Qasim Javed
d271ada0c9 Add system/bt to Rust allowed paths.
Test: m nothing

Change-Id: Ie467a622dfdb8aea94a06f5fe20183dc9d149254
2020-10-29 11:18:08 -07:00
Patrice Arruda
e92c30dff1 Capture the build command that executed the build system.
We need to capture the user build command to understand what additional
arguments is passed in to Soong. This data is useful to know which
targets are being executed and have statistics on top build targets
are specified.

Bug: b/169453712
Test: ran "m blueprint_tools", "m nothing", "lunch 1" and checked
      in soong_metrics proto.
Change-Id: I7ab94dcf037211f1abffe6afd012b03ffbf99756
2020-10-29 11:01:32 -07:00
Ulyana Trafimovich
42cac12541 Merge "Drop "android.hidl.manager" -> "android.hidl.base" dependency from class loader context." 2020-10-29 17:51:48 +00:00
Ulyana Trafimovich
810025466f Merge "Move class loader context definitions to a separate file." 2020-10-29 17:51:24 +00:00
Ulya Trafimovich
6961267298 Add unit tests for class loader context.
Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Change-Id: I96a1e6cef86652e429b1678a655fc0b02f40d00c
2020-10-29 17:47:34 +00:00
Ulya Trafimovich
ccc8c85a7c Add ModuleInstallPathContextForTesting.
Test: m nothing
Change-Id: Ibcd6016cc43300fcc7f0ec511a3159886d0fdc7d
2020-10-29 17:47:34 +00:00
Ulya Trafimovich
180fecedf0 Drop "android.hidl.manager" -> "android.hidl.base" dependency from class loader context.
This dependency is incorrect and shouldn't exist.
It has been removed in https://r.android.com/1467918

Test: m nothing
Bug: 170710203
Bug: 132357300
Change-Id: If3036437e138b552436f135425e6bd15be043678
2020-10-29 17:47:34 +00:00
Ulya Trafimovich
eb26886c85 Move class loader context definitions to a separate file.
Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Change-Id: I1e7e9db1654d0b835276be1cfa6a8eeffc5e96ee
2020-10-29 17:47:34 +00:00
Paul Duffin
9a89a2a0ea Move boot jars package check from make
Adds a singleton that traverses the module variants finding the ones
that are in the list (updatable and non-updatable) of boot jars and
add a ninja rule to ensure that they only contain packages from an
allowed list.

Replaces a hack that ignored any prebuilt boot jars supplied as dex
file with an equivalent one to ensure that they are still ignored.
A follow up change that switches to checking dex jars will allow the
hack to be removed.

The boot jars check can be strict or lax. If strict then all the boot
jars listed in the configuration must be found, otherwise it will only
check the ones it finds. It is strict by default unless
TARGET_BUILD_UNBUNDLED=true or ALLOW_MISSING_DEPENDENCIES=true.

Moves the script and data file from build/make.

Test: m check-boot-jars - for failing and passing cases
      SKIP_BOOT_JARS_CHECK=true - no check-boot-jars target created
	  ALLOW_MISSING_DEPENDENCIES=true - not strict
	  TARGET_BUILD_UNBUNDLED=true - not strict
      verified manually that apart from path differences the same
      files (same check sum) were checked in both old make checks and
      the new Soong ones
      EMMA_INSTRUMENT=true EMMA_INSTRUMENT_FRAMEWORK=true m check-boot-jars
Bug: 171479578
Change-Id: I9d81d6650ba64fc0d48d2dab4ba5a3ba8dd03dec
2020-10-29 17:20:06 +00:00
Yifan Hong
8452b1a993 Merge "Make vendor_ramdisk_available." 2020-10-29 17:17:29 +00:00
Nikita Ioffe
7f17421012 Update apex/allowed_deps.txt
Bug: 171330443
Bug: 171668006
Test: build/soong/scripts/update-apex-allowed-deps.sh
Change-Id: Id239ea9b31ce23f045f0db4027db33658ef3da14
Merged-In: Id239ea9b31ce23f045f0db4027db33658ef3da14
2020-10-29 15:35:01 +00:00
Ulyana Trafimovich
e430ac338a Merge "Move part of logic from construct_context.py to Soong." 2020-10-29 13:53:17 +00:00
Ulyana Trafimovich
05b9a65f25 Merge "Add separate dependency tags for compat libs based on SDK version." 2020-10-29 10:27:59 +00:00
Ulyana Trafimovich
17da06cfd9 Merge "Fix handling of "android.test.mock" in class loader context." 2020-10-29 10:27:33 +00:00
Martin Stjernholm
59e0c7a98f Disable sanitizer flags and output files for sanitizer-enabled SDK members.
Test: build/soong/soong_ui.bash --make-mode \
      TARGET_PRODUCT=aosp_blueline_hwasan \
      TARGET_BUILD_VARIANT=userdebug nothing
Test: build/soong/scripts/build-aml-prebuilts.sh \
      runtime-module-{sdk,host-exports} \
      platform-mainline-{sdk,test-exports}
  Check that generated Android.bp files don't differ
Bug: 171754462
Change-Id: Ieef40ff202ceda2d9e613a5d9295f5c9d8160626
2020-10-29 00:26:24 +00:00
Chris Parsons
b0f8ac4ff0 Batch cquery requests for mixed builds
This adds an extra step to mixed builds: creating a master BUILD/bzl
file pair to facilitate running a single cquery command to analyze
all soong->bazel edges in a single request.

Test: Mixed build tested with aosp/1441774, verified ninja outputs
depend on `bazel-out/` intermediates
Test: Manually verified contents of master BUILD and bzl files

Change-Id: I04803bcc91ac4182578f505b3f42893061ddd167
2020-10-28 21:56:14 +00:00
Yifan Hong
a03f8fe6da Merge "Add target.vendor_ramdisk." 2020-10-28 20:02:24 +00:00
Christopher Parsons
752212243f Merge "Change mixed soong/bazel builds to use USE_BAZEL_ANALYSIS" 2020-10-28 20:01:17 +00:00
Yifan Hong
8683648981 Merge "cc/linker.go: fix typo." 2020-10-28 17:29:05 +00:00
Ulyana Trafimovich
c5b6f32751 Merge "Unify handling of compat and normal libs in class loader contexts." 2020-10-28 17:05:05 +00:00
Nikita Ioffe
7a19ee30ba Merge "Update apex/allowed_deps.txt" 2020-10-28 12:57:25 +00:00
Paul Duffin
c097474f21 Merge "Retry: Make ConfiguredJarList immutable" 2020-10-28 10:12:16 +00:00
Treehugger Robot
9d181ec5d5 Merge "Add smoke test for Bazel." 2020-10-28 06:21:58 +00:00
Yifan Hong
02e7dec682 Make vendor_ramdisk_available.
Test: pass
Bug: 156098440
Change-Id: Id980fae69db9c3e506dc89e3776c918e675e252b
2020-10-27 20:51:49 -07:00
Yifan Hong
6da33c2af6 Add target.vendor_ramdisk.
Add exclude_srcs, exclude_static_libs, exclude_shared_libs, and
cflags.

Test: pass
Bug: 156098440

Change-Id: I2e10e1f7145932e9bc64f72fa95d0aac776f7f63
2020-10-27 20:51:49 -07:00
Yifan Hong
d2057fbb8e cc/linker.go: fix typo.
The ramdisk variant is using properties from the recovery variant.

Test: TH
Change-Id: Ib2a349e307ab0389413c2d550cf7deb4c7c8ecf4
2020-10-27 20:51:49 -07:00
Treehugger Robot
6776629f63 Merge "Set stem and suffix for vndk prebuilt libraries" 2020-10-28 01:25:36 +00:00
Nikita Ioffe
769f4a5c03 Update apex/allowed_deps.txt
Bug: 171330443
Test: build/soong/scripts/update-apex-allowed-deps.sh
Change-Id: Ic7ddd86cefc0c6a26de294b6e73b3e23d4791f5c
Merged-In: Ic7ddd86cefc0c6a26de294b6e73b3e23d4791f5c
2020-10-28 01:17:55 +00:00
Chris Parsons
8b77a009e2 Change mixed soong/bazel builds to use USE_BAZEL_ANALYSIS
As a result, one can enable bazel-as-ninja-executor separately from
mixed builds.

Test: Manually verified building image with and without bazelenv.sh.
Change-Id: Ia97806fb41e1de850a80b9483ed8a5ff50d9dab2
2020-10-27 18:59:25 -04:00
Paul Duffin
7d584e9360 Retry: Make ConfiguredJarList immutable
By making the Append and RemoveList methods return a new list instead
of modifying the existing list it makes the ConfiguredJarList usages
easier to reason about and safer to use, especially considering that
they are primarily used in global configuration.

Added some tests for Append/RemoveList to ensure that they work and
do not modify the original or result in newly created lists sharing
storage with the original which would lead to corruption.

Bug: 171756871
Bug: 171479578
Test: m nothing
      EMMA_INSTRUMENT=true EMMA_INSTRUMENT_FRAMEWORK=true m nothing
Change-Id: I541c4686ecdd45c6a0c8b1c93fedf0fcd5952e2b
2020-10-27 17:37:20 +00:00
Ulya Trafimovich
8130c482ab Move part of logic from construct_context.py to Soong.
construct_context.py is a script that runs at Ninja stage and constructs
class loader context arguments for dex2oat. Previously it accepted lists
of library paths and concatenated them into one class loader context
string. The script also handled the dependency of "android.hidl.manager"
on "android.hidl.base" that is not tracked by the build system and must
be handled in a special way.

Now that class loader context representation is going to change from
flat lists to trees, passing paths to individual libraries as script
arguments is no longer possible, because a list of paths cannot
represent a class loader context tree. Passing the trees in a serialized
form is also inconvenient, because the script would have to parse them,
which would complicate it a lot.

Therefore this patch ports all the concatenation and "android.hidl.base"
handling to Soong. It is not possible to port the remaining script to
Soong because Soong has no information about the targetSdkVersiion of
the dexpreopted apps (it is in the manifest and sometimes in an APK, and
has to be extracted at Ninja time).

Test: construct_context_test.py
Test: new subtests in TestUsesLibs
Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Change-Id: Icdb03cf00d1e27e4cff3844b89bfaec4de502dd7
2020-10-27 17:05:30 +00:00