Commit Graph

2115 Commits

Author SHA1 Message Date
Paul Duffin
891e77641d Remove unnecessary methods/fields of DefaultsModule/Base am: 3f98d148df
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1418390

Change-Id: Ib44fe28b451daa19ef9b7fec83fc549924ba5024
2020-09-03 15:41:27 +00:00
Paul Duffin
f2dc411692 Extract dist properties from commonProperties am: ed87513b0b
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1418389

Change-Id: I9a59d1a4154409a6a3d1efa590cccd2064b6c389
2020-09-03 15:18:46 +00:00
Paul Duffin
3f98d148df Remove unnecessary methods/fields of DefaultsModule/Base
The commonProperties and defaultsVisibilityProperties fields are only
used as containers for default values for properties and only within
the InitDefaultsModule() and so can be removed and replaced with an
empty structure instead.

Test: m checkbuild and TH
Change-Id: Ic5038d6b84b33afdf4859ced08e59871b9cde4a2
2020-09-03 10:12:10 +01:00
Paul Duffin
ed87513b0b Extract dist properties from commonProperties
Common properties are automatically inherited from a parent module
(i.e. one that calls CreateModule()) to the child module that it
creates. This makes no sense for dist/dists properties so this change
extracts them into their own structure separate to the
commonProperties.

Test: m checkbuild and TreeHugger
Bug: 160448975
Change-Id: Icceb20455e13394dd3b3bce464fb9bb34882d6c3
2020-09-03 10:12:10 +01:00
Colin Cross
6deda0a39a Merge "Revert "Create os and arch variants for GoBinaryTool modules"" am: 02789ab327
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1419428

Change-Id: I3d563ce07c76ae3d0200be599f7576d4253b3bcd
2020-09-03 04:31:55 +00:00
Colin Cross
02789ab327 Merge "Revert "Create os and arch variants for GoBinaryTool modules"" 2020-09-03 03:46:26 +00:00
Colin Cross
d4bdabcfad Revert "Create os and arch variants for GoBinaryTool modules"
This reverts commit 1ec1255476.

Reason for revert: b/167632665, broke linux_bionic builds.

Change-Id: I544fb5fd8aab06efa5618f076af3032271d4020f
2020-09-03 03:45:42 +00:00
Colin Cross
90d0500699 Merge changes I59b7a32a,Ida7bc75a am: 0329f2398c
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1406259

Change-Id: I9b2b709c59630394aa528c64b4fbe60b52d50be5
2020-09-03 00:21:49 +00:00
Colin Cross
0329f2398c Merge changes I59b7a32a,Ida7bc75a
* changes:
  Only request image and version variations for device SDK dependencies
  Create os and arch variants for GoBinaryTool modules
2020-09-02 23:51:53 +00:00
Xin Li
65cb5c69ea Merge Android R (rvc-dev-plus-aosp-without-vendor@6692709)
Bug: 166295507
Merged-In: Ifca7b65f4e27bf14cdc30f72f790b0de90130bae
Change-Id: I3a39be5f0b8736de4822c6a14072c78d4e4ad89d
2020-08-29 01:17:45 -07:00
Thiébaud Weksteen
3600b80e6f Return all rules when TestingModule.Rule fails
Similarly to Output, we return the list of Rules that have been
generated for TestingModule. This helps debugging failing tests.

Test: m nothing
Change-Id: I3542f4e4632f94fb84208c2e48e629271a373fd4
2020-08-28 08:15:09 +00:00
Jeongik Cha
219141c6bb Introduce BOARD_CURRENT_API_LEVEL_FOR_VENDOR_MODULES
If BOARD_CURRENT_API_LEVEL_FOR_VENDOR_MODULES has a numeric value,
it replaces "current" or "system_current" with the version which
the flag indicates.

Bug: 163009188
Test: BOARD_CURRENT_API_LEVEL_FOR_VENDOR_MODULES=29 m, and then check if every vendor
java module's sdk_version is 29 if its sdk_version was current.

Change-Id: I17b49b8e02caf2d1bc57b91648d4420f3ad9fcb9
2020-08-27 23:09:54 +09:00
Ulyana Trafimovich
03333d0e2f Merge "Export error reporting function to allow using it in other packages." 2020-08-27 09:07:51 +00:00
Colin Cross
1ec1255476 Create os and arch variants for GoBinaryTool modules
AddFarVariationDependencies was broken, which allowed modules to add
dependencies on GoBinaryTool modules using os and arch variations
even though GoBinaryTool modules did not have any variations.  Mutate
GoBinaryTool modules in the os and arch mutators so that they have the
expected os and arch variations.

Test: all soong tests
Change-Id: Ida7bc75a51ab1d2d38a6be11f574399097380cc9
2020-08-25 13:53:35 -07:00
Ulya Trafimovich
5ab276ad1d Export error reporting function to allow using it in other packages.
Test: lunch aosp_cf_x86_phone-userdebug && m
Change-Id: I3a1d936ac57966b2f47cb1d9a56e0f96f47b807d
2020-08-25 12:45:15 +01:00
Treehugger Robot
4c2e4f381a Merge "Support ninja rsp files in soong_zip" 2020-08-24 18:40:56 +00:00
Ulyana Trafimovich
49b1e9b6ce Merge "Use common helper functions for getting sorted map keys." 2020-08-21 09:36:30 +00:00
Treehugger Robot
68bfe074c6 Merge "Reland: Deduplicate APEX variants that would build identically" 2020-08-21 06:46:51 +00:00
Treehugger Robot
c23d6031b0 Merge "Capture list of unused methods when shrinking in R8" 2020-08-21 00:28:24 +00:00
Ulya Trafimovich
b8063c6a86 Use common helper functions for getting sorted map keys.
Add a new helper SortedIntKeys similar to SortedStringKeys.

Test: lunch aosp_cf_x86_phone-userdebug && m
Change-Id: I08a43ec2cae7d1a82531295aca1a0658e3a0dd6f
2020-08-20 12:37:03 +01:00
Colin Cross
053fca10c9 Support ninja rsp files in soong_zip
Add a -r argument to soong_zip that reads a list of files from a file
like the -l argument but treats it as a Ninja rsp file with escaping.
Replace the -l arguments in Soong that are using rsp files with -r.

Fixes: 162435077
Test: TestReadRespFile, TestZip
Change-Id: I4605312e99406ab1bd0c37af9c5ad212393f0403
2020-08-19 21:18:56 +00:00
Colin Cross
cb6143a142 Capture list of unused methods when shrinking in R8
Use the -printusage flag in R8 to output a list of the unused
methods.  Some of the files can be large (2MB for DocumentsUI,
87MB for all of AOSP), so immediately zip them and remove the
originals.  The zipped files will be merged and disted.

Bug: 151857441
Test: m TARGET_BUILD_APPS=DocumentsUI dist
Change-Id: I780e84e80eba7fe4d4fa15fec0f461890afd900b
2020-08-19 11:19:14 -07:00
Colin Cross
aede88c1c7 Reland: Deduplicate APEX variants that would build identically
APEX variants that share the same SDK version and updatability
almost always use identical command line arguments to build but
with different intermediates directories.  This causes unnecessary
build time and disk space for duplicated work.

Deduplicate APEX variants that would build identically.  Create
aliases from the per-APEX variations to the new shared variations
so that the APEX modules can continue to depend on them via the
APEX name as the variation.

This has one significant change in behavior.  Before this change,
if an APEX had two libraries in its direct dependencies and one
of those libraries depended on the other, and the second library
had stubs, then the first library would depend on the implementation
of the second library and not the stubs.  After this change, if
the first library is also present in a second APEX but the second
library is not, then the common variant shared between the two
APEXes would use the stubs, not the implementation.

In a correctly configured set of build rules this change will
be irrelevant, because if the compilation worked for the second
APEX using stubs then it will work for the common variant using
stubs.  However, if an incorrect change to the build rules is
made this could lead to confusing errors, as a previously-working
common variant could suddenly stop building when a module is added
to a new APEX without its dependencies that require implementation
APIs to compile.

This change reduces the number of modules in an AOSP arm64-userdebug
build by 3% (52242 to 50586), reduces the number of variants of the
libcutils module from 74 to 53, and reduces the number of variants
of the massive libart[d] modules from 44 to 32.

This relands I0529837476a253c32b3dfb98dcccf107427c742c with a fix
to always mark permissions XML files of java_sdk_library modules as
unique per apex since they contain the APEX filename, and a fix
to UpdateUniqueApexVariationsForDeps to check ApexInfo.InApexes
instead of DepIsInSameApex to check if two modules are in the same
apex to account for a module that depends on another in a way that
doesn't normally include the dependency in the APEX (e.g. a libs
property), but the dependency is directly included in the APEX.

Bug: 164216768
Test: go test ./build/soong/apex/...
Change-Id: I2ae170601f764e5b88d0be2e0e6adc84e3a4d9cc
2020-08-19 10:21:17 -07:00
Pete Bentley
0c7b26e0e2 Revert "Deduplicate APEX variants that would build identically"
This reverts commit d6b2525b00.

Reason for revert: <Breaks tests - confirmed by Forrest: b/165188843>

Change-Id: I7cb68cb87522415004390c0672dc774e0067b122
2020-08-18 13:44:59 +00:00
Colin Cross
d6b2525b00 Deduplicate APEX variants that would build identically
APEX variants that share the same SDK version and updatability
almost always use identical command line arguments to build but
with different intermediates directories.  This causes unnecessary
build time and disk space for duplicated work.

Deduplicate APEX variants that would build identically.  Create
aliases from the per-APEX variations to the new shared variations
so that the APEX modules can continue to depend on them via the
APEX name as the variation.

This has one significant change in behavior.  Before this change,
if an APEX had two libraries in its direct dependencies and one
of those libraries depended on the other, and the second library
had stubs, then the first library would depend on the implementation
of the second library and not the stubs.  After this change, if
the first library is also present in a second APEX but the second
library is not, then the common variant shared between the two
APEXes would use the stubs, not the implementation.

In a correctly configured set of build rules this change will
be irrelevant, because if the compilation worked for the second
APEX using stubs then it will work for the common variant using
stubs.  However, if an incorrect change to the build rules is
made this could lead to confusing errors, as a previously-working
common variant could suddenly stop building when a module is added
to a new APEX without its dependencies that require implementation
APIs to compile.

This change reduces the number of modules in an AOSP arm64-userdebug
build by 3% (52242 to 50586), reduces the number of variants of the
libcutils module from 74 to 53, and reduces the number of variants
of the massive libart[d] modules from 44 to 32.

Bug: 164216768
Test: go test ./build/soong/apex/...
Change-Id: I0529837476a253c32b3dfb98dcccf107427c742c
2020-08-17 15:18:31 -07:00
Treehugger Robot
8cb67e4358 Merge "Override EarlyModuleContext.Namespace" 2020-08-17 19:37:39 +00:00
Colin Cross
133ebefb8d Override EarlyModuleContext.Namespace
Make android.EarlyModuleContext return a *Namespace.

Test: m checkbuild
Change-Id: I4c29d5e69e90eb9b7afbefc1355f63991a3e3331
2020-08-14 17:38:45 -07:00
Colin Cross
e07f2316b7 Rename ApexName to ApexVariationName
In preparation for reusing the same variation for multiple apexes,
rename ApexName to ApexVariationName.

Bug: 164216768
Test: all soong tests
Change-Id: I88f2c5b192ffa27acd38e01952d0cefd413222a0
2020-08-13 17:50:29 -07:00
Colin Cross
1b9604b463 Add CreateAliasVariation to BottomUpMutatorContext
Propagate CreateAliasVariation from Blueprint to Soong.

Bug: 164216768
Test: m checkbuild
Change-Id: I635bf67481092b83a8d26ef1cb06e2e3e21a595f
2020-08-13 17:50:05 -07:00
Colin Cross
beae6ecbd4 Shorten missing module panic message
Only print the list of variants of the matching module when
ctx.ModuleForTesting finds the module but not the variant.

Test: all soong tests
Change-Id: I51ffdde4645db39ec1d37ec018e0dea11d74280e
2020-08-13 11:52:21 -07:00
Anton Hansson
3d02e1f3c5 Merge "Update neverallows path for sdkextensions" 2020-08-11 11:55:39 +00:00
Anton Hansson
4b8e64b1ce Update neverallows path for sdkextensions
The modules sources have been moved, update the sdk variant whitelist.

Bug: 156286599
Test: m
Exempt-From-Owner-Approval: cherry-pick from internal
Change-Id: Idbd02b17b1c741f1dc20a3742ff57e8469a846c0
Merged-In: Idbd02b17b1c741f1dc20a3742ff57e8469a846c0
2020-08-11 10:32:21 +01:00
Dan Willemsen
9f43597ff7 Remove obsolete PDK build functionality
This hasn't worked for a couple years, and continues to bitrot. Just
remove it.

Adds a bpfix rule so that we can eventually remove the
product_variables.pdk definition, which is now always a no-op.

Test: treehugger
Change-Id: I830b54d419b59f6db1d4617b45e61a78234f57a7
Merged-In: I830b54d419b59f6db1d4617b45e61a78234f57a7
2020-08-11 01:11:44 +00:00
Jingwen Chen
fc16b6684e Merge "Fix droiddoc disting when WITHOUT_CHECK_API is true." 2020-08-10 05:22:24 +00:00
Jeongik Cha
6df337255c Merge "Introduce AlwaysUsePrebuiltSdks" 2020-08-10 00:51:56 +00:00
Dan Willemsen
a047f0120c Merge "Reland "Add prebuilt_build_tool to allow genrules to use prebuilt tools"" 2020-08-07 20:27:40 +00:00
Treehugger Robot
f3ab72936b Merge "Add BoardKernelModuleInterfaceVersions." 2020-08-07 18:37:36 +00:00
Martin Stjernholm
9f381d5ce3 Merge "Only output make targets for uninstallable static libs in the APEX unavailable-to-platform case." 2020-08-07 11:30:29 +00:00
Jeongik Cha
816a23a50d Introduce AlwaysUsePrebuiltSdks
Instead of UnbundledBuild, use AlwaysUsePrebuiltSdks
to determine if java modules needs to be built against prebuilt sdks.
And rename UnbundledBuildUsePrebuiltSdks to AlwaysUsePrebuiltSdks to
express its behavior more correctly.(It can be orthgonal to "Unbundled")

Bug: 160390776
Test: TARGET_BUILD_UNBUNDLED_IMAGE=true m vendorimage

Change-Id: I0be7265c1959d8774c295372cd7a9250169f6df9
2020-08-07 12:15:52 +09:00
Treehugger Robot
8b073fc7f1 Merge "Reland "Support per-module MakeVars"" 2020-08-07 00:50:24 +00:00
Jingwen Chen
2106bf8458 Merge "Create a AOSP Bazel overlay workspace with Soong" 2020-08-06 23:56:54 +00:00
Dan Willemsen
751ae87ed2 Reland "Add prebuilt_build_tool to allow genrules to use prebuilt tools"
Includes fixes for out dirs at absolute paths, and symlinks to absolute
paths. Also squashes support for exporting make vars, which previously
happened in a later patch (which got moved first, since it's mostly
infrastructure changes).

Original message:

cc_prebuilt_binary doesn't work well for host tools because they'll
often be using their own versions of shared libraries that may not be
compatible with what we build.

So add a module type that allows genrules to use one of these prebuilts
as a tool. Like other prebuilts, we'll use the source module if we have
it, or the prebuilt otherwise.

It supports adding extra dependencies for shared libraries or other data
files that are necessary to run the tool. Any genrules using the tool
will be rerun if any of the dependencies change.

Bug: 128690776
Test: treehugger
Test: build one-true-awk using genrule with bison
Test: with OUT_DIR=$PWD/out
Test: with OUT_DIR pointing to an absolute symlink
Change-Id: I662296cb55503b6a10ec5e5465b138e9edeb71f3
2020-08-06 15:56:38 -07:00
Dan Willemsen
6a6478d49e Reland "Support per-module MakeVars"
This allows setting per-module make variables earlier in the build,
particularly for prebuilt_build_tool users like LEX/M4/BISON (which will
come in a later CL). I moved filegroup over because it's a simpler
common interface, but it doesn't strictly need it.

Test: Inspect out/soong/make_vars-*.mk, out/soong/late-*.mk
Test: treehugger
Change-Id: I38631129d82b0aad2438e83610bb6a97568bf932
2020-08-06 15:56:09 -07:00
Yifan Hong
42bef8d0cc Add BoardKernelModuleInterfaceVersions.
This is a list of KMI versions for this board. Only gki_apex modules
with matching kmi_version field is enabled.

Test: build GKI apexes
Bug: 162888350
Change-Id: Ieb2116b79e3985ba15155a3cb8a2c340b46473c7
2020-08-06 15:47:58 -07:00
Martin Stjernholm
9e9bb7f223 Only output make targets for uninstallable static libs in the APEX
unavailable-to-platform case.

This removes the special case added in https://r.android.com/1274763
from SkipInstall(), so that it doesn't cause conflicting AndroidMk
entries when a cc_prebuilt_library_static module has prefer:true and
the corresponding source module exists.

Test: `m` in a tree with a snapshot created from art-module-sdk where
  the libartimagevalues module has prefer:true
Bug: 151303681
Change-Id: I651ae325753b707296892adb4cae80daaddb6af2
2020-08-06 22:52:58 +01:00
Ulyana Trafimovich
910eb70891 Merge "Add structured representation for colon-separated jar lists." 2020-08-06 15:07:27 +00:00
Jingwen Chen
5ba7e479d1 Create a AOSP Bazel overlay workspace with Soong
The Bazel overlay is a directory at out/soong/bazel_overlay that
replicates the layout of the AOSP Soong module tree, but as a Bazel
workspace. Each Soong module variant is represented as a BUILD target
created with the `soong_module` rule.

To create this overlay, run `m bazel_overlay`.

A `soong_module` target can depend on other `soong_module` targets.
These dependencies replicate each module's `directDeps` in the Blueprint
graph, just before `PrepareBuildActions`.

This enables users to use bazel query as a way to introspect the Soong
module graph. For example,

- Direct reverse dependencies of //bionic/libc:generated_android_ids in
//bionic/libc/...:

$ bazel query 'rdeps(//bionic/libc/...,
//bionic/libc:generated_android_ids, 1)'
//bionic/libc:libc_bionic_ndk--android_recovery_arm_armv7-a-neon_static
//bionic/libc:libc_bionic_ndk--android_ramdisk_arm_armv7-a-neon_static
//bionic/libc:libc_bionic_ndk--android_arm_armv7-a-neon_static_com.android.runtime
//bionic/libc:libc_bionic_ndk--android_arm_armv7-a-neon_static
//bionic/libc:generated_android_ids

- Why does com.android.runtime depend on lzma?

$ bazel query
'somepath(//bionic/apex:com.android.runtime--android_common_com.android.runtime_image,
//external/lzma/...)'
//bionic/apex:com.android.runtime--android_common_com.android.runtime_image
//bionic/libc/malloc_debug:libc_malloc_debug--android_arm_armv7-a-neon_shared_com.android.runtime
//system/core/libunwindstack:libunwindstack--android_arm_armv7-a-neon_shared_com.android.runtime
//external/lzma/C:liblzma--android_arm_armv7-a-neon_shared_com.android.runtime

- What does the dep graph of //bionic/libc:crtbegin_so look like?

$ bazel query
'deps(//bionic/libc:crtbegin_so--android_arm_armv7-a-neon)'
--output=graph > graph.in && dot -Tpng < graph.in > graph.png

https://photos.app.goo.gl/DfsdoFRNsRjGwTmy8

Test:  croot && m bazel_overlay && cd out/soong/bazel_overlay && bazel
query //... && bazel query 'rdeps(//bionic/libc/...,
//bionic/libc:generated_android_ids, 1)'

Signed-off-by: Jingwen Chen <jingwen@google.com>
Change-Id: I3bf40309bfb2d963bb8a688706385a57ee304c37#
2020-08-06 10:24:21 +00:00
Colin Cross
2eddd06879 Merge "Add libraryDependencyTag to track dependencies on static and shared libraries" 2020-08-06 00:06:28 +00:00
Martin Stjernholm
42e6dabe81 Merge "Disable all host OS variants except those explicitly included." 2020-08-05 14:50:48 +00:00
Ulya Trafimovich
249386ad69 Add structured representation for colon-separated jar lists.
With the addition of apexes and /system_ext some of the bootclasspath
and system server jars have moved from /system to the new locations.
This has been implemented by using lists of colon-separated strings
called "apex-jar pairs" (although "apex" was misleading as it could
refer to "platform" or "system_ext", not necessarily a real apex).

Using the colon-separated string representation is inconvenient, as it
requires splitting and reassembling the list components many times,
which harms performance and makes error handling difficult. Therefore
this patch refactors the colon-separated lists into a struct that
hides the implementation details.

Test: lunch aosp_cf_x86_phone-userdebug && m
Change-Id: Id248ce639a267076294f4d4d73971da2f2f77208
2020-08-05 09:59:30 +01:00