Commit Graph

166 Commits

Author SHA1 Message Date
Inseob Kim
f84e9c05e2 Add debug ramdisk variant
A module will be installed to debug_ramdisk (or
debug_ramdisk/first_stage_ramdisk if recovery as boot is true) if
debug_ramdisk is set to true.

Bug: 184004542
Test: soong test
Change-Id: Ic5a4d27407e506fffa462de2149e0785f11b2ac7
2021-04-15 00:56:30 +00:00
Colin Cross
66d154e436 Merge "Follow blueprint change to return GlobResult from Glob" 2021-04-13 17:01:39 +00:00
Colin Cross
82ea3fb273 Follow blueprint change to return GlobResult from Glob
Follow I2159cc9d85f388073198eac7456e5bf43e813096 that makes Glob
return a GlobResult.

Bug: 159845846
Test: glob_test.go
Change-Id: Ia771bdbdf1eb668623c4b3f00bf65e0e1e3a55c0
2021-04-09 09:46:18 -07:00
Jingwen Chen
6393098ecf bp2build: cc_library converter for //bionic/libdl:libdl_android
This CL contains the converter for libdl_android, a cc_library that
expands into a cc_shared_library and a regular cc_library.

Test: TH
Test: bp2build; bazel test //build/bazel/tests/...

Change-Id: If70641a538211b0d6b2aac0e4d0d06912318304d
2021-04-09 09:42:53 +00:00
Paul Duffin
afdd4061ee Rename OptionalPath.RelativeToPath -> RelativeToTop
Bug: 183650682
Test: m nothing
Change-Id: I95239c2273e715e8c4c15a4a02967e01f6913234
2021-03-30 20:06:22 +01:00
Paul Duffin
544c368d2e Merge "Add OptionalPath.RelativeToTop" 2021-03-30 16:43:23 +00:00
Jingwen Chen
0702791a99 bp2build: arch-configurable selects for label list attrs.
This CL adds the configurable LabelListAttribute support to bp2build.

Test: go test
Change-Id: I2ef9e385d9cf1b1845988128eca1d8cda1ecb5e8
2021-03-30 13:22:48 +00:00
Paul Duffin
a5b8135bdf Add OptionalPath.RelativeToTop
Needed for some follow up changes.

Bug: 183650682
Test: m nothing
Change-Id: If38dfe8e919094a7365bcee6cb71bfd0409f4454
2021-03-29 17:29:30 +01:00
Martin Stjernholm
32312eb75a Don't panic on "go test" invocations from the command line.
Test: cd build/soong; go test ./android
Bug: 183650682
Change-Id: I2d1bbda21cb262eafc7f7d329206720809399985
2021-03-28 22:06:33 +01:00
Paul Duffin
85d8f0dcd9 Add Path.RelativeToTop
Makes Path.RelativeToTop the source of truth as to what `relative to
top` means.

Uses it in PathRelativeToTop so that it will be tested by the existing
tests that use AssertPathRelativeToTop.

Also adds RelativeToTop() to WritablePaths and Paths.

Bug: 183650682
Test: m droid
Change-Id: I88dc56afd1314e51b4e41795a2448cab4ce8a899
2021-03-25 13:14:37 +00:00
Paul Duffin
74abc5d4ee Remove config from basePath
Bug: 183650682
Test: m droid
Change-Id: I60eb2b8ed0317e874ecdafd5cb37d7c997d41bb8
2021-03-25 09:18:29 +00:00
Paul Duffin
d65c58b204 Add buildDir to WritablePath implementations
First, the buildDir() method was renamed to getBuildDir() to avoid
clashing with the buildDir field.

Then, a buildDir was added to both `OutputPath` and `InstallPath` but
not to `PhonyPath` as it does not contain any path components. Instead
the `PhonyPath.getBuildDir()` was changed to simply return "".

Bug: 183650682
Test: m droid
Change-Id: I12e1854c829b980c5c01205753c62c00dc0a4774
2021-03-25 09:18:29 +00:00
Paul Duffin
580efc8716 Add srcDir to SourcePath
Bug: 183650682
Test: m droid
Change-Id: I08f7dba449538d289a6c4d6b5325c1a833cff834
2021-03-25 09:18:29 +00:00
Jingwen Chen
95c6eb3959 Merge "bp2build: add allowlist for package-level conversions." 2021-03-24 21:38:07 +00:00
Jingwen Chen
12b4c2706d bp2build: add allowlist for package-level conversions.
This CL adds the support for specifying lists of directories in
build/soong/android/bazel.go, which are then written into
out/soong/bp2build/MANIFEST. Using this configuration,
modules/directories can either default to bp2build_available: true or
false, while still retaining the ability to opt-in or out at the module level.

It also ensures that ConvertWithBp2Build returns true iff the module
type has a registered bp2build converter.

Test: go tests
Test: demo.sh full
Test: TreeHugger presubmits for bp2build and mixed builds.

Change-Id: I0e0f6f4b1b2ec045f2f1c338f7084defc5d23a55
2021-03-24 02:27:19 -04:00
Colin Cross
fa65cee27f Fix genrule depending on disabled module with ALLOW_MISSING_DEPENDENCIES=true
If a genrule depends on a module that is disabled, in this case because
it is a device module in a host-only build, it can cause panics when
getPathsFromModuleDep retrieves a nil Path from the disabled module.
Treat disabled modules as missing dependencies.

Test: TestGenruleAllowMissingDependencies
Change-Id: I3c689c6b5505b21eaf7ae7cb93c00f96f438ac17
2021-03-23 16:17:37 -07:00
Liz Kammer
bdc609972c Support autoconverted modules in mixed builds
modules converted with bp2build_available are will also be available to
be used in mixed builds.

Test: build/bazel/scripts/milestone-2/demo.sh full
Test: go tests
Change-Id: I49f16ec3ba5bb11dfed8066af069c27eb04371fb
2021-03-16 15:11:49 -04:00
Petri Gynther
ac22956dcb Rename vendor-ramdisk to vendor_ramdisk
Rename to get consistent ramdisk directory naming in
out/target/product/<name>:
  debug_ramdisk
  ramdisk
  vendor_debug_ramdisk
  vendor_ramdisk

Test: build and inspect out/target/product/<name>
Change-Id: Ib86e6509bd910e8c95bcb6e3f7bb639ebbfe3758
2021-03-03 00:15:54 -08:00
Pedro Loureiro
8a75058d3f Merge "Add support for lint baseline files" 2021-02-18 15:53:13 +00:00
Pedro Loureiro
5d190cc24e Add support for lint baseline files
Test: m droid
Test: go test ^TestJavaLint # (from soong/build/java)

Change-Id: I249a0a0597b0bf8495460ed283b476ad2eb36edc
2021-02-18 11:15:30 +00:00
Treehugger Robot
5e3fa0a9a5 Merge "Clarify ExistentPathForSource docs" 2021-02-17 02:02:30 +00:00
Liz Kammer
ebfcf672ef Omit bazel.LabelList attrs if no path is specified
Test: go test build_conversion_test.go
Change-Id: Ibcd81049788e99422569fe74e5f3834df4a05122
2021-02-16 15:00:05 -05:00
Liz Kammer
7aa52883dd Clarify ExistentPathForSource docs
Specify that these paths are relative to SrcDir, not relative to a
module's local source directory.

Test: n/a
Change-Id: I9a80f19b2eb3a15e6276e98200796b0680af9ee7
2021-02-11 09:16:14 -05:00
Colin Cross
dcf71b299c Convert java.Dependency to JavaInfo provider
Export information about java dependencies through a Provider
instead of accessing the module directly.

Test: java_test.go
Test: no changes to build.ninja
Change-Id: Ifc5d566bf6f6ebc0ad399e948effaa1ef6a22876
2021-02-09 15:36:25 -08:00
Liz Kammer
356f7d45c1 bp2build: convert paths/module refs to Bazel label
This currently expands all globs, still need to support converting glob
syntax.

Test: go build_conversion_test
Test: GENERATE_BAZEL_FILES=true m nothing
Test: m nothing
Bug: 165114590
Change-Id: If7b26e8e663d17566fad9614ca87a8da1f095284
2021-02-04 13:45:56 -05:00
Paul Duffin
0267d49255 Make OutputPath implement objPathProvider
Needed to allow OutputPath to replace some usages of ModuleOutPath.

Bug: 179124768
Test: m droid
Change-Id: I0f746a11c53ed8c9c9193b57afc0b06c0249ff75
2021-02-02 13:33:51 +00:00
Liz Kammer
a830f3abc2 Refactor android/paths.go to allow reuse
Extracts out the relevant pieces of ModuleContext into a minimal
interfaces necessary for path handling for Modules. This will allow
reuse in queryview by allowing it to implement only the parts of the
module interface necessary for path expansion in order to properly
handle path properties.

Test: go test soong tests
Change-Id: Iea39a365e5a2ff55071e435fb64f9e6464bbcafd
2021-01-26 12:51:02 -05:00
Liz Kammer
2dd9ca422d Refactor queryview.
Splits queryview into queryview and bp2build. The latter runs as a
presingleton (could be converted to a singleton). This prevents needing
to run soong_ui a subsequent time to build the query. Queryview remains
as a separate step to prevent increasing runtime due to this generation
every time Soong runs.

Currently this is running as a presingleton as this gives a translation
of Android.bp files after only LoadHooks have run, no mutators.

Test: go tests
Test: m queryview && bazel query --config=queryview //...
Change-Id: If2ba28c7ef60fbd41f43bda6623d41c8c7d23a1b
2021-01-07 14:34:00 -05:00
Chris Parsons
dbcb1ff469 Use aquery to declare bazel actions in the ninja file.
This effectively moves execution of Bazel actions outside of soong_build
and alongside ninja execution of the actual ninja files, whether that be
by ninja or by Bazel itself.

This almost allows for mixed builds and Bazel-as-Ninja-executor to
coexist, but requires hacks explained in b/175307058.

Test: Treehugger
Test: lunch aosp_flame && USE_BAZEL_ANALYSIS=1 m libc
Test: lunch aosp_flame && USE_BAZEL=1 USE_BAZEL_ANALYSIS=1 m libc,
though this requires a hack of the main BUILD file. See b/175307058

Change-Id: Ia2f6b0f1057e8cea3809de66d8287f13d84b510c
2020-12-15 17:39:23 -05:00
Colin Cross
5d5839522c Export files to install as a depset
Export files to install through a depset instead of a list to reduce
the size at each module.

Bug: 124313442
Test: m checkbuild
Change-Id: I6a51c89acc9f0f3a9c3c792d3ef8a7cfd6b8bef2
2020-12-07 12:27:50 -08:00
Hans Månsson
d3f2bd79e8 WriteFileRule: Chunk long content and merge them to result
sbox.textproto files are created when handling genrule. The command
for the genrule and output files are written to this file in the
following format:

commands <
  command: "...."
  copy_after: <
    from: "out/asm-generic/auxvec.h"
    to: "out/soong/.intermediates/kernel/msm-4.19/qti_generate_kernel_headers_arm/gen/asm-generic/auxvec.h"
  >
  copy_after: <
    from: "out/asm-generic/bitsperlong.h"
     to:
     "out/soong/.intermediates/kernel/msm-4.19/qti_generate_kernel_headers_arm/gen/asm-generic/bitsperlong.h"
  >
  ....
>

This file grow by one copy_after entry for each output file.

When generating kenrnel headers where the number of output files are
~1000 we run into problems as the contents of sbox.textproto files
are written to disk by generating a shell script using the following
template: /bin/bash -c 'echo -e "$$0" > $out' $content
If $content is very long as in the case of generating kernel headers we
run into the issue where the command line is so long that the shell
script return E2BIG.

Fix this issue by chuking contents into smaller files and then merge
them as a final step.

Test: Build
Issue: 174444967
Change-Id: I1a74023b4222d19672e4df7edb19810a9cf2136f
2020-12-02 15:58:48 +01:00
Alex Humesky
29e3bbe8b8 Update comments in genrule.go.
Test: Noop
Change-Id: Ib341899ea0f04e059d5471ee6e5bff8ebba82583
2020-11-30 12:30:20 -05:00
Jingwen Chen
cda22c9bb9 Refactor 'in_make' to mean Kati is not skipped.
In Nougat and before, Make wrote a marker file to indicate that
soong_build was invoked from Make to change certain behaviors of Soong
at build time.
https://cs.android.com/android/platform/superproject/+/android-7.1.2_r36:build/core/soong.mk;l=70-73;drc=ae18638b0406ad107b0882a02a13cdd8b92f2a4e

Things have changed, and now soong_build is invoked from soong_ui, which
supports a --skip-make configuration flag:
https://cs.android.com/android/platform/superproject/+/master:build/soong/ui/build/build.go;l=31-33;drc=680387bf1d3ce7cbc77f535be7c42cec411b1687

Thus, the various remnants of 'EmbeddedInMake' and 'inMake'
configuration are misleading, since soong_build is no longer invoked
from Make. This CL refactors all instances to actually mean that
Kati is enabled (not skipped with --skip-make), and will run after
soong_build finishes, so Kati-specific behavior like the AndroidMk
singleton should run.

Test: TH presubmit

Change-Id: I576ab8e54f99f5c8ddf9feaf9a828019b279e266
2020-11-23 00:29:18 -05:00
Ulya Trafimovich
ccc8c85a7c Add ModuleInstallPathContextForTesting.
Test: m nothing
Change-Id: Ibcd6016cc43300fcc7f0ec511a3159886d0fdc7d
2020-10-29 17:47:34 +00:00
Treehugger Robot
07bc11722b Merge "Fix path conflict for vendor_ramdisk_available" 2020-10-27 06:27:12 +00:00
Treehugger Robot
f26f734867 Merge "InstallPath keeps its partition dir" 2020-10-27 01:33:12 +00:00
Yifan Hong
39143a941f Fix path conflict for vendor_ramdisk_available
... and ramdisk_available modules. If a module is both
vendor_ramdisk_available and ramdisk_available, on a device
that mark recovery_as_boot and move_recovery_resources_to_vendor_boot
simultaneously (and incorrectly),
both will be installed to recovery/root/first_stage_ramdisk. Fix the
path conflict of the two variants by moving the vendor_ramdisk variant
to vendor-ramdisk/first_stage_ramdisk instead.

Also update comments for Vendor_ramdisk_available.

Test: m nothing -j
Bug: 156098440

Change-Id: I2b776b6fd8f5a2c361c0f6a89231e3cebc2646f0
2020-10-26 12:50:05 -07:00
Yifan Hong
dd8dacc131 Vendor ramdisk modules install to correct location
Install to recovery/root/first_stage_ramdisk if
BOARD_MOVE_RECOVERY_RESOURCES_TO_VENDOR_BOOT,
otherwise vendor-ramdisk. In addition, append /system
if not InstallInRoot().

On devices with dedicated recovery partition,
BOARD_MOVE_RECOVERY_RESOURCES_TO_VENDOR_BOOT is not set,
and this installs to the correct place (under $OUT/vendor-ramdisk).

On devices without a dedicated recovery partition:
- To install a module available before switching root
  to /first_stage_ramdisk, e.g. a binary under /system/bin,
  use recovery{_available} and install the recovery variant
  of the module.
- To install a module available after switching root
  to /first_stage_ramdisk, e.g. a binary under
  /first_stage_ramdisk/system/bin,
  use vendor_ramdisk{_available} and install the vendor_ramdisk
  variant of the module.

Test: pass
Bug: 156098440

Change-Id: I1af3f8889891a3e58263cda36f0680ce2b480499
2020-10-22 10:26:26 -07:00
Yifan Hong
60e0cfb5cb Add vendor-ramdisk image to Soong.
Add vendor_ramdisk_available and vendor_ramdisk attribute to
various rules. When a vendor_ramdisk variant of a module is
generated, it is installed to $OUT/vendor-ramdisk.

It is similar to a ramdisk image.
Test: m nothing -j

Change-Id: Ib2d16459f3094dbe21c3bdb7c016cb4b2bf62765
2020-10-22 10:26:26 -07:00
Jiyong Park
957bcd9c32 InstallPath keeps its partition dir
This change introduces the concept of partition dir for InstallPaths.
It's the path to the partition where the InstallPath is rooted at. For
example, it's out/soong/target/product/<device>/<partitoon> for paths
created for device modules. For host modules, it is defined as
out/soong/host/<host_os>-<host_arch>.

The partition dir is obtained using the new PartitionDir() function.

Another change is that a freshly created InstallPath (usually via
PathForModuleInstall) is the result of joining PartitionDir() and the
remaining path elements. For example, PathForModuleInstall(ctx, "foo",
"bar").Rel() now returns "foo/bar". Previously, that call returned the
relative path from config.buildDir() ("out/soong"). This change is in
line with the behavior of other path-creating functions like
PathForModuleSrc where Rel() returns the path relative to the
contextually determined path like the module source directory.

Notice that the Join() call to InstallPath doesn't change
PartitionDir(), while does change the result of Rel().

p := PathForModuleInstall(ctx, "foo", "bar")
p.PartitionDir() is out/soong/host/linux-x86
p.Rel() is foo/bar

q := p.Join(ctx, "baz")
q.PartitionDir() is still out/soong/host/linux-x86
q.Rel() now returns baz

Bug: N/A
Test: m nothing
Change-Id: I916bb1c782a4bfe0fbd4854e349cd2a2a42f56b6
2020-10-22 22:04:08 +09:00
Jiyong Park
87788b5247 Don't assume host arch is always x86
This change fixes some places where host arch is assumed to be x86 to
form a path element like linux-x86. In preparation for non-x86 host
targets, the host arch part is derived from the context.

In addition, InstallForceOS() is refactored so that it can override not
only OsType, but also ArchType. Without this, the paths for the
robolectic modules will be changed from linux-x86 to linux-common, which
breaks several other places where the old paths are expected.

Bug: 134795810
Test: m
Change-Id: Ib38c715948ae546e55021ece82bac1d82e9e5da0
2020-09-08 15:12:10 +09:00
Ulya Trafimovich
5ab276ad1d Export error reporting function to allow using it in other packages.
Test: lunch aosp_cf_x86_phone-userdebug && m
Change-Id: I3a1d936ac57966b2f47cb1d9a56e0f96f47b807d
2020-08-25 12:45:15 +01:00
Chris Parsons
216e10a0f6 Install data_libs using relative_install_path property
Test: Added to data_libs tests
Test: Manually verified with bionic-unit-tests
Test: Treehugger
Change-Id: I28a8e08e3409f1e7c7bb72f4351310b57f35f513
2020-07-20 13:59:24 -04:00
Colin Cross
70f0b53000 Merge changes from topic "lint-unbundled-apps"
* changes:
  Build a zip of transitive lint reports for apps
  Add DepSets
2020-07-16 23:06:22 +00:00
Colin Cross
c0efd1db13 Build a zip of transitive lint reports for apps
Add a rule to build a zip containing the lint reports from transitive
dependencies for apps, and pass it to Make.

Bug: 153485543
Test: m TARGET_BUILD_APPS=Gallery2 lint-check
Change-Id: I523c09016251377ff89d76084769be7401b95425
2020-07-16 12:51:42 -07:00
Jooyung Han
ecc495fd09 Merge "Apply "excludes" for OutputFileProducer" 2020-07-08 06:40:08 +00:00
Christopher Parsons
0eeff82c98 Merge "Add missing soong_docs inputs" 2020-07-06 22:34:51 +00:00
Jooyung Han
7607dd3564 Apply "excludes" for OutputFileProducer
"excludes" should be applied both SourceFileProducer and
OutputFileProducer. But we've been missing the latter.
(see android/paths_test.go)

There's one more to note. Previously, the exclusion was done by
modifying slice directly, which caused some problem(b/76179848).
The fix was returning a copy from Srcs() method. But there're still many
implementors which just return internal slices directly.

This change creates a new array when it needs to exclude something from
returned slice.

Bug: n/a
Test: m (soong test added)
Change-Id: I1ba91e490cb55fc243020681667278cdc75f59df
2020-07-05 10:23:14 +09:00
Chris Parsons
8f232a2702 Add missing soong_docs inputs
Specifically, make Android.bp.list and soong.variables explicit inputs

While this is not a comprehensive list of all inputs
of this action (as the action depends on all blueprint
files in the source tree), this is closer to the truth.

This is a rollforward CL, which was originally rolled
back, as path validation failed for when OUT_DIR was
an absolute path. Validation has now been relaxed.

Test: Manually verified ninja output, checkbuild approved validation
  for aosp-crosshatch, and manually ran frameworks/rs/build_rs.py with
  patch fix
Change-Id: I4eb0d517f57336dd54eaa4bd31f46df9e93e6da2
2020-06-26 10:36:18 -04:00
Jingwen Chen
40fd90ae52 Support multiple dists per Android.bp module, and dist output selection.
This CL adds "dists" to the base property struct to support multiple
dist file configurations, and generic tag support to dist tagged outputs
of modules.

Fixes: b/152834186
Test: soong tests and `m sdk dist`

Change-Id: I80c86bc9b7b09e671f640a4480c45d438bdd9a2a
Signed-off-by: Jingwen Chen <jingwen@google.com>
2020-06-25 12:42:07 +00:00