Commit Graph

995 Commits

Author SHA1 Message Date
Treehugger Robot
7d76e8cc9d Merge "Fix kernel build tools" into main 2023-08-04 17:15:10 +00:00
Cole Faust
d1acaa4169 Add a linux_86 config option
Test: b test --config=linux_x86 //build/bazel/rules/apex/...
Change-Id: I4b98935f8664b93162b28229134cf4b56603002b
2023-08-03 17:04:03 -07:00
Spandan Das
59b8a6687d Merge "cc Bp2build support for genrules that generate .proto file" into main 2023-08-03 21:51:11 +00:00
Cole Faust
95c5cf88bd Fix kernel build tools
That build doesn't set platform_sdk_version.

Bug: 269577299
Test: ./prebuilts/kernel-build-tools/build-prebuilts.sh on the kernel-build-tools branch
Change-Id: I68eb33bfb80770cef9c3a9c5e01f1ed746a2d069
2023-08-03 13:49:27 -07:00
Spandan Das
a99348dca4 cc Bp2build support for genrules that generate .proto file
If `srcs` contains a gensrcs/genrule module, the current bp2build module
will put it in the catch-all `srcs` attribute. This is reserved for .cpp
sources, so if the genrule produces a .proto/.aidl/... file, this will
fail.

This handles genrules that produce .proto files. To implement this, this
creates an additional partition that detects if the other module is a
genrule/gensrc that produces .proto files. If true, it will append it to
the proto partition.

This CL does not handle
- genrule that produce .c/.aidl/.yacc/.... files. They will continue to
  be partitioned into the catch-all partition
- java modules

Test: unit tests
Test: TH
Bug: 293205700
Change-Id: Ib720fdf3a053ff5dd256e6faa632e3fa7776966d
2023-08-03 18:56:05 +00:00
Cole Faust
87c0c33a65 Make bp2build-generated selects() based on product config build settings
...instead of based on constraint settings.

Bug: 269577299
Test: m nothing and ./build/bazel/ci/bp2build.sh
Change-Id: Ib9caec79c92b8fd304e46be841de5612bd1637e3
2023-08-03 10:12:09 -07:00
Treehugger Robot
687041c9e0 Merge "Add go modules to bp2build progress dashboard" into main 2023-08-02 12:02:19 +00:00
Spandan Das
41f1eeef91 Add go modules to bp2build progress dashboard
Since these modules have been converted, we should tag them as converted
in the dashboard.

Test: Put a print statement for `metrics` and made sure that the go
modules appear there
Bug: 294098662

Change-Id: I6282100111030a94f15f330916eaf41fcfc16e1a
2023-08-01 22:28:16 +00:00
Cole Faust
bc65a3fea8 Revert^2 "Initial implementation of the bazel sandwich"
c13fad8181

Change-Id: I478562c8fd89e62983feb5b52b62aad851d40f00
2023-08-01 10:27:57 -07:00
Jooyung Han
09195f4ffd Merge "Revert "Initial implementation of the bazel sandwich"" into main 2023-08-01 06:50:25 +00:00
Jooyung Han
c13fad8181 Revert "Initial implementation of the bazel sandwich"
Revert submission 2651299-bazel_sandwich

Reason for revert: b/293883239, checking if this breaks the build.

Reverted changes: /q/submissionid:2651299-bazel_sandwich

Change-Id: I5f1d4bb13d21e6599f5c353dcaba2375f5ec234d
2023-08-01 05:15:17 +00:00
Treehugger Robot
61a27f80cd Merge "Initial implementation of the bazel sandwich" into main 2023-07-31 23:00:55 +00:00
Cole Faust
a20d947329 Initial implementation of the bazel sandwich
The "bazel sandwich" is a mechanism for bazel to depend on make/soong
outputs. The name comes from the fact that bazel is now at the top
and bottom of the build graph. This is intended to allow us to work
on converting the partition builds to bazel while not all of the
dependencies of the partition have been converted.

It works by adding the bazel_sandwich_import_file rule, which emits a
dangling symlink that starts with bazel_sandwich:, and includes
information that the aquery handler in soong reads. The aquery handler
rewrites the symlink so that it points to a file generated by
make/soong, and adds a ninja dependency from the symlink to the file
it's targeting.

This allows us to depend on make-built files from bazel, but notably
it doesn't allow us to depend on analysis-time information from make.
This shouldn't be a problem for the partitions, but limits the use of
the bazel sandwich to similar, less complicated types of builds.

go/roboleaf-bazel-sandwich

Bug: 265127181
Test: m bazel_sandwich
Change-Id: Ic41bae7be0b55f251d04a6a95f846c50ce897adc
2023-07-31 11:53:41 -07:00
Treehugger Robot
dc3606a81d Merge "Gen a header library when genrules export includes" into main 2023-07-31 17:48:44 +00:00
Spandan Das
3671dcd39a Merge changes Ie292173d,I23f14f83 into main
* changes:
  Panic if soongConfig and productConfig values do not match
  Check if soong_config_var propertystruct is zero before panicking
2023-07-28 16:26:44 +00:00
Treehugger Robot
6677600f35 Merge "Emit gtest in BUILD files only if Android.bp sets it" into main 2023-07-27 23:52:46 +00:00
Spandan Das
474d43bda6 Check if soong_config_var propertystruct is zero before panicking
If we have a soong_config_module_type with target.android_<arch>.*
properties, we should only panic if a soong_config_var is used to select
on those properties. We already call `IsZero` in an upper function, but
we need to call it in `AddSoongConfigPropertiesFromTargetStruct` as well
because conditions_default gets its own struct.

Test: unit tests
Test: repro'd cmd in b/293206386#comment1, it passes now
Bug: 293206386
Change-Id: I23f14f83cc9be66487625e292e26b204dfea4f9a
2023-07-27 23:18:57 +00:00
Spandan Das
a11df791ed Merge changes from topic "go-test-bp2build" into main
* changes:
  Do not convert soong's tests in bp2build
  Create additional test target for go modules in bp2build
2023-07-27 23:16:29 +00:00
Spandan Das
89aa0f71c9 Do not convert soong's tests in bp2build
These tests cannot be run with b since it uses functions that should not be
used outside tests, and it enforces this by looking at os.Args. Since
the pattern (`-test`) is missing in os.Args, these tests panic in b.

A module can be a dep of soong_build via two mechanisms
1. Listed in deps explicitly
2. Via plugin_for to create a reverse dep

This CL denylists (1) explicitly. e.g. soong-android. Another option
would have been to denylist by using Deps() of soong-build, but that
would denylist `golang-protobuf-proto` as well

This CL denylists (2) programmatically.

Note that this CL only denylists creation of the `go_test` target. We
will still create `go_binary` and `go_library` for soong since there
might be tools that depend on soong.
e.g.
```
--> depends on
dexpreot_gen --> soong-dexpreopt --> soong-android (test failure)
```

(The above example means that we might need to convert _some_ of soong
after all, but we can refactor the common bits to common utils. Keeping
this out of scope for now).

Test: b test //build/soong/... //system/tools/...
too)
Bug: 284483729

Change-Id: I809012bdd383febca30da95e991edfde5fd4d9b9
2023-07-27 17:11:51 +00:00
Spandan Das
682e78686b Create additional test target for go modules in bp2build
For go modules with non-empty testSrcs, we will create an additional
go_test target. To build a standalone test executable, we need to
include the source files in the compilation unit. This will be done
using the `embed` attribute
1. For tests of go_library, this is straightforward. It will embed the
   go_library and "inherit" its .go files and deps
2. For tetss of go_binary, we need to create an additional go_source
   target since go_binary cannot be embedded inside a go_test

Using `b test` for these tests revealed that certain tests are not
hermitic and rely on `testdata` files checked into the tree. This CL
introduces an allowlist to skip generating go_test targets for them.

Test: bp2build unit test
Test: TH
Bug: 284483729
Bug: 288491147
Change-Id: Ic736d655babc2f6067e4da75384900b7b8bdc2ed
2023-07-27 17:11:42 +00:00
Spandan Das
e50fd111f3 Emit gtest in BUILD files only if Android.bp sets it
cc_test bazel macro will default `gtest` to True (similar to Soong). So
we can skip explicitly setting this in the generated BUILD files. The
benefit will be that it will make the generated files less verbose,
without loss of information.

This will be implemented by changing its datatype to *bool from bool.

Test: go test ./bp2build
Test: TH
Change-Id: I284e10f1d58c8e7893b170209827f7d5084ca95e
2023-07-26 17:42:21 +00:00
Spandan Das
d1cd3518a8 Support cc_test.isolated in bp2build
This property implicitly adds `libgtest_isolated_main` to the static
libs of the test. bp2build will make this and `liblog` explicit in BUILD
files

Since the deps are made explicit in BUILD files, `isolated` becomes a
no-op for cc_test. Remove this property from cc_test.

Test: unit tests
Bug: 244432609
Change-Id: I189a7b6b62d9064f4b2abad49ac4975468046498
2023-07-26 17:06:27 +00:00
Spandan Das
f5a8655cd8 Fix difference in default cc_test.isolated in soong and bp2build
Soong defaults to false, while bp2build defaults to true. This CL fixes
bp2build's default to false to match Soong.

To prevent future differences, bp2build now uses the `isolated()`
function used by Soong. The type of the context object has been changed
to EarlyModuleContext so that the ctx object does not depend on
mutations that might not run in bp2build.

Test: TH
Bug: 244432609
Change-Id: I15346107896312961e8d12270cf2f9a2a48827e1
2023-07-26 17:06:27 +00:00
Treehugger Robot
8fbc42cfd7 Merge "Create build settings for all product config variables" into main 2023-07-26 02:41:46 +00:00
Cole Faust
f055db60d4 Create build settings for all product config variables
Bug: 269577299
Test: Presubmits
Change-Id: Id8b7d65d657fa20ad8591e58d5173445b0e4f4df
2023-07-25 11:47:20 -07:00
Treehugger Robot
35448be862 Merge "Make libtest_* deps explicit in BUILD files" into main 2023-07-25 18:46:39 +00:00
Liz Kammer
0db0e34c68 Gen a header library when genrules export includes
Embedding multiple includes from a genrule may be difficult to read,
instead we generate a header library that contains the headers and all
include dirs that Soong generates.

Test: go test bp2build tests
Change-Id: I590c74c133f015f27cccf5a2fd916153ad9c125e
2023-07-24 13:57:17 -04:00
Spandan Das
651203de35 Make libtest_* deps explicit in BUILD files
This is currently implicitly added by the cc_test macro. This can cause
duplicate issues if users write a BUILD file as
```
cc_test (
  name = "mytest",
  deps = ["libgtest"],
)
```
Making this dep explicit in BUILD files prevents this issue.

The property `gtest` will not be removed as part of this CL. cc_test
macro will use this property to add some gtest specific copts

Test: unit tests
Bug: 249374572
Change-Id: Ife931e26367fd4aab302bedc709cf907e2c01245
2023-07-21 23:04:32 +00:00
Zi Wang
8348ce92f5 Merge "java_test macro needs srcs and deps to create .jar for tradefed_test_suite" into main 2023-07-21 20:48:13 +00:00
Romain Jobredeaux
cc1b67631c Add DeviceAbi to product variable to build setting export
Change-Id: Ibcb9e77a28bf0c4c0915f7013a7408f65429ae15
2023-07-21 10:05:28 -04:00
Cole Faust
88c8efb18c Add CFI product config variables to platform_mappings
Bug: 269577299
Fixes: 283130542
Test: b test --config=android //build/bazel/rules/cc/...
Change-Id: I2fc4094167c48b1f3b22bee6b7ada309278f7250
2023-07-20 09:45:36 -07:00
Spandan Das
958ca02582 Merge changes from topic "build_go_source_mixed_builds" into main
* changes:
  Delete aliases to prebuilts
  Add functionality to sandbox mixed build actions
2023-07-19 17:21:25 +00:00
Liz Kammer
0a681989ae Merge "Handle static binary repetition of system deps" into main 2023-07-19 14:16:52 +00:00
Zi Wang
7873f613c4 java_test macro needs srcs and deps to create .jar for tradefed_test_suite
Test: java_test_host_conversion_test.go and TH

Bug: 280452825
Change-Id: Ibf0c7eed415fc58a4d228f0347ab125fdc4466e4
2023-07-17 16:39:07 -07:00
Cole Faust
f8231dd0ea Platform mapping-based product config
This allows us to set product variables as build settings instead
of loading them from a target's provider, which further allows us
to read product config variables in transitions.

Bug: 287539062
Bug: 269577299
Test: Presubmits
Change-Id: I8497703f706162572ceb3486240e1eb02a37f5f6
2023-07-17 16:27:08 -07:00
Alix Espino
2a779ea829 Merge changes from topics "errProne_bp2build_manually_enabled", "error_prone_config" into main
* changes:
  Bp2build for errorprone modules that manually enabled/disabled it
  Make errorprone a configurable attribute for bazel conversion
2023-07-17 13:54:50 +00:00
Trevor Radcliffe
3edec2151e Merge "Update name of blocklist feature in bp2build" into main 2023-07-14 19:59:50 +00:00
Spandan Das
2a55cea4a0 Delete aliases to prebuilts
These tools will now be built from source using rules_go

Test: TH
Bug: 284483729
Change-Id: I6dde9f1418aaa516c4c6f8c8897dd20f4becfb51
2023-07-14 00:43:57 +00:00
Spandan Das
06f3c9bb42 Merge changes from topic "go_bp2build" into main
* changes:
  Respect package boundaries in bp2build conversion of go modules
  Create a temporary denylist for go binaries used in mixed builds
  Partial bp2build conversion of blueprint_go_binary
  Partial bp2build conversion of bootstratp_go_package
2023-07-13 16:16:00 +00:00
Alix
b1e5c6a69a Bp2build for errorprone modules that manually enabled/disabled it
Test: go test ./bp2build
Change-Id: Ie60c0959ee9ae8ce86c11a8e85a0bc7592f63df8
2023-07-13 15:50:18 +00:00
Spandan Das
0a8a27500e Respect package boundaries in bp2build conversion of go modules
bp2build's codegen context does not implement
BazelPathConversionContext. To reuse the utility function
transformPackagePaths, update its signature

(Also make deps of go_library unique to make the conversion resilient)

Test: go test ./bp2build
Change-Id: I126b1057d2b26bc6c7d3be2780f1b62d28323cf0
2023-07-13 00:22:26 +00:00
Spandan Das
69afa98fbd Create a temporary denylist for go binaries used in mixed builds
This allows us to rollout building _some_ go targets using rules_go
without affecting mixed builds.

Test: Presubmit
Bug: 284483729
Change-Id: I0ccb4c9b90614369147a380f44f7ae372ef9396e
2023-07-13 00:22:26 +00:00
Trevor Radcliffe
d71487173e Update name of blocklist feature in bp2build
Bug: 290908194
Test: Forrest
Change-Id: I0576fb644edb6b6043df9cf01b508ee887f946f1
2023-07-12 13:45:10 +00:00
Spandan Das
de623294fe Partial bp2build conversion of blueprint_go_binary
This module type does not implement android.Module, and therefore we
cannot write a conventional bp2build converter for this module type.
Instead this has been special-cased inside bp2build/build_conversion.go.

There is one major deviation between Soong and Bazel's
go_binary/go_library. Soong
collects the deps in the transitve closure and puts them on compile/link
paths. Bazel OTOH, requires the direct imports to be listed in deps of the binary
explicitly (AFAIK). Since bp2build cannot determine the list of direct
imports from the list of transitive deps, put all the transitive deps in
`deps`

Test: unit tests
Test: TH
Bug: 284483729
Change-Id: I004aaf8607fef1697a0d9e7d018ad657b67778ac
2023-07-11 22:03:48 +00:00
Spandan Das
ea2abba3a9 Partial bp2build conversion of bootstratp_go_package
This module type does not implement android.Module, and therefore we
cannot write a conventional bp2build converter for this module type.
Instead this has been special-cased inside bp2build/build_conversion.go.

Because of the above, we also do not have access to useful functions
available in the ctx object of ConvertWithBp2build. This includes
1. Finding the package (directory) of a dep. This requires getting a
   handle of the underlying module from its name (string). To solve, we
   do a pre-visit to collect this information. This did not increase the
   wall time. On my machine, `m bp2build --skip-soong-tests` takes ~14s
   before and after this CL
2. Converting srcs to labels. This requires glob and package boundary
   resolution. This CL introduces a partial implementation for this
   function. (glob patterns are not used in go tools)

For (1), I considered creating a `ModuleFromName` on
`blueprint.Context` instead of a pre-run, but this increased the time to ~27s.

Test: unit tests
Test: TH
Bug: 284483729

Change-Id: Ifeb029103d14947352556dba295591dd7038b090
2023-07-11 22:03:26 +00:00
Romain Jobredeaux
496660d38a Support target and max sdk version in app bp2build.
Test: Unit tests + CI
Bug: 285304294
Bug: 280465047
Change-Id: I9bb1b6f9729ad783a43837d77dcf2d02b50ca77e
2023-07-10 11:50:00 -04:00
Liz Kammer
b492843c3f Handle static binary repetition of system deps
Test: b build `bmod toybox-static`
Test: go test bp2build tests
Change-Id: Id728f6fd08832a4fc153f0ff5282cdfb6b19c2f6
2023-07-10 10:28:02 -04:00
Cole Faust
8ee97f7a20 Merge "Revert "Add unit test for parsing build files in bp2build"" 2023-06-27 22:52:59 +00:00
Cole Faust
3d927238cc Revert "Add unit test for parsing build files in bp2build"
This reverts commit e1f25230df.

Reason for revert: breaks bp2build ci

Change-Id: I7320ef679d8a1ba1d605cf0d3781854b97816358
2023-06-27 22:52:07 +00:00
Treehugger Robot
5694f595bb Merge "Add unit test for parsing build files in bp2build" 2023-06-27 17:46:09 +00:00