1037 Commits

Author SHA1 Message Date
Spandan Das
4e5a194b90 Add proto.local_include_dirs support in bp2build
This is a followup to aosp/2711093 which added support for
proto.include_dirs. local_include_dirs is simlar, except that is
relative to the module directory.

Test: go test ./bp2build
Bug: 285140726
Change-Id: I32ddc7371048672d6935f827c8aee9d767305e2c
2023-08-24 19:00:12 +00:00
Spandan Das
e0f2ed56a0 Add manual tag to proto_library at the root package
This is a fix similar to aosp/2707793. This adds manual tags to the
top-level proto_library created in the root package, plus any
dynamically created proto_library targets for proto.include_dirs. These
proto_library targets might not specify their deps correctly and might
be unbuildable.

(We need to keep these targets because they provide a ProtoInfo for
cc/java/py source gen)

Test: go test ./bp2build
Change-Id: Ic00f05186327fcfcc8d33a0a2c0891ed619b7acb
2023-08-24 19:00:11 +00:00
Jason Wu
66de6c445f Merge "bp2build converter for sh_test followup" into main 2023-08-24 15:19:28 +00:00
Vinh Tran
80f6b2147f Export RustDefaultVersion to Bazel
The config constant is consumed by rust toolchain implemented in aosp/2709668.

Bug: 295918553
Test: go test
Change-Id: I0a28a802ad28f6ee24dfb1b5bdab72094534ae04
2023-08-23 22:55:40 -04:00
Jooyung Han
138ee09e28 Merge changes from topic "deprecate-zip-apex" into main
* changes:
  apex: Remove host support
  apex: Remove apexPackagingMutator
  apex: Remove 'zip' apex support
2023-08-23 23:48:43 +00:00
Jooyung Han
8d4a1f03b0 apex: Remove host support
Since we no longer support "zip" APEX, we don't need "host" support.

For example, we don't need go/python binary support.

Bug: 279835185
Test: m
Change-Id: I6e8d2b205e42662f31866dc9ac7507524effd144
2023-08-23 14:01:21 +09:00
Jason Wu
6d8d44af24 bp2build converter for sh_test followup
Test: m nothing
Bug: 283486885
Change-Id: Ib8229e75dfcd9fd251fb1a83485cf5f88bdc3afb
2023-08-22 23:07:33 -04:00
Yu Liu
d136a6aa01 Support dynamic config file for cc_test.
Bug: 280028705
Test: local build
Change-Id: Ife445f6a7642bf08651ed7a9501a60950f05f9c4
2023-08-22 17:33:20 +00:00
Treehugger Robot
4147e5c1d1 Merge "Handle proto.include_dirs in bp2build for CC" into main 2023-08-21 20:43:02 +00:00
Yike Zhang
fd7870f231 Merge "Give cc_test rule information to determine the test types" into main 2023-08-18 17:28:05 +00:00
Treehugger Robot
c21f48def3 Merge "Fix replacements of namespace module srcs in genrule" into main 2023-08-18 02:10:13 +00:00
Spandan Das
f62e80a127 Fix replacements of namespace module srcs in genrule
expandSrcsForBazel always prefixed : in OriginalModuleName. The
exceptions to this are filegroups that appear in a different soong
namespace. For these cases, we were not correctly substituting the soong
module name with the equivalent bazel label.

Test: go test ./bp2build
Change-Id: If090f3f8819835177c1f4d191b3eef6bb6e30ace
2023-08-17 22:50:07 +00:00
yike
fdca7fe03b Give cc_test rule information to determine the test types
There are three types of tests, deviceless tests, device-driven tests
and host-driven device tests. But currently we don't have information
to get the type of a test and can't generate test targets on desired
types.

Test: b test //platform_testing/tests/example/native:hello_world_test
Test: b test //packages/modules/adb:adbd_test
Bug: 296312548
Change-Id: I3f022ef769636d508e055477623a4d1a6a1d9044
2023-08-17 21:28:47 +00:00
Trevor Radcliffe
2ec55ba261 Merge "bp2build for sanitizer blocklists long term fix" into main 2023-08-17 21:02:03 +00:00
Romain Jobredeaux
491fa2fe50 Merge "Support asset_dirs property in bp2build for android_{app,library}" into main 2023-08-17 17:43:18 +00:00
Romain Jobredeaux
7a71e07040 Support asset_dirs property in bp2build for android_{app,library}
Bug: 276928228
Test: Unit Tests
Change-Id: I6a2899de21e046e54cd0cd71314bf7aec4a470f6
2023-08-17 10:18:55 -04:00
Spandan Das
ec39d516af Handle proto.include_dirs in bp2build for CC
Soong's proto.include_dirs becomes the -I path for aprotoc cpp code
generation. This does not translate well to Bazel because it runs this
action in a sandbox. Even if we construct an -I path, the .proto files
will not be there. This CL attempts to handle this kind of dependency
automatically via bp2build.

For this hypothetical example
```
foo.proto # contains `import bar.proto"
Android.bp # cc_library {srcs:"foo.proto", p.include_dirs:["subdir"]},

subdir/bar.proto

```

Implementation details for CcProtoGen of foo
- Glob the labels of .proto files for each includeDir, and create a
  proto_library that encapsulates them.
- Since Bazel poses a contraint that proto_library target needs to be
  in the same pacakge as the .proto file, the proto_library might be created
  in a subdirectory with an import_prefix
- Add bar's proto_library as transitive deps of foo's cc_proto_library.
  This will be added to -I during aprotoc. We cannot add them to `deps`,
  otherwise bar's symbols will be statically linked into foo's .a
  file.

Implementation details for clang compile of foo
At the end of CcProtoGen, we have converted foo.proto
to .cpp/.h files. To compile them to .a files, we need the .h files
generated from bar.proto. Soong specifies this at the
top-level cc_library soong module, so add those deps as the
implementation deps for clang compile.

(Will add support for java in a follow-up CL)

Test: go test ./bp2build
Test: built some internal modules that were previously blocked by this
Bug: 285140726

Change-Id: I7e1f9f0d1b1ba916a7ba8278f6cfb342b381d695
2023-08-17 00:16:00 +00:00
Treehugger Robot
f768e6e27e Merge "Bp2build product variables on non-arch-variant module types" into main 2023-08-15 21:51:31 +00:00
Cole Faust
ed940008ac Bp2build product variables on non-arch-variant module types
Mostly so that genrules can have their command qualified on a product
variable.

Bug: 295910468
Test: m nothing
Change-Id: I28cc18a1b3b00368f4768a1326259a656ce970ec
2023-08-15 11:59:24 -07:00
Liz Kammer
d365cc4d86 Merge changes from topic "bazel_module_labels" into main
* changes:
  Set Bazel_module for (un)converted sysprop libs
  Make ShouldConvertWithBp2build avail in loadhooks
2023-08-15 14:11:25 +00:00
Cole Faust
9db358d43d Merge "Bp2build genrule commands as configurable properties" into main 2023-08-14 21:01:48 +00:00
Trevor Radcliffe
d9b7f17f37 bp2build for sanitizer blocklists long term fix
Bug: 286872909
Test: Unit tests
Test: b build relevant targets
Change-Id: I553091f76fca936006651b1ed22c8fe4d176e18f
2023-08-14 19:29:39 +00:00
Spandan Das
215adb43d3 Fix possible orphaned proto_library targets
This is a fix for aosp/2693190 that handled .proto files that end up in
different bazel packages. It did it by creating proto_library targets in
the correct bazel package.

Changing the granularity causes issues if the the new proto_library in
the subpackage imports a .proto file from a parent package or a
different package. e.g.
```
tmp
├── foo.proto
└── subdir/import_foo.proto # contains an `import "foo.proto"`
└── subdir/Android.bp # package boundary
├── Android.bp # contains a cc_library with foo.proto and
# subdir/import_foo.proto
```

At ToT, the ProtoInfo we provide to CcProtoGen is correct, but the
proto_library in subdir/BUILD will not compile because it does not have
a dep on the proto_library in ./BUILD

This CL creates a workaround by adding `manual` to the proto_library
targets. This CL is based on the assumption that the buildable unit in
bp2build is cc_library_*, and not proto_library necessarily (atleast
till we do a manual/automated cleanup)

Test: Created an integration test in build/bazel
Test: go test ./bp2build
Bug: 292583584
Bug: 246997908
Change-Id: I73120be2411967cb144f37ed4417f76ecf1a6ffa
2023-08-14 19:17:39 +00:00
Cole Faust
f0d4d4f9f2 Bp2build genrule commands as configurable properties
So that users can use soong config variables / product variables
to adjust a genrule's command.

Bug: 295910468
Test: m nothing
Change-Id: I9fedf8d5d52e515c3fdb913411ce1b3fecb7ba81
2023-08-14 11:58:20 -07:00
Liz Kammer
c86e094e13 Make ShouldConvertWithBp2build avail in loadhooks
Test: Soong go tests
Change-Id: I65af040152107ce50b1a97efcee9364f28bad08b
2023-08-14 11:03:05 -04:00
Juan Yescas
83a740e888 Merge "soong: Add product variable DevicePageSizeAgnostic" into main 2023-08-11 05:45:44 +00:00
Liz Kammer
7dc6bcbd58 fix protos in another dir + a module that uses it
Protos in another directory were using import prefix, which was
prepending the repository-relative path with the value, instead, we want
to strip the prefix of the directory of the module the protos were used
in such that they can be referenced at the appropriate relative path.

Reference on import_prefix:
https://bazel.build/reference/be/protocol-buffer#proto_library.import_prefix

Test: b build //packages/modules/adb:libfastdeploy_host
Change-Id: If050b0f5fc5103bd9cc5a99703bd604325aa4204
2023-08-10 14:19:39 -04:00
Juan Yescas
0106560e34 soong: Add product variable DevicePageSizeAgnostic
The product variable DevicePageSizeAgnostic will determine
whether AOSP is kernel page size agnostic or not.

Test: source build/envsetup.sh
      lunch aosp_cf_arm64_phone_pgagnostic
      m
      cat out/soong/build.aosp_cf_arm64_phone_pgagnostic.ninja | grep __BIONIC_NO_PAGE_SIZE_MACRO
Bug: 289419664
Change-Id: I33a1a4f1967d7e78432b7b8d90357d16b2002dcc
2023-08-09 16:46:07 -07:00
Treehugger Robot
69bda98f50 Merge changes I816f209e,I9e4d51c3 into main
* changes:
  Handle enabled: false via conditions_default
  Handle nil enabled values
2023-08-08 19:35:54 +00:00
Spandan Das
846ce68a2f Handle enabled: false via conditions_default
In this Android.bp file
```
my_cc_defaults {
  enabled: false,
  soong_config_variables: {
    my_bool_variable: {
       conditions_default: {enabled: false},
    }
  }
}
```
The inner enabled: false is a no-op because the top-level enabled is
false. Currently, bp2build will raise an exception for this Android.bp
file.

However, it does not need to. `productVariableConfigEnableLabels` runs
only if the top-level enabled is false. If it sees enabled: false via
conditions_default, it should just ignore it since it is a no-op.

Test: go test ./bp2build
Bug: 210546943
Change-Id: I816f209eaf21de65ddfbc2893e5255be94bcaa11
2023-08-08 16:54:34 +00:00
Treehugger Robot
21b19f9ca8 Merge "Use string.Builder" into main 2023-08-08 14:35:15 +00:00
usta
da2a211bbd Use string.Builder
with 4000X clones of adbd, bp2build.Codegen time goes from 36s to 7s

Test: m bp2build and verify same BUILD.bazel files
Bug: 292281398
Change-Id: Ibd3b328a917fe5d1bdad67c67116da336f9033cb
2023-08-08 13:29:07 +00:00
Spandan Das
479e39f8fb Handle nil enabled values
If enabled does not appear inside `soong_config_vars`, we can ignore it.

Bug: 210546943
Test: go test ./bp2build

Change-Id: I9e4d51c3b683f262921449634f827915ce87dc8d
2023-08-08 02:19:38 +00:00
Spandan Das
09f6b1390c Merge changes I8ca452aa,I437fc890,If9446700 into main
* changes:
  Handle .proto files in different package for filegroups
  Handle .proto files that end up in a different package
  Allow creation of BazelTargets in a different directory
2023-08-08 00:51:52 +00:00
Spandan Das
df3ec82b62 Handle .proto files in different package for filegroups
Followup to aosp/2693190, this CL adds the support to filegroups.
<mod>__bp2build_converted is now an alias to a proto_library
target <mod>_proto. This proto_library will be created in a different
package if the .proto file exists in a different package.

Test: bp2build unit tests
Test: TH
Bug: 292583584
Change-Id: I8ca452aacf1a86dfc9e218464e38aab89afa5a29
2023-08-07 21:39:05 +00:00
Spandan Das
c53767e434 Handle .proto files that end up in a different package
Bazel poses a strict requirement that .proto files and proto_library
must be in the same package. This CL handles this automatically by
creating the proto_library in a separate dir/package if necessary

Implementation details
- Partition the `srcs` by package. `srcs` has been computed using
  `transformSubpackagePath`, so the information about packages is
  available at this point
- Create a proto_library in each package by using
  `CommonAttributes.Dir`. Collect all these additional libraries
  and put them in `info.Proto_libraries` so that they get added as deps
  of (cc|python|...)_proto_library
- Add an import_prefix to the proto_library in subpackages relative to
  the current directory. This relies on the assumption that every src is
  beneath the current directory (Soong will complain if a path in
  Android.bp contains ../)

filegroup module type uses a separate code-path to create proto_library.
This will be handled in the next CL in stack.

Test: bp2build unit tests
Test: TH
Test: Built the failing internal module mentioned in
b/292583584#comment1

Bug: 292583584

Change-Id: I437fc89092321b26c5f0511387cde9e84084d6f9
2023-08-07 19:18:03 +00:00
Spandan Das
3131d679f2 Allow creation of BazelTargets in a different directory
The current API restricts creation of targets to the directory of the
visited soong module. This CL proposes adding a `Dir` property in
`CommonAttributes` that can be used to create a bazel target in
a specific dir. The use case for this is to dynamically create
additional targets for proto_library that are adjacent to .proto files
(Bazel poses a strict requirement about proto_library being in the
same package as the .proto file, but Soong does not)

Usage is restricted to dirs that have an existing Android.bp file. There
are some places in bp2build where we use existence of Android.bp/BUILD
on filesystem to curate a compatible fully qualified path (e.g. headers).
If we use `CommonAttributes.Dir` to arbritraily create BUILD
files, then it might render those curated labels incompatible.

Test: go test ./bp2build

Change-Id: If9446700457eddfb389be9d9bde39087f67daa60
2023-08-07 19:17:03 +00:00
Treehugger Robot
38a72e717b Merge "cosmetic: unused argument" into main 2023-08-07 14:43:51 +00:00
Treehugger Robot
7660e71895 Merge "add bp2build subevents" into main 2023-08-07 14:43:01 +00:00
Cole Faust
946d02cb80 Remove final usages of product_vars_providing_rule
Bug: 269577299
Test: Presubmits
Change-Id: Iad1df30ba2ff0256e56919411d98c1bd9f7fa5a7
2023-08-04 14:58:05 -07:00
usta
40caf95004 cosmetic: unused argument
Test: m nothing
Bug: NA
Change-Id: I2a647e2e7c9c2fd3881c18e3cbb072b260cd9659
2023-08-04 16:53:31 -04:00
usta
aaf2fd13e1 add bp2build subevents
Test: verified new events in bp2build_metrics.pb
Bug: 286412441
Change-Id: Ib803c3a3d1a8e7eaae5fb4c01c7769a72f4664ff
2023-08-04 16:47:41 -04:00
Treehugger Robot
7d76e8cc9d Merge "Fix kernel build tools" into main 2023-08-04 17:15:10 +00:00
Cole Faust
d1acaa4169 Add a linux_86 config option
Test: b test --config=linux_x86 //build/bazel/rules/apex/...
Change-Id: I4b98935f8664b93162b28229134cf4b56603002b
2023-08-03 17:04:03 -07:00
Spandan Das
59b8a6687d Merge "cc Bp2build support for genrules that generate .proto file" into main 2023-08-03 21:51:11 +00:00
Cole Faust
95c5cf88bd Fix kernel build tools
That build doesn't set platform_sdk_version.

Bug: 269577299
Test: ./prebuilts/kernel-build-tools/build-prebuilts.sh on the kernel-build-tools branch
Change-Id: I68eb33bfb80770cef9c3a9c5e01f1ed746a2d069
2023-08-03 13:49:27 -07:00
Spandan Das
a99348dca4 cc Bp2build support for genrules that generate .proto file
If `srcs` contains a gensrcs/genrule module, the current bp2build module
will put it in the catch-all `srcs` attribute. This is reserved for .cpp
sources, so if the genrule produces a .proto/.aidl/... file, this will
fail.

This handles genrules that produce .proto files. To implement this, this
creates an additional partition that detects if the other module is a
genrule/gensrc that produces .proto files. If true, it will append it to
the proto partition.

This CL does not handle
- genrule that produce .c/.aidl/.yacc/.... files. They will continue to
  be partitioned into the catch-all partition
- java modules

Test: unit tests
Test: TH
Bug: 293205700
Change-Id: Ib720fdf3a053ff5dd256e6faa632e3fa7776966d
2023-08-03 18:56:05 +00:00
Cole Faust
87c0c33a65 Make bp2build-generated selects() based on product config build settings
...instead of based on constraint settings.

Bug: 269577299
Test: m nothing and ./build/bazel/ci/bp2build.sh
Change-Id: Ib9caec79c92b8fd304e46be841de5612bd1637e3
2023-08-03 10:12:09 -07:00
Treehugger Robot
687041c9e0 Merge "Add go modules to bp2build progress dashboard" into main 2023-08-02 12:02:19 +00:00
Spandan Das
41f1eeef91 Add go modules to bp2build progress dashboard
Since these modules have been converted, we should tag them as converted
in the dashboard.

Test: Put a print statement for `metrics` and made sure that the go
modules appear there
Bug: 294098662

Change-Id: I6282100111030a94f15f330916eaf41fcfc16e1a
2023-08-01 22:28:16 +00:00