Commit Graph

243 Commits

Author SHA1 Message Date
Jingwen Chen
228ef1e370 Allowlist //system/apex.
This bp2builds the one cc_binary in that package that uses
library_linking_strategy_cc_defaults. That target is buildable, but
there's still a bp2build_available allowlist bug that causes the select
statements and static_libs deps to not show up for the target (only
dynamic_deps are), due to how bp2build_available is special cased for
soong_config_module_type in the loadhook.

Test: b build //system/apex:library_linking_strategy_sample_binary
Change-Id: Ic38a1bfaa476108db63348d030e196d8561300ca
2021-11-23 08:37:30 +00:00
Jingwen Chen
84817de033 Add support for merging defaults soong_config_module_types into bp2build
select statements.

This supports defaults from the same or different namespaces, and
transitively defaults as well.

Test: soong unit tests
Test: CI
Change-Id: I99435bacfcfbfe20ad753b8021a1779531d7595a
2021-11-23 08:37:30 +00:00
Liz Kammer
c31929962b Quote aquery commands.
Bazel run shell actions begin `bin/bash -c <command>`, without escaping,
the command can be treated as an argument to `bin/bash`.

Test: build/bazel/mixed_droid.sh
Change-Id: I423cb393da2e6ac97448ec77b2596f12670dfd31
2021-11-16 17:31:05 -05:00
Wei Li
455ba83a94 Fix the output handling of Bazel aquery for py_binary targets so soong's python_binary_host modules can be converted to py_binary and built with Bazel dependent targets in mixed build.
Test: USE_BAZEL_ANALYSIS=1 m genfunctosyscallnrs
Test: USE_BAZEL_ANALYSIS=1 m func_to_syscall_nrs
Test: USE_BAZEL_ANALYSIS=1 m libseccomp_policy
Bug: 201094425
Bug: 197135289
Bug: 204949078
Change-Id: If98630830d687ca814d6ae09c1e610f2c3a371b1
2021-11-16 20:49:28 +00:00
Treehugger Robot
59a5cb072b Merge "Refactor ProductConfigProperties to use a struct key instead of an string key with hardcoded patterns." 2021-11-16 04:55:15 +00:00
Jingwen Chen
25825ca08d Refactor ProductConfigProperties to use a struct key instead of an
string key with hardcoded patterns.

This fixes a bug with label list conditions_default attrs where the
attribute values get clobbered in a map with the keys
"conditions_default" (with a default empty list) and
"acme__feature__conditions_default" (with a non-empty list) when
generating the LabelListAttribute.

Test: CI
Change-Id: I5429e40f747b7a0ed559f8a468a4831cd32df2c0
2021-11-16 02:50:22 +00:00
Rupert Shuttleworth
c0d9f8bd5e Merge "Allow libbuildversion to build with Bazel, but skip its 'tests' subdir for now." 2021-11-15 09:00:12 +00:00
Chris Parsons
393002a913 Remove modules from denylist after host C++ support
Test: mixed_droid
Change-Id: If9b04625b0ad4fce52283eb683f1342e24bb97f2
2021-11-11 13:56:56 -05:00
Chris Parsons
494eef3dc8 Support host platform cc builds
This involves handling of linux-* platforms (identifying them
appropriately as host), disabling unsupported Windows platform, and
denylisting of still-problematic modules.

Test: mixed_droid, with verification that shared library artifacts
originate from bazel-out

Change-Id: Ib52db49a2d2a3c1ff9b76af23fd4f22cfc9182d0
2021-11-10 21:57:15 +00:00
Rupert Shuttleworth
2ac9c48ee6 Allow libbuildversion to build with Bazel, but skip its 'tests' subdir for now.
Bug: 198241918

Test: build/bazel/scripts/run_presubmits.sh

Change-Id: I9ab423bf8131de855cf0c380cd42e9fd662c7bfb
2021-11-10 11:16:05 -05:00
Jingwen Chen
a47f28d28e bp2build: add support for soong_config_module_type.
Test: CI, go unit test
Bug: 198556411
Change-Id: Idf862904d51d822f92af0c072341c31b7a02fc64
2021-11-08 13:38:28 +00:00
Treehugger Robot
a3bd963988 Merge changes Id11f4fb1,Id573d970,Ia94a0b5c
* changes:
  Fix //conditions:default excludes computation for LabelListAttribute.
  bp2build: split Bazel conversion context into smaller ones, and change TopDownMutatorContext signatures to use Bazel conversion context.
  Add an error check in `bazelPackage` for malformed labels.
2021-11-03 15:36:28 +00:00
Liz Kammer
5506c1658c Merge "Update bp2build denylist cause" 2021-11-03 14:19:16 +00:00
Wei Li
c989eaf4c3 Keep the following targets disabled in mixed build since there are errors in mixed_droid CI after go/oag/1861755 is submitted.
func_to_syscall_nrs
libseccomp_policy_app_zygote_sources
libseccomp_policy_app_sources
libseccomp_policy_system_sources

Test: CI
Bug: 204949078

Change-Id: Iaf9d7cd8831da984bcee82399be138ebe3aab9e8
2021-11-03 04:15:36 +00:00
Liz Kammer
b84b8c9c9a Update bp2build denylist cause
The binary is converted; however, there are still issues with the host
toolchain.

Test: build/bazel/ci/bp2build.sh
Change-Id: Idf864ea6c64c0c7bbbaa0c9d43d6eac5120e0268
2021-11-02 15:03:12 -04:00
Wei Li
664a4fd0af Fix python_binary_host module in mixed build.
Test: USE_BAZEL_ANALYSIS=1 m genfunctosyscallnrs
Test: USE_BAZEL_ANALYSIS=1 m func_to_syscall_nrs
Test: USE_BAZEL_ANALYSIS=1 m libseccomp_policy
Bug: 201094425, 197135289

Change-Id: Ibc8b99a92149410c8a879b7a4facf6c8961a7b9f
2021-11-02 17:04:38 +00:00
Jingwen Chen
55bc820d66 bp2build: split Bazel conversion context into smaller ones,
and change TopDownMutatorContext signatures to use Bazel conversion context.

This minimizes the context interfaces/functions actually needed to
convert a module, and makes such interfaces easier to mock/test.

Test: CI
Change-Id: Id573d97023d59e06ef70e1f54437024d3f7aadbd
2021-11-02 06:51:27 +00:00
Liz Kammer
2b8004b1ef bp2build: Add support for cc_binary.
Bug: 197920036
Test: build/bazel/ci/bp2build.sh
Change-Id: I2c4200967653af15a330ab8cbaf796b70d43f32d
2021-10-19 14:12:39 -04:00
Jingwen Chen
df27b7a1e0 Denylist genrules that create the same file in the same package.
Test: bp2build presubmits

Bug: 194642721
Bug: 203369847
Change-Id: Id0faff5fcf4136fac5236980dd60c5d8e54d9547
2021-10-18 06:33:16 +00:00
Liz Kammer
2fc348986c Add missing directories to bp2build.
Added directories with deps of adbd that were not converted and added
them to be converted via bp2build.

Test: bp2build.sh
Test: mixed_droid.sh
Change-Id: Iacb634b23e1fdc2767a0923882b71f482ebca53d
2021-10-13 17:07:41 -04:00
Chris Parsons
b97a817fdb Remove more out-of-date items from denylist
Test: mixed_droid CI
Change-Id: Ifc5bb10328860a5d979eb627a5347d8c23203155
2021-10-04 12:48:29 -04:00
Chris Parsons
c39f63358b Remove out-of-date items from bp2build denylist
Test: mixed_droid CI
Change-Id: Ic51cf7ed193641882dbe0018818d8e3bcab53507
2021-10-01 18:01:11 -04:00
Chris Parsons
2dde0cb3de bp2build: Handle target.linux
Also refactor target.bionic to be handled not as its own configuration
axis, but instead to be grouped into os selects handling.

This allows us to remove libbase and its dependencies from the bp2build
denylist.

Test: mixed_droid.sh CI
Change-Id: I92f30074d286306207653fe37589835ae3db16c4
2021-10-01 14:56:39 -04:00
Chris Parsons
b1f405e66d Temporarily disable libbase
This should unbreak the build and give me some time to investigate the
root cause.

Test: USE_BAZEL_ANALYSIS=1 m libandroidfw
Change-Id: Ia843ae5b92a70131b4989b4bdebc063866cb1075
2021-09-30 17:25:32 -04:00
Chris Parsons
a37e195182 Support building libcrypto via mixed builds
This required the following fixes to bp2build:
  - Correctly handle Bionic_* and Linux_* targets
  - Correctly handle cc_object's selects
  - Generate linker_script, stl, and system_dynamic_deps for cc_object in bp2build

Test: USE_BAZEL_ANALYSIS=1 m adbd
Change-Id: I753fd18df8ae551fb69df07e4174527c5388f289
2021-09-30 12:30:36 -04:00
Liz Kammer
0f3b7d2fc4 Add deps of protos to bp2build.
Update BazelOutPath to implement genPathProvider. This allows Bazel
outputs (incl filegroup) to be used as an input to generating a file
(e.g. proto).

Test: bp2build.sh
Test: mixed_build.sh
Bug: 200601772
Change-Id: I5ad67ade193025652100b214996b26ce9ca9bf83
2021-09-29 10:31:27 -04:00
Liz Kammer
55682fea4b Sort bp2build directory allowlist.
Test: bp2build.sh
Change-Id: I48157fd4e02fef3a6917f0a40418cc929168ffff
2021-09-29 10:30:35 -04:00
Jingwen Chen
b59f6e2397 bp2build //system/core/libasyncio:libasyncio
Test: presubmits
Change-Id: Ie748d2017eaf9043392975ca0db8ae4f673ba059
2021-09-28 00:48:14 +00:00
Jingwen Chen
8385250f4f bp2build //external/lz4/lib:liblz4.
Fixes: 198241792
Test: presubmits
Test: USE_BAZEL_ANALYSIS=1 m adbd
Change-Id: Ifc0326805f39259f06348dcc0b1b4a7c86afc856
2021-09-28 00:05:20 +00:00
Romain Jobredeaux
3bafa69336 Merge "Exclude WallpaperPicker app and pom2bp-generated BUILD files from bp2build." 2021-09-27 20:54:51 +00:00
Chris Parsons
a967f253e6 Add stl property support for bp2build
Test: mixed_libc
Test: USE_BAZEL_ANALYSIS=1 m libbase
Change-Id: If56d4a728de3ae4022cc9dadadc9bf8b952f9eaf
2021-09-24 13:51:56 -04:00
Liz Kammer
7a210ac233 bp2build: Add support for export_.*headers props
Soong supports export_.*_headers properties, the libraries contained in
this list must also be within a shared/static/whole_static/header libs
property. For bp2build, we eliminate this duplication. The libraries
not listed in an export_.*_headers property will migrate to an attribute
prepended with implementation_, those in export_.*_headers will not have
a prefix.

Test: build/bazel/ci/bp2build.sh
Test: build/bazel/ci/mixed_libc.sh
Bug: 198241472
Change-Id: I3eb84c983ec5d241c8a568e411dfd5619d3184a7
2021-09-23 16:07:45 -04:00
Chris Parsons
1826621bdd Disable mixed builds for non-device OS
We don't currently support propagation of OS in mixed builds, and our
host toolchains are not properly configured.

This allows us to continue to make progress on device OS until host is
supported.

Test: mixed_libc CI
Test: USE_BAZEL_ANALYSI=1 m adbd
Change-Id: I2bf8b3f9ca7928dd33bf2fb3d4fa067da0f593f3
2021-09-23 14:11:42 -04:00
Wei Li
bcd399479a Add bp2build converter for cc_genrule.
Test: cc_genrule_conversion_test.go

Change-Id: I19290b417d6336020a15ba7fa772ee0c76c58225
2021-09-23 05:41:20 +00:00
Romain Jobredeaux
a2c081413a Exclude WallpaperPicker app and pom2bp-generated BUILD files from bp2build.
Bug: 198418951
Test: b build packages/apps/WallpaperPicker
Change-Id: I86eb64c734eb262a497595f525027534cdabdf32
2021-09-22 14:43:43 -04:00
Romain Jobredeaux
f05bea080b Merge "Exclude directories with manual BUILD files from bp2build." 2021-09-22 17:28:31 +00:00
Chris Parsons
953b35623a bp2build: reference static variants directly
Previously, when referencing a cc_library as a static_lib from a
converted module, the corresponding target would depend on the "root
target" of that cc_library. This resulted in a superfluous dependency on
the shared target of the cc_library dependency. This superfluous
dependency could sometimes result in cycles.

This change ensures the static target name is directly specified in such
cases.

Test: mixed_libc.sh CI
Test: USE_BAZEL_ANALYSIS=1 m fmtlib
Change-Id: I36ee9f0a9017b1e9d73df9e3174669107f0afd4f
2021-09-20 15:15:29 -04:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
ac5097fcf4 Create bp2build converter for cc_shared_library
Factor out shareable code with cc_static_library

Remove `Linkstatic` field for shared cases,
because it's redundant with the rule

Bug: 198241907
Test: cc_library_shared_conversion_test.go
Test: mixed_{libc,droid}.sh
Change-Id: I5be3b66f812893bce41edb425fbd143ab22d9051
2021-09-16 15:00:56 +00:00
Liz Kammer
83d07a4c41 Merge "Expand handling of unconverted deps in bp2build" 2021-09-15 13:49:48 +00:00
Liz Kammer
37997c404b Add system/sepolicy/apex to bp2build allowlist
Technically, tzdata apex depends on
//system/sepolicy/apex:com.android.tzdata-file_contexts. Builds suceeded
regardless due to bp2build generating a filegroup for all files in a
directory.

This also requires an update to the check in apex/builder.go to address
the fact that outputs (including filegroups) from Bazel in mixed builds
will be in a Bazel out subdirectory. This change also corrects a
potential bug that would not support all subdirectories under
system/sepolicy as golang's path.Match does not treat ** as recursive.

Test: go test apex tests
Test: build/bazel/ci/mixed_droid.sh
Change-Id: I1076d60cf271e4cdd1ea990156ab0a39fcfa6535
2021-09-14 18:00:22 -04:00
Liz Kammer
6eff323206 Expand handling of unconverted deps in bp2build
Support three options for converting modules with unconverted
dependencies
1. (default) Warn when converting a module if it has unconverted deps.
2. Error when encountering a module with unconverted deps. (not hooked
up yet)

Test: build/bazel/ci/bp2build.sh
Test: build/bazel/ci/mixed_libc.sh
Test: BP2BUILD_ERROR_UNCONVERTED=1 build/bazel/ci/bp2build.sh with
      unconverted deps -- get appropriate error
Bug: 181155349
Change-Id: Ifaabf0cd2e43e963366dc137159c705294165c3d
2021-09-14 19:55:55 +00:00
Jingwen Chen
1e347860d5 bp2build: allowlist //external/libcap/...
This builds cap_names.list.h, which uses an eponymous filegroup
"generate_cap_names_list.awk" in Soong, but uses the file target
directly in Bazel.

This also improve filegroup support for mixed builds, by issuing a
cquery call _without_ arch. Filegroups in Soong don't have configurable
properties, so don't generate Bazel filegroups into buildroot's
config_nodes (which was x86_64 by default).

The mixed_build_root now looks like this:

```
config_node(...)
config_node(...)
config_node(...)
config_node(...)

...

filegroup(name = "common",
    srcs = ["@//bionic/linker:linker_sources_x86",
            "@//bionic/libc:kernel_input_headers",
            "@//system/timezone/apex:com.android.tzdata-androidManifest",
            "@//external/libcap:generate_cap_names_list.awk",
            "@//bionic/linker:linker_sources_arm64",
            "@//bionic/linker:linker_sources",
            "@//bionic/libc:libc_sources_shared_arm",
            "@//bionic/linker:linker_sources_x86_64",
            "@//bionic/libc:all_kernel_uapi_headers",
            "@//build/bazel/examples/apex/minimal:build.bazel.examples.apex.minimal-file_contexts",
            "@//system/core/libcutils:android_filesystem_config_header",
            "@//bionic/libc:libc_sources_static",
            "@//bionic/linker:linker_sources_arm",
            "@//bionic/libc/tools:bionic-gensyscalls",
            "@//bionic/tools:bionic-generate-version-script",
            "@//bionic/libc:libc_sources_shared"],
)

mixed_build_root(name = "buildroot",
    deps = [":x86",
            ":arm64",
            ":arm",
            ":common",
            ":x86_64"],
)
```

Test: CI
Fixes: 198595323
Fixes: 198235838
Change-Id: I6df9a14da556cf358d96e6a99b514f66a2638295
2021-09-09 07:54:16 +00:00
Romain Jobredeaux
9e09bba237 Exclude directories with manual BUILD files from bp2build.
Test: b build packages/apps/QuickSearchBox
Change-Id: I2534a328e28a6fff1385bfccdc3b461198bc1f88
2021-09-08 18:09:02 +00:00
Jingwen Chen
294e774e28 bp2build: allowlist more packages.
* //system/core/libprocessgroup:libprocessgroup_headers
* //external/brotli/... and build libbrotli.
* //external/libcxx/...

This also adds //system/core/libprocessgroup to the denylist, but
non-recursively, because there are other modules in its subpackages that
don't build yet, so CI's bp2build-incremental's `//system/...` build can keep
working.

Fixes: 186826841
Fixes: 198148733
Fixes: 197954422

Test: ran mixed_droid-clean locally
Test: presubmits
Change-Id: I5f21287f5374811c4bfcf7b7d64430fe4f9b2900
2021-09-02 10:49:58 +00:00
Liz Kammer
f4afb53d4a Revert "bp2build allowlist //external/libcxx/..."
Revert submission 1814446

Reason for revert: causes failure of mixed_droid
Reverted Changes:
If5285f161:bp2build allowlist //external/libcxx/...
I4e7787374:bp2build: build //system/core/libprocessgroup:libp...
Ibec6d5fbe:bp2build: allowlist //external/brotli/... and buil...

Change-Id: I37327057a3df44cbb6e57d7f54120b3448da6057
2021-08-31 13:18:45 +00:00
Liz Kammer
1fdd6fafd5 Revert "bp2build: build //system/core/libprocessgroup:libprocess..."
Revert submission 1814446

Reason for revert: causes failure of mixed_droid
Reverted Changes:
If5285f161:bp2build allowlist //external/libcxx/...
I4e7787374:bp2build: build //system/core/libprocessgroup:libp...
Ibec6d5fbe:bp2build: allowlist //external/brotli/... and buil...

Change-Id: Idcacf7b8e48e1cd156717d6b7a6d1f9534207d49
2021-08-31 13:18:45 +00:00
Liz Kammer
a73a503d90 Revert "bp2build: allowlist //external/brotli/... and build libb..."
Revert submission 1814446

Reason for revert: causes failure of mixed_droid
Reverted Changes:
If5285f161:bp2build allowlist //external/libcxx/...
I4e7787374:bp2build: build //system/core/libprocessgroup:libp...
Ibec6d5fbe:bp2build: allowlist //external/brotli/... and buil...

Change-Id: I0486afb20cbd0d994c4821969ce0fdb3e35a9e43
2021-08-31 13:18:45 +00:00
Jingwen Chen
60da00eb83 bp2build: allowlist //external/brotli/... and build libbrotli.
Test: bp2build ci
Fixes: 198148733
Change-Id: Ibec6d5fbe73bdec649b6dd5e8e667b261ae88ba5
2021-08-31 10:11:02 +00:00
Jingwen Chen
60eaabbe75 bp2build: build //system/core/libprocessgroup:libprocessgroup_headers
This adds //system/core/libprocessgroup to the denylist, but
non-recursively, because there are other modules in its subpackages that
don't build yet, so CI's bp2build-incremental's `//system/...` build can keep
working.

Fixes: 186826841
Test: bp2build-incremental //system/...
Change-Id: I4e778737414364964de6328d7420512da224809c
2021-08-31 10:11:02 +00:00
Jingwen Chen
d2fec261fc bp2build allowlist //external/libcxx/...
Test: b build //external/libcxx/...
Fixes: 197954422
Change-Id: If5285f161a9e15f3401b442d699464f522124d3d
2021-08-31 10:10:01 +00:00