Adds support to warn rather than error about missing deps in order to
suport modules in build/soong with deps outside of build/sooong in
bp2build integration tests.
Test: build/bazel/ci/bp2build.sh
Change-Id: I1282bccd37a3fc9f33555d34e68d7f0873d8272c
Each conversion required defining a separate mutator, which will each
operate on _all_ modules and requires each to repeat checks whether the
mutator should operator. Instead, we introduce a single mutator and
modules can define a ConvertWithBp2build to implement bp2build
conversion for that module.
Test: bp2build.sh
Bug: 183079158
Change-Id: I99d4b51f441c2903879092c5b56313d606d4338d
Previously, could not set an empty string as a value of an attribute;
however, this is necessary in some cases. To not unnecessarily create an
empty string, use string pointers for attributes rather than strings.
Test: go test bp2build tests
Change-Id: I03b3a3567452d455246d22d81f86c317d06b7c39
Moves to specifying attributes as a map, such at it is possible to add
additional attributes conditionally. This is in particular useful once
supporting the `enabled` property which will add
`target_compatible_with`
Test: go test soong tests
Change-Id: Iade8eed1ce3acb1d1712a9ee3119d9ae59675624
Soong supports additional arch-variant targets based on arch/os aside
from strictly arch/os names. Handle the unconverted cases based solely
on arch/os.
Test: build/bazel/ci/bp2build.sh
Change-Id: I8dc987b1aa4a4ea646dd5b03a94b84297ba6113a
Introduce `commonAttributes` & `fillCommonBp2BuildModuleAttrs used in
CreateBazelTargetModule
Adapt `bp2BuildInfo` to use `commonAttrs` instead of `Name`.
And thus also all downstream users of `CreateBazelTargetModule`.
As initial user, the Soong `required` property will be
translated to Bazel's `data`.
Bug: 198146582, 196091467
Test: build_converstion_test.go:TestCommonBp2BuildModuleAttrs
Test: go test
Test: mixed_{libc,droid}.sh
Change-Id: Ib500e40f7e2cb48c459f1ebe3188962fc41ec124
Support three options for converting modules with unconverted
dependencies
1. (default) Warn when converting a module if it has unconverted deps.
2. Error when encountering a module with unconverted deps. (not hooked
up yet)
Test: build/bazel/ci/bp2build.sh
Test: build/bazel/ci/mixed_libc.sh
Test: BP2BUILD_ERROR_UNCONVERTED=1 build/bazel/ci/bp2build.sh with
unconverted deps -- get appropriate error
Bug: 181155349
Change-Id: Ifaabf0cd2e43e963366dc137159c705294165c3d
For structs that are embedded, Blueprint does not nest under the
embedded name, flattening them into the original struct for blueprint
files (e.g.
9fd2ed93df:proptools/unpack_test.go;l=402-431;drc=3adb2409648d6f8b25354ac47f083dae87731f10).
We should do the same for bp2build.
This will also allow us to embed structs for bp2build conversion
allowing more reuse.
Test: go test bp2build tests
Change-Id: I9ce088462adaf59bffa80bea76cd488e31f98e9d
with the same name.
Also add capability to test for errors raised in bp2build mutators /
contexts.
This CL does two things to filegroups:
1) If the filegroup has only 1 source file with the same name as itself,
don't generate a filegroup target. Instead, dependents will depend
directly on the Bazel file target instead.
2) If the filegroup has more than 1 source file and 1 of them has the
same name as itself, the bp2build mutator will error out. If bp2build
on CI passes, it means that the source tree / product we're testing
against does not have such a case (which seems to be true for most
source trees).
Either way, this will allow us to be unblocked for most of the errant
filegroups (case 1) in the tree.
Test: CI
Test: New test cases in filegroup_conversion_test.go
Fixes: 194762573
Change-Id: I830c53efc8808569afe3c5f9f08436855bcdafed
A performance improvement for bp2build as Blueprint/Soong no longer have
the overhead of additional modules. The creation of these modules
results in:
* traversal of additional modules for each subsequent mutator
* synchronization over a go channel to collect newly created modules:
https://cs.android.com/android/platform/superproject/+/master:build/blueprint/context.go;l=2594,2600;drc=1602226f23181b8c3fbfcaf3358f0297e839d7d3
We avoid both of these by storing the information directly in the
underlying module.
Also as a fringe benefit, removes some necessary boilerplate for
conversion.
For benchmarks, reduces runtime ~1% for 1% converted, ~24% for 100%
converted. See more: go/benchmarks-for-https:-r.android.com-1792714
Test: ran benchmarks/tests in bp2build
Test: build/bazel/ci/bp2build.sh
Change-Id: Ie9273b8cbab5bc6edac1728067ce184382feb211
Refactor bp2build to retrieve modules directly by name, instead of via
DirectDeps. This functions properly as bp2build has no need for variant
information of the blueprint graph.
Test: USE_BAZEL_ANALYSIS=1 m fmtlib
Change-Id: Ief4b67bc56f24929871af772f3a742f07085bf8c
This CL adds support to bp2build/Soong to dump a BUILD file under
out/soong/soong_injection/targets containing alias targets to their real
targets for every converted Soong module, regardless of whether they are
handcrafted or generated.
Test: TH
Change-Id: Ic1816fda5d019c395301618134fac68b3057d752
This is to simplify the process of resolving label + exclude labels
across the various configuration axes we have and across the various
properties/modules that use this behavior.
Test: ci/bp2build.sh && ci/mixed_droid.sh
Change-Id: I8efae3e75ddb365384f5caaf5bb504a5206618d3
This change enables checked-in BUILD files like
prebuilts/clang/host/linux-x86/BUILD.bazel to be merged cleanly with the
bp2build generated one into the synthetic workspace.
The checked-in BUILD file contains a package() declaration that bp2build
also generates. To avoid double declaration, the BUILD file writer now
checks if the BazelTargets contain handcrafted targets. If so, it
delegates the package declaration to the handcrafted BUILD file instead.
This change also sorts the bp2build targets before the handcrafted ones,
and adds a section header to demarcate the two sets of targets.
Test: TH
Change-Id: I3ecdeaab3226b895b623daf0791d24a657f7a7c6
In a non-top level Android.bp file, exclude_srcs was not working at all
due to a bug in expandSrcsForBazel. GlobFiles was expanding a glob
relative to root, but the expandedExcludes list was relative to the
module dir, causing the glob function to not consider the
expandedExcludes list at all.
Add tests to demonstrate that this is working now.
Test: TH
Bug: 186388919
Change-Id: Ice8254231d085b39126e91b823a09ec328ee0ae0
This CL refactors the bp2build code generator to pretty print lists with
a single element on one line, instead of taking up three lines, which
can make BUILD files unnecessarily long. A single line singleton list is
also more commonly used in BUILD files.
Test: TH
Change-Id: Ic9e44741bbb070c8f45925466b9ccdd0608498b2
See b/184026959 for an instance where a genrule's cmd contains \n, and
the codegenerator generates an applied newline. It should be the
original newline control sequence literal.
Test: TH, bp2build unit test
Fixes: 184026959
Change-Id: Idb99ea87c75dfa4cb3ae679f98d8f61cb5a5ab09
This CL adds the support for specifying lists of directories in
build/soong/android/bazel.go, which are then written into
out/soong/bp2build/MANIFEST. Using this configuration,
modules/directories can either default to bp2build_available: true or
false, while still retaining the ability to opt-in or out at the module level.
It also ensures that ConvertWithBp2Build returns true iff the module
type has a registered bp2build converter.
Test: go tests
Test: demo.sh full
Test: TreeHugger presubmits for bp2build and mixed builds.
Change-Id: I0e0f6f4b1b2ec045f2f1c338f7084defc5d23a55
If both bp2build_available and label are specified, label will be
preferred.
Initially, we copy the entire BUILD.bazel file. Eventually we may move
this to use bazel query for a more accurate result.
Test: go test *
Test: build/bazel/scripts/milestone-2/demo.sh full
Test: GENERATE_BAZEL_FILES=true m nothing
edit bionic/libc/tools/BUILD.bazel
GENERATE_BAZEL_FILES=true m nothing and verify changes picked up
Bug: 180516554
Change-Id: I43025583300e6b10d2c18032cd4a76237b578d59
Sample output:
[bp2build] cc_library_headers: 5 targets
[bp2build] cc_object: 5 targets
[bp2build] filegroup: 4 targets
[bp2build] genrule: 4 targets
[bp2build] sh_binary: 1 targets
[bp2build] Generated 19 total BUILD targets from 39270 Android.bp modules.
This CL adds an additional CodegenMetrics return value to
GenerateBazelTargets calls, which are called from bp2build, queryview,
and their tests. For this UI, we only want to use it for bp2build, and
not queryview or tests, since it's not useful for the former, and can
pollute the CLI for the latter.
Test: build/bazel/scripts/milestone-2/demo.sh
Change-Id: Ic84307a1ed1a25e360c9b23459e5449d932bc2e7
This CL adds a per-target allowlist to instruct bp2build on which modules it should generate Bazel targets for.
Test: soong tests
Change-Id: I869e66fce405c2c6689b381569b8cc0118cbcf76
This currently expands all globs, still need to support converting glob
syntax.
Test: go build_conversion_test
Test: GENERATE_BAZEL_FILES=true m nothing
Test: m nothing
Bug: 165114590
Change-Id: If7b26e8e663d17566fad9614ca87a8da1f095284
This CL brings the defaults mutators into bp2build, reusing the
implementation of flattening defaults from *_defaults module deps into
the modules themselves.
This is an approach to converge defaults semantics with Bazel, which
doesn't have existing equivalent abstractions, other than macros, which
require more infrastructure to be built.
Test: TH
Test: build_conversion_test.go
Change-Id: Ia1443f8e6c3eb277fa861bb96625425a22cfc6d7
This CL adds support to bp2build for declaring the location of the
Starlark rule definition when creating BazelTargetModules. This is
needed for non-native rules that needs to be loaded from .bzl files
somewhere in the tree.
Since load statements are aggregated at the top of the BUILD file, away
from the targets that actually use them, this CL also introduces an
abstraction to group BazelTargets together and compute their load
statements and target string representations separately, allowing load
statements to be decoupled and written into a BUILD file before the
targets themselves.
Test: soong tests
Test: TH
Test: GENERATE_BAZEL_FILES=true m nothing && build/bazel/scripts/bp2build-sync.sh write && bazel cquery //bionic/...
Fixes: 178531760
Test: TH
Change-Id: Ie5f793a00006eb024eaef07ddd9fde7aaefc054e
The previous implementation relied on the implicit registration of Bp2Build mutators, resulting in test non-hermeticity. Refactor bp2build tests to explicitly specify the bp2build mutators under test.
Test: Soong tests
Test: TH
Change-Id: I9b9674bad1ea533b3bd31b07077a9e02c99b4c1d
The CodegenMode enum helps to differentiate between bp2build and queryview modes of generating BUILD files from the Soong module graph.
bp2build is used for generating BUILD files from an unconfigured Soong module graph for Soong->Bazel conversion, whereas QueryView is a front-end for querying the existing Soong configured module graph with Bazel query.
Test: go test
Test: TH
Change-Id: I5709f591f1ae08b2770adf272d8ce60c9ee1f46a
There's a module type in internal main that uses the module_deps
property, which clashes with QueryView's module_deps attribute. A fix is
to prefix the attribute as soong_module_deps to avoid the conflict.
While we're at this, rename the other internal attribute names to be
consistent as well.
Test: soong tests
Test: m queryview && bazel query //... --config=queryview
Fixes: 178351366
Change-Id: I84961c8f58c0c0fac1a1c2cbc6bc625499ac895e
Results of `cquery //bionic/...`: https://paste.googleplex.com/5608885982330880?raw
Note that this CL does not contain the global module name mapping to
fully qualified label lookup mechanism, so while the BUILD targets are
queryable, they are not buildable (yet).
Test: GENERATE_BAZEL_FILES=true m nothing && ./build/bazel/scripts/bp2build-sync.sh write && bazel cquery //bionic/...
Change-Id: I6f2316222723d8b612d9b041ca82a188cd05bac9
This CL creates the framework necessary for generating
BazelTargetModules from regular Soong Android modules.
BazelTargetModules are code-generated into Bazel targets in BUILD files.
See the follow-up CL for examples of creating filegroup/genrule
BazelTargetModules.
Test: GENERATE_BAZEL_FILES=true m nothing # creates out/soong/bp2build
with no BUILD files, because there are no BazelTargetModules in the
module graph.
Change-Id: I33a96365bd439043b13af6db9e439592e9983188
Implement bp2build codegen as a discrete step that runs after an
alternatively registered pipeline of mutators, instead of a
presingleton.
bp2build codegen requires a Context that supports VisitAllModules and
PathContext, so this CL also makes a BpToBuildWrapperContext that
conforms to PathContext by adding two method implementations.
Test: GENERATE_BAZEL_FILES=true m nothing && bazel query //... --config=bp2build | wc -l # 31433
Test: m queryview && bazel query //... --config=queryview # 63638
Change-Id: I0dd359746584b228046d2d0ff00895f28f9bdfc3
Splits queryview into queryview and bp2build. The latter runs as a
presingleton (could be converted to a singleton). This prevents needing
to run soong_ui a subsequent time to build the query. Queryview remains
as a separate step to prevent increasing runtime due to this generation
every time Soong runs.
Currently this is running as a presingleton as this gives a translation
of Android.bp files after only LoadHooks have run, no mutators.
Test: go tests
Test: m queryview && bazel query --config=queryview //...
Change-Id: If2ba28c7ef60fbd41f43bda6623d41c8c7d23a1b
