Commit Graph

71 Commits

Author SHA1 Message Date
Spandan Das
6a448ec1a3 Add a function to create config_setting(s)
The use case for this is creating config_setting(s) specific to an apex
variant and selecting stub/impl in that config_setting. We likely need
only a handful of such config_setting(s), but determining that list
requires iterating the build graph.

Test: go test ./bp2build
Change-Id: I9aa552e3d0bcf67513023c3a7d4bbf8fae464ee4
2023-04-28 19:43:43 +00:00
MarkDacek
9c094ca153 Fix issue where bazel-force-enabled-modules aren't actually analyzed.
Bug: 273910287
Test: run mixed_mode_test.sh

Change-Id: I309c423b6e7f505ad1db0d9327e7de6b8298dfe9
2023-04-14 02:17:54 +00:00
Spandan Das
abedff0ca7 Add a method in bp2build to create aliases in another directory
The expected use case for this is to create aliases for stub libraries
in the @api_surfaces repository in build/bazel/api_surfaces.

This restricts the scope to just aliases. If we have a use case for
generating actual Bazel targets in another directory, a workaround could
be to generate the targets in the current directory (via
CreateBazelTargetModule) and aliases to it in the other directory

Test: go test ./bp2build
Change-Id: I6b63d9d018618d447fc7c260a2a94aaa00e57a4d
2023-03-08 21:40:26 +00:00
Cole Faust
18994c73f1 Replace SortedStringKeys with SortedKeys
Now that we have generics.

Bug: 193460475
Test: presubmits
Change-Id: I1594fd8feb505175d5c09c03ef397e5ffd5b09cb
2023-02-28 16:51:32 -08:00
Cole Faust
b85d1a15cc Bp2build support for multiple product configs
Create a
build/bazel/product_config/generated/products/<product_name>/BUILD
file that contains the platform definitions needed for
a particular product. Currently we just create it for the
current lunch target, but the idea is that eventually when
all product config is in starlark, all the products will
have their platform definitions in the tree at once.

Bug: 249685973
Test: Presubmits
Change-Id: I08c82ff28dcf62f09d3b1d2e3186a6b961e12f6e
2023-01-05 12:49:14 -08:00
Paul Duffin
c639059fd8 Make CodegenContext reference *android.Context
Previously, it stored an android.Context (no pointer) which while it
worked (because the current contents are themselves pointers) it is
inconsistent with how the rest of the code references it and could
cause weird behavior if additional fields were added to the Context.

Test: m nothing
      m bp2build
Change-Id: I9c70f925dde85e5b15431cc232b0c2eb6371ec6d
2022-11-05 17:43:12 +00:00
usta
4f5d2c1e97 include symlink metrics in bp2build_metrics.pb
Bug: b/256212479
Test: Prior to thi CL bp2build.symlink_forest event was missing in bp2build_metrics.pb after a clean mixed build
Change-Id: I53bfc4114a383c0d1f9c4c7945e7d4c69bc50b0c
2022-10-31 11:38:10 -04:00
Spandan Das
5af0bd3e48 Create a new mode in soong_ui to generate API only BUILD files
The generated Bazel workspace will only contain api specific targets.
This is feasible since these targets do not have any cross dependencies
with the targets in the bp2build workspace

The advantages of a new mode are
1. Does not pollute bp2build workspace with api targets
2. Does not block api targets with the current allowlist conversion
   mechansims in bp2build
(In the future we might want to combine these two workspaces)

A Soong module type will generate a Bazel target if it implements
ApiProvider interface

Test: m apigen
Test: m nothing

Change-Id: I69c57ca6539f932e0ad554ce84a87fb7936fdba0
2022-10-04 17:32:56 +00:00
Usta Shrestha
c6057154b8 cosmetic: switch instead of if-else
Bug: N/A
Test: m bp2build
Change-Id: Idf37af869f814a0a75957808f3f33200ff9800c0
2022-09-24 04:29:53 +00:00
Kevin Dagostino
60f562ac7e adding /out/soong_injection/metrics/converted_modules_path_map.json
This file can be used by atest/scripts/tools to determine which modules
have been converted and what their bazel build path is.

Test: b build //packages/modules/adb/...
will generate the file at /out/soong_injection/metrics/converted_modules_path_map.json
Also added a unit test to check that the file is created.

{
  "adbd": "//packages/modules/adb",
  "adbd_test": "//packages/modules/adb",
  "add_ext4_encrypt": "//external/e2fsprogs/contrib",
  ....
}

Change-Id: Ibd424c487840d84c8a4fb635828a73541220b36b
2022-09-20 04:01:13 +00:00
Sasha Smundak
8bea267caf Implement bp2build for the package module
* We are interested mostly in the conversion the `default_applicable_licenses`
  attribute, as `default_visibility` cannot be handled until every module's
  `visibility` is handled.
* Several referenced license modules had to be manually enabled for
  conversion, and likewise a few trivial Android.bp containing only
  package and license modules.
* As Bazel allows only a single `package` rule, the package rules in a
  manually converted BUILD.bazel files were removed (in
  external/protobuf and prebuilts/clang/host/linux-x86 trees).
* The converted package rule is emitted before the `load` statements per
  Bazel documentation.

Bug: 190817312
Test: treehugger
Change-Id: If8bf6fee1580177de3bb402923615bcd48923ed2
2022-09-14 13:49:27 -07:00
Sasha Smundak
fb58949414 Handle nameless modules during bp2build conversion.
So far it is `package` module.

Test: treehugger
Change-Id: Ibf3af0b0bf0a532e2ea1d478275a5ac3784ef170
2022-09-14 09:32:17 -07:00
Alix
94e2603ab7 refactoring build_conversion.go to remove panic
Returning errors throughout instead of calling panic()
Errors will be more useful for testing

Test: bp2build tests
Change-Id: I3b03f0a30e7a80878e91c7f0e2df5a94d9d6b780
2022-09-09 20:25:51 +00:00
Cole Faust
ea602c5a41 Remove old BUILD file merging code
It's not needed anymore since aosp/2197837,
it should've been removed in that cl but I forgot.

Bug: 234167862
Test: ./build/bazel/ci/bp2build.sh
Change-Id: I3d67a6e1894ad401525070ad37d3158708898306
2022-08-31 15:12:47 -07:00
Cole Faust
324a92e404 Always merge build files
Previous behavior:

- Packge not listed in bp2buildKeepExistingBuildFile:
    - Use bp2build generated build file
- Package listed in bp2buildKeepExistingBuildFile:
    - Use handcrafted build file even if there were allowlisted bp2build
      modules in the same package.
- Package listed in bp2buildKeepExistingBuildFile and a soong module has
  a bp2build: { label } attribute:
    - Merge the handcrafted and bp2build generated build files

New behavior:

- Packge not listed in bp2buildKeepExistingBuildFile:
    - Use bp2build generated build file
- Package listed in bp2buildKeepExistingBuildFile:
    - Merge with bp2build generated build file.

Bug: 234167862
Test: ./build/bazel/ci/bp2build.sh
Change-Id: Ifbaf4f8f0f5158b5b2bd6d534eb2311e2e5f399b
2022-08-30 16:07:23 -07:00
Sasha Smundak
f2bb26f492 Show module's type and path when reporting dependency problems
Test: treehugger
Change-Id: I25c3eddbeeb10bd6b9bfa9e869f6fbc98d2a6ec4
2022-08-04 14:49:25 -07:00
Usta Shrestha
db46a9bbb8 Cosmetic Changes plus log errors
Test: N/A
Bug: N/A
Change-Id: I39e622a93270c922bdbea27ca79632a606431a2c
2022-08-02 20:24:57 +00:00
Sam Delmerico
932c01cf9e export Java variables to Bazel
Test: build/bazel/bp2build.sh
Change-Id: Ia06f9265c9f96e6add6edbd0cee925fe37b430d3
2022-04-08 14:15:43 +00:00
Liz Kammer
72beb34609 Add package for printing starlark formatted data
Bug: 216168792
Test: build/bazel/ci/bp2build.sh
Change-Id: I3a06b19396f7ffe1c638042cda7e731dd840f1d6
2022-02-08 17:32:28 -05:00
Chris Parsons
492bd91a51 Add bp2build metrics per module type
For example, new metrics at HEAD highlight that 99 of 1379 total
cc_library_static modules are converted.

Test: m bp2build, printproto
Change-Id: I6cc4227124e9a130b75911f3e40e6585d731d00a
2022-01-21 11:52:24 -05:00
Liz Kammer
daa09efb5e Start unblocking com.android.runtime deps
Adds support to warn rather than error about missing deps in order to
suport modules in build/soong with deps outside of build/sooong in
bp2build integration tests.

Test: build/bazel/ci/bp2build.sh
Change-Id: I1282bccd37a3fc9f33555d34e68d7f0873d8272c
2022-01-06 10:40:43 -05:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
947fdbfdee Log bp2build_metrics .pb
Also share `Save(pb proto.Message, filepath string)`

Bug: 201539536
Test: bp2build_metrics.pb has expected content & path
Test: m nothing
Test: {bp2build,mixed_{libc,droid}}.sh
Test: CI
Change-Id: I7d8ad87fca6a4b0355010090a527f5ae67b27c88
2021-12-13 13:03:44 -05:00
Liz Kammer
46fb7aba4d Support empty strings in bp2build
Previously, could not set an empty string as a value of an attribute;
however, this is necessary in some cases. To not unnecessarily create an
empty string, use string pointers for attributes rather than strings.

Test: go test bp2build tests
Change-Id: I03b3a3567452d455246d22d81f86c317d06b7c39
2021-12-01 10:14:49 -05:00
Jingwen Chen
58ff6801f4 Fix bp2build select generation for inter-attribute soong config
variable usage.

There's bug a in the current soong_config_variable handling
implementation where a soong_config_variable sets conditions_default
value for an attr, and a non-conditions_default value for another attr.
This results in the former attr not properly setting the zero value for
the non-conditions_default select key, resulting in the pretty printer
omitting the attribute totally.

The current implementation in this CL ensures that the zero value is set
whenever this happens at the module level. This is seen in
library_linking_strategy_cc_defaults (see comments in code, and the new
tests)

Test: CI
Bug: 198556411

Change-Id: Ibaeb94508c51a7429fb7a08df610cbb5470f76d2
2021-11-23 08:37:30 +00:00
Liz Kammer
0eae52e0de bp2build; Update handling of linker flags
Test: build/bazel/ci/bp2build.sh
Bug: 197920036
Change-Id: I6e3100574fa0e40bcd8cf0e6af0efd3310aa41bf
2021-10-07 16:07:40 -04:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
447f6c99c9 Bp2Build common properties auto-handling
Introduce `commonAttributes` & `fillCommonBp2BuildModuleAttrs used in
CreateBazelTargetModule

Adapt `bp2BuildInfo` to use `commonAttrs` instead of `Name`.
And thus also all downstream users of `CreateBazelTargetModule`.

As initial user, the Soong `required` property will be
translated to Bazel's `data`.

Bug: 198146582, 196091467
Test: build_converstion_test.go:TestCommonBp2BuildModuleAttrs
Test: go test
Test: mixed_{libc,droid}.sh
Change-Id: Ib500e40f7e2cb48c459f1ebe3188962fc41ec124
2021-10-04 14:43:04 +00:00
Liz Kammer
7a210ac233 bp2build: Add support for export_.*headers props
Soong supports export_.*_headers properties, the libraries contained in
this list must also be within a shared/static/whole_static/header libs
property. For bp2build, we eliminate this duplication. The libraries
not listed in an export_.*_headers property will migrate to an attribute
prepended with implementation_, those in export_.*_headers will not have
a prefix.

Test: build/bazel/ci/bp2build.sh
Test: build/bazel/ci/mixed_libc.sh
Bug: 198241472
Change-Id: I3eb84c983ec5d241c8a568e411dfd5619d3184a7
2021-09-23 16:07:45 -04:00
Jingwen Chen
310bc8f5f4 bp2build: Refactor metrics collection to use exported functions.
This CL splits the metric count into handcrafted, generated, and
unconverted modules.

Test: CI
Test: m bp2build, the metrics reported are identical.
Change-Id: I80e3d2fd2989222bd40b6433b4a1924b4943d0ba
2021-09-20 11:54:00 +00:00
Jingwen Chen
afb84bd78f Fix bp2build-progress generating an empty converted_modules list, and
duplicate module names.

We were using the value receiver on AddConvertedModule before, which
didn't actually mutate the internal slice state.

Test: Run bp2build-progress.py
Change-Id: Ic60f8f6214a91dc129abf62c57dd0b85efffb283
2021-09-20 10:31:46 +00:00
Jingwen Chen
6117450ead bp2build: add converted modules to codegen metrics, and remove the
compat layer.

The list of converted modules is used for tracking bp2build progress.

The module_name_to_label aliases weren't really adding much value to
mixed builds, so let's drop the feature for now and simplify the code.

Test: CI
Test: USE_BAZEL_ANALYSIS=1 m droid
Change-Id: I3269344cfe6d1c72c108d66c6e820bc0ec19bb34
2021-09-17 08:47:16 +00:00
Jingwen Chen
16d90a8954 Remove __bp2build__ prefix trimming.
This is no longer needed after r.android.com/1792714

Test: CI
Change-Id: I808c3124cf1a4f3ebe5d25e41f346a818c3cc57c
2021-09-17 07:16:59 +00:00
Liz Kammer
83d07a4c41 Merge "Expand handling of unconverted deps in bp2build" 2021-09-15 13:49:48 +00:00
Liz Kammer
6eff323206 Expand handling of unconverted deps in bp2build
Support three options for converting modules with unconverted
dependencies
1. (default) Warn when converting a module if it has unconverted deps.
2. Error when encountering a module with unconverted deps. (not hooked
up yet)

Test: build/bazel/ci/bp2build.sh
Test: build/bazel/ci/mixed_libc.sh
Test: BP2BUILD_ERROR_UNCONVERTED=1 build/bazel/ci/bp2build.sh with
      unconverted deps -- get appropriate error
Bug: 181155349
Change-Id: Ifaabf0cd2e43e963366dc137159c705294165c3d
2021-09-14 19:55:55 +00:00
Liz Kammer
32a0339340 Bp2build: handle embedded structs as blueprint
For structs that are embedded, Blueprint does not nest under the
embedded name, flattening them into the original struct for blueprint
files (e.g.
9fd2ed93df:proptools/unpack_test.go;l=402-431;drc=3adb2409648d6f8b25354ac47f083dae87731f10).
We should do the same for bp2build.

This will also allow us to embed structs for bp2build conversion
allowing more reuse.

Test: go test bp2build tests
Change-Id: I9ce088462adaf59bffa80bea76cd488e31f98e9d
2021-09-14 14:41:36 -04:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
0da7ce6f39 Document testing & build_conversion purposes
Formatter also relocated some imports

Test: Existing tests pass
Change-Id: I48b1cb65400e463a6b4e91fd9fe76e13d8d81cd6
2021-08-23 22:04:53 +00:00
Liz Kammer
2ada09a546 Don't create a new module for bp2build conversion.
A performance improvement for bp2build as Blueprint/Soong no longer have
the overhead of additional modules. The creation of these modules
results in:
* traversal of additional modules for each subsequent mutator
* synchronization over a go channel to collect newly created modules:
https://cs.android.com/android/platform/superproject/+/master:build/blueprint/context.go;l=2594,2600;drc=1602226f23181b8c3fbfcaf3358f0297e839d7d3

We avoid both of these by storing the information directly in the
underlying module.

Also as a fringe benefit, removes some necessary boilerplate for
conversion.

For benchmarks, reduces runtime ~1% for 1% converted, ~24% for 100%
converted. See more: go/benchmarks-for-https:-r.android.com-1792714

Test: ran benchmarks/tests in bp2build
Test: build/bazel/ci/bp2build.sh
Change-Id: Ie9273b8cbab5bc6edac1728067ce184382feb211
2021-08-17 15:57:09 -04:00
Jingwen Chen
c63677b3c9 Direct Bazel builds from m.
This CL adds support to bp2build/Soong to dump a BUILD file under
out/soong/soong_injection/targets containing alias targets to their real
targets for every converted Soong module, regardless of whether they are
handcrafted or generated.

Test: TH
Change-Id: Ic1816fda5d019c395301618134fac68b3057d752
2021-06-21 06:12:34 +00:00
Jingwen Chen
3d383bbeb9 bp2build: support strip properties.
This CL adds support to bp2build for generating all strip-related
properties into a strip dictionary as a parameter in the cc_library
macro.

With the dictionary, it's easy to organize related attributes and
directly expand into a stripped_shared_library's attributes.

Test: //build/bazel/tests/bionic:compare_libc_stripping
Test: TH
Fixes: b/187928597
Change-Id: Ifea68d48fe295e71a43b12876cb168c475a62187
2021-06-10 05:22:50 +00:00
Liz Kammer
d366c909ca Handle no_libcrt in bp2build.
Test: ci/bp2build.sh
Bug: 187928307
Change-Id: Ib80c4318169652b322e5d878c8784679e42f87dd
2021-06-08 17:18:22 -04:00
Liz Kammer
2b07ec7b3b Support empty srcs for configurable attrs
Test: mixed build linkerconfig
Change-Id: I1ee56f7fa46ce2971ecae6d2ca9258631a32f46a
2021-05-26 16:58:54 -04:00
Rupert Shuttleworth
c194ffbcf3 Make GetTargetProperties() aware of more complex targets, like 'android_arm', instead of just 'android'.
Test: Added new unit test and updated existing tests.

Test: bazel build //bionic/... //external/... //frameworks/... //system/...

Test: ./build/bazel/scripts/run_presubmits.sh

Change-Id: I250d1964f5cf42b92ddb929379d35d8c844423f7
2021-05-25 19:40:16 -04:00
Jingwen Chen
4910976314 Make BUILD file merging slightly smarter.
This change enables checked-in BUILD files like
prebuilts/clang/host/linux-x86/BUILD.bazel to be merged cleanly with the
bp2build generated one into the synthetic workspace.

The checked-in BUILD file contains a package() declaration that bp2build
also generates. To avoid double declaration, the BUILD file writer now
checks if the BazelTargets contain handcrafted targets. If so, it
delegates the package declaration to the handcrafted BUILD file instead.

This change also sorts the bp2build targets before the handcrafted ones,
and adds a section header to demarcate the two sets of targets.

Test: TH
Change-Id: I3ecdeaab3226b895b623daf0791d24a657f7a7c6
2021-05-25 07:01:55 +00:00
Lukacs T. Berki
1353e59690 Handle the version_script property.
Doesn't work when depends on arch/target/etc., but good enough for
libdl_android.

Bug: 186650430
Test: Presubmits.
Change-Id: Ib0facb41a89454717c74663e5e078aedd33d1b9c
2021-04-30 16:46:41 +02:00
Rupert Shuttleworth
2a4fc3ecdc Generate BUILD files for every directory that has an Android.bp file.
Test: Added an integration test
Test: bazel build --package_path=out/soong/workspace //bionic/...

Change-Id: Ie34bd23ab3c5428e6c9c9919e5fb6fcb4e709adc
2021-04-21 07:15:50 -04:00
Jingwen Chen
b4628eb03f bp2build: codegen singleton lists on one line.
This CL refactors the bp2build code generator to pretty print lists with
a single element on one line, instead of taking up three lines, which
can make BUILD files unnecessarily long. A single line singleton list is
also more commonly used in BUILD files.

Test: TH
Change-Id: Ic9e44741bbb070c8f45925466b9ccdd0608498b2
2021-04-09 09:42:57 +00:00
Jingwen Chen
6393098ecf bp2build: cc_library converter for //bionic/libdl:libdl_android
This CL contains the converter for libdl_android, a cc_library that
expands into a cc_shared_library and a regular cc_library.

Test: TH
Test: bp2build; bazel test //build/bazel/tests/...

Change-Id: If70641a538211b0d6b2aac0e4d0d06912318304d
2021-04-09 09:42:53 +00:00
Jingwen Chen
c1c2650532 Add target/os configurable string_list attrs.
Starting with copts for cc_object, with an extracted function that can
be shared with other cc_* module types.

Test: TH
Change-Id: I9025232e83a3dcd0ca243387486fafbdbd3e2d9b
2021-04-05 11:27:19 +00:00
Jingwen Chen
91220d7334 Add os/target configurable selects for label list attributes.
This CL is pretty large, so I recommend starting with reading the newly
added tests for the expected behavior.

This change works in conjunction with the linked CLs in the Gerrit topic.
Those CLs add support for new platform() definitions for OS targets
specified in Soong's arch.go, which are configurable through
Android.bp's `target {}` property. It works similary to previous CLs
adding support for the `arch {}` property.

These configurable props are keyed by the OS: android, linux_bionic,
windows, and so on. They map to `select` statements in label list
attributes, which this CL enables for cc_library_headers' header_libs
and export_header_lib_headers props.

This enables //bionic/libc:libc_headers to be generated correctly, from:

    cc_library_headers {
        name: "libc_headers",
        target: {
            android: {
                header_libs: ["libc_headers_arch"],
                export_header_lib_headers: ["libc_headers_arch"],
            },
            linux_bionic: {
                header_libs: ["libc_headers_arch"],
                export_header_lib_headers: ["libc_headers_arch"],
            },
        },
        // omitted props
    }

to:

    cc_library_headers(
        name = "libc_headers",
        deps = [] + select({
            "//build/bazel/platforms/os:android": [
                ":libc_headers_arch",
            ],
            "//build/bazel/platforms/os:linux_bionic": [
                ":libc_headers_arch",
            ],
            "//conditions:default": [],
        }),
    )

Test: TH
Test: Verify generated //bionic/libc:libc_headers
Fixes: 183597786

Change-Id: I01016cc2cc9a71449f02300d747f01decebf3f6e
2021-04-02 08:17:34 +00:00
Jingwen Chen
a5dcb03450 Merge "bp2build/queryview: codegen control sequences literally." 2021-03-30 21:57:09 +00:00
Jingwen Chen
0702791a99 bp2build: arch-configurable selects for label list attrs.
This CL adds the configurable LabelListAttribute support to bp2build.

Test: go test
Change-Id: I2ef9e385d9cf1b1845988128eca1d8cda1ecb5e8
2021-03-30 13:22:48 +00:00