Commit Graph

218 Commits

Author SHA1 Message Date
Christopher Parsons
ed73b76513 Merge "Split asm and c flags and srcs in bp2build output" 2021-05-25 14:58:32 +00:00
Liz Kammer
e4982e8828 Mark test helpers as helpers.
This enables identifying the correct location of test failures.

from godocs:
Helper marks the calling function as a test helper function.
When printing file and line information, that function will be skipped.
Helper may be called simultaneously from multiple goroutines.

Test: go test bp2build tests
Change-Id: I52430e541494c78f5222c11510a4d273fe205a06
2021-05-25 14:41:43 +00:00
Jingwen Chen
962a8f0253 Merge changes I3ecdeaab,I43720641
* changes:
  Make BUILD file merging slightly smarter.
  bp2build/b: exit early in GENERATE_BAZEL_FILES=1.
2021-05-25 08:56:21 +00:00
Jingwen Chen
4910976314 Make BUILD file merging slightly smarter.
This change enables checked-in BUILD files like
prebuilts/clang/host/linux-x86/BUILD.bazel to be merged cleanly with the
bp2build generated one into the synthetic workspace.

The checked-in BUILD file contains a package() declaration that bp2build
also generates. To avoid double declaration, the BUILD file writer now
checks if the BazelTargets contain handcrafted targets. If so, it
delegates the package declaration to the handcrafted BUILD file instead.

This change also sorts the bp2build targets before the handcrafted ones,
and adds a section header to demarcate the two sets of targets.

Test: TH
Change-Id: I3ecdeaab3226b895b623daf0791d24a657f7a7c6
2021-05-25 07:01:55 +00:00
Chris Parsons
af24cdd99f Split asm and c flags and srcs in bp2build output
This allows removal of almost all current items from the mixed build
denylist, which were previously broken due to being unable to separately
control flags for compilations of different languages within the same
target.

Note that this does not appropriately implement asm/c srcs and flags for
either the shared variant or the static variant. This will require a
followup.

Test: bp2build.sh and mixed_libc.sh CI scripts
Test: Updated b2build tests

Change-Id: I28cf7437ee96cdf2fdbcb1eda2303691cff08ba4
2021-05-24 21:49:14 -04:00
Liz Kammer
e3e4a5f2d8 Handle arch/os-specific product variables
Bug: 183595873
Test: go test bp2build tests
Change-Id: I36e93ae1eb2943555dd304d5bdf62d995e77b437
2021-05-24 14:32:38 -04:00
Liz Kammer
6fd7b3fee9 Handle product config vars in bp2build.
Test: bp2build ci  & mixed build libc
Bug: 183595873
Change-Id: I2d87434ff4df5a24efc5e3e38f087de035228934
2021-05-24 14:22:31 -04:00
Lukacs T. Berki
c1cc3b9678 Split each test case to a different test function.
This is so that they can be individually debugged.

Test: Presubmits.
Change-Id: I7d929c4126bba7470aaa1c0def85bad65429ffdc
2021-05-23 20:41:14 +02:00
Chris Parsons
d6358775c8 Propagate unexported deps via implementation_deps
Test: bp2build and mixed_libc CI
Test: Manually verified that libc_bionic_ndk compilation gets the
appropriate headers (and no extra headers) from downstream

Change-Id: I79eb6e8ec1d415bd50d12105da4cf97101f95474
2021-05-20 12:05:24 -04:00
Liz Kammer
50d601731b Merge "Dump bazel product config in Soong" 2021-05-20 14:09:18 +00:00
Liz Kammer
09f947d67e Dump bazel product config in Soong
Bug: 187862880
Test: build/bazel/ci/bp2build.sh
Test: build/soong/tests/bp2build_bazel_test.sh
Change-Id: I24b09baad973e25bec4476e1ea4a7692b72b7d20
2021-05-19 17:38:56 -04:00
Rupert Shuttleworth
06559d07cb Move some implicitly shared testing code for bp2build into testing.go.
Test: go test
Change-Id: I403d703184e517dbffbdccb8007229cad00858c1
2021-05-19 09:14:26 -04:00
Rupert Shuttleworth
413a7a97fb Have bp2build generate BUILD.bazel instead of generating BUILD.
Test: TH
Change-Id: I465d29da96dd77c432890a38a56203e291b8ceed
2021-05-18 09:01:36 -04:00
Chris Parsons
484e50aa7c bp2build: support generated sources and hdrs
There are two pieces to make this work:
1. Local include paths must include $(BINDIR)-relative paths, to support
generated headers in those directories.
2. The srcs that bp2build outputs for BUILD targets must include labels
for targets given in generated_hdrs and generated_srcs.

Support for exported_generated_hdrs intentionally deferred.

This allows us to remove several targets from the bp2build denylist.
Some are moved to the mixed build denylist, because genreated headers are
still unsupported in mixed builds.
Test: bp2build.sh CI script

Change-Id: Ib4f9dac20f6445487b8dad53b91eac01f437a590
2021-05-17 14:30:07 -04:00
Rupert Shuttleworth
0096079bbc Make bazel build //external/... work by only allowing "approved" existing BUILD files in the Bazel workspace.
Test: bazel build //bionic/... //external/... //frameworks/... //system/...

Test: ./build/bazel/scripts/run_presubmits.sh

Change-Id: I91865ca87c6535053e7a14d2526ff3ce0991bfea
2021-05-17 06:34:53 -04:00
Jingwen Chen
ccfd7fc939 Merge "bp2build: build //external/scudo/..." 2021-05-13 12:19:43 +00:00
Jingwen Chen
1d35a87072 Merge "bp2build: export some cc toolchain flags into Starlark." 2021-05-13 10:31:54 +00:00
Jingwen Chen
75be1cae8f bp2build: build //external/scudo/...
Test: TH
Fixes: 187158841
Change-Id: I73c1d8fe075d2534c2389973b9381405d9389044
2021-05-13 06:25:49 +00:00
Jingwen Chen
bf61afb7f7 bp2build: export some cc toolchain flags into Starlark.
This CL exports common/global/device/host clang/ld/ldd flags
from their Ninja variable initialization locations in
cc/config/global.go and cc/config/clang.go to make Bazel's cc_toolchain
and Soong's cc actions more consistent with each other.

This does not handle env-dependent or arch-specific toolchain flags
yet (logic in compiler.go and linker.go).

Test: TH
Bug: 187086342
Bug: 187084737
Bug: 186628704
Bug: 187857770
Change-Id: Ie403d7cd23f35160897b9dd902c799cbf1bd7f0c
2021-05-13 06:25:47 +00:00
Lukács T. Berki
a412c74fc9 Merge "Implement configured version_script." 2021-05-12 18:14:56 +00:00
Jingwen Chen
3950cd6ed4 bp2build: remove libc_tzcode from denylist.
This now works with --features no_copts_tokenization. With tokenization,
the escaped empty string '\"\"' became an actual empty string '' on the
command line, setting -DWILDABBR to the wrong value.

However, no_copts_tokenization unveiled other problems with Android.bp
flags, such as spaces in existing cflags like
https://cs.android.com/android/platform/superproject/+/master:bionic/libc/Android.bp;l=288;drc=a0a4a6c2967b5b3c02c951ea1145f32ed5564ab9
- this trips up Bazel's copts when generated literally, so the
fix (other than splitting on space and making Soong accept strings that
don't start with dashes as cflags) is to make bp2build split cflags on
spaces before generating them as copts.

Test: bp2build, build bionic/...
Fixes: 186822591
Change-Id: Icf10bd20f6fb81db0b719ca0555fc70c75b91a79
2021-05-12 10:46:15 +00:00
Lukacs T. Berki
56bb083989 Implement configured version_script.
Bug: 186650430
Test: Presubmits.
Change-Id: I3e363ea9254a859dc8f485c9f273f6b3677f7645
2021-05-12 12:36:45 +02:00
Chris Parsons
0864831019 Handle cc_library shared/static only properties
Also handle whole_static_libs via a different attribute
(whole_archive_deps), separating these dependencies from regular static
deps.

Test: Build //bionic/libdl with bazel in conjunction with bzl
changes

Change-Id: I45019b6127a0d2731592ec35537ca15e77001cc2
2021-05-10 14:06:47 -04:00
Rupert Shuttleworth
143be94016 Add temporary workaround for pack_relocations linker flag conversion for bp2build.
Test: added unit test
Test: bazel build //bionic/...
Change-Id: I28217026f5846820ea2cc32cf8790948a54a9365
2021-05-10 05:17:42 -04:00
Rupert Shuttleworth
c50fa8dd05 Add support for shared_libs in cc_library targets.
Test: cd bp2build; go test

Test: bazel build //bionic/...

Test: ./build/bazel/scripts/run_presubmits.sh

Change-Id: I71e279470a0d69b243dd0a2b53ce31842fd36ee4
2021-05-10 03:35:32 -04:00
Lukacs T. Berki
598dd00236 Refactor how bp2build gets arch-specific props.
Then plumb them to LabelAttribute.

This refactoring is required because the previous implementation did not
handle properties in shards other than the first one (e.g.
version_script) well. In addition, it also makes the code paths between
bp2build and analysis more similar.

Bug: 186650430
Test: Presubmits.
Change-Id: Ic4393e8ae47f4e88816bf45c89399efd61494d22
2021-05-05 09:00:01 +02:00
Jingwen Chen
2b54eb8375 bp2build: remove manifest file for bp2build-sync.
The latter doesn't exist anymore, so there's no more need for the
bp2build file manifest.

Bug: 185817076
Test: TH
Change-Id: I6638b84eb371ced98474e3ca5026e3b33b824767
2021-05-03 06:49:54 +00:00
Chris Parsons
c424b76f76 Support multilib properties in bp2build
This combines properties among "multilib" and "arch" stanzas in selects
generated by bp2build.

With this fix, libc_gdtoa may be removed from the denylist.

This change also refactors a portion of arch.go, specifically bp2build's
arch mutator, adding a number of comments along the way, to hopefully
make this code clearer for future readers.

Test: mixed_libc.sh
Change-Id: If2beea672957cfb1af6760406ba507181ec38f77
2021-04-30 13:21:36 -04:00
Lukacs T. Berki
1353e59690 Handle the version_script property.
Doesn't work when depends on arch/target/etc., but good enough for
libdl_android.

Bug: 186650430
Test: Presubmits.
Change-Id: Ib0facb41a89454717c74663e5e078aedd33d1b9c
2021-04-30 16:46:41 +02:00
Jingwen Chen
53681ef590 bp2build: make libdl build.
This CL adds support for the static and shared props, forwarding
whole_static_libs and srcs to the underlying shared library's
static_deps and static library's srcs respectively.

Test: TH
Fixes: 186609667
Change-Id: I84f838361c7758b256b2a163af894c502de03109
2021-04-29 08:59:02 +00:00
Liz Kammer
2b50ce669c Make GetDirectDep implementation match comment
The comment on the interface of GetDirectDep states if there are
multiple deps with different tags, the first will be returned; however,
the current behavior is to panic if there are multiple deps. The
behavior now:
* a single dep, return the module and tag
* a single module with different tags: return module with first tag
* multiple modules: panic
* no module: return nil, nil

Bug: 186488405
Test: ~/aosp/build/bazel/ci/bp2build.sh
Test: m nothing
Change-Id: Id1e7315e7874b4a683ad7357ed2793822315821f
2021-04-27 15:53:51 -04:00
Treehugger Robot
6de44e74c8 Merge "bp2build: fix exclude_srcs in subpackages." 2021-04-27 13:48:46 +00:00
Jingwen Chen
e32e9e07d7 bp2build: arch-specific exclude_srcs and srcs.
OS-specific exclude_srcs and srcs will be done in a follow-up, due to
complexities from merging multiple select statements together.

Test: TH
Bug: 186153868
Change-Id: I01b881e9a5a7cd41b4a507f8be7e9e65eab37bdc
2021-04-27 11:02:07 +00:00
Jingwen Chen
4ecc67d3c5 bp2build: fix exclude_srcs in subpackages.
In a non-top level Android.bp file, exclude_srcs was not working at all
due to a bug in expandSrcsForBazel. GlobFiles was expanding a glob
relative to root, but the expandedExcludes list was relative to the
module dir, causing the glob function to not consider the
expandedExcludes list at all.

Add tests to demonstrate that this is working now.

Test: TH
Bug: 186388919
Change-Id: Ice8254231d085b39126e91b823a09ec328ee0ae0
2021-04-27 09:51:08 +00:00
Jingwen Chen
882bcc1c1c bp2build: remove header globs in generated srcs.
Not needed anymore for bp2build-incremental since https://android-review.googlesource.com/q/topic:no-include-check.

Not needed for mixed builds either, since cc compile actions aren't sandboxed.

Fixes: 186488830
Test: treehugger and go tests

Change-Id: Ib5d4908dcce6bf910a653c457bb251d726e717d4
2021-04-27 06:26:40 +00:00
Rupert Shuttleworth
375451edfe Add support for per-target exported includes.
Test: Added unit test
Change-Id: I8ab58ddc73c15d5ec0f515e43407542bd4e8081a
2021-04-26 07:49:08 -04:00
Liz Kammer
dbe1ef2149 Merge "Add bp2build arch-specific paths mutator" 2021-04-23 12:50:05 +00:00
Liz Kammer
78c68655e1 Write all BUILD files for queryview
Some BUILD files cannot be written to support bp2build as they already
exist in the tree; however, all BUILD files should be written for
queryview as they are not intended to coexist with handwritten BUILD
files.

Test: $ bazel query --config=queryview
  "somepath(//system/linkerconfig:linkerconfig--android_arm64_armv8-a ,
  //bionic/libc:libc_openbsd--android_arm64_armv8-a_static)"
Test: build/bazel/ci/bp2build.sh
Fixes: 186132826
Change-Id: I2cb26b7482e7eb5523cee33452be3753ad37e1b0
2021-04-22 14:40:17 -04:00
Jingwen Chen
ed9c17d033 bp2build: refactor/standardize cc_* bp2build converters
This CL refactors the cc* bp2build converters to use the common
attribute extractors in cc/bp2build.go.

This also adds include_build_directory to be handled by the compiler
attr extractor to generate recursive headers as inputs.

This also turns include_dirs and local_include_dirs into the
execroot-relative -I flags.

e.g. if a module in  bionic/libc has "private" in local_include_dirs,
the "-Ibionic/libc/private" copt is generated for it.

Fixes: 185139955

Test: TH
Test: Forrest for mixed_clean-droid
Change-Id: Ib67056482227e62068fbbea0455035bdf5d56319
2021-04-22 08:31:24 +00:00
Liz Kammer
4562a3b218 Add bp2build arch-specific paths mutator
Adds deps for properties tagged `android:"path"` within arch, multilib,
and target properties.

Test: build/bazel/ci/bp2build.sh
Test: m nothing
Bug: 185217298
Change-Id: I0230da399d2c4e984b837f69523fa09eadba3ff1
2021-04-21 18:21:50 -04:00
Rupert Shuttleworth
2a4fc3ecdc Generate BUILD files for every directory that has an Android.bp file.
Test: Added an integration test
Test: bazel build --package_path=out/soong/workspace //bionic/...

Change-Id: Ie34bd23ab3c5428e6c9c9919e5fb6fcb4e709adc
2021-04-21 07:15:50 -04:00
Lukacs T. Berki
b21166e236 Make symlink_forest.go prefer generated files.
Now, if the same file exists in the generated tree and the source tree,
it symlinks in the generated file instead of failing outright.

Drive-by fix: print errors for all conflicts instead of bailing out on
the first one.

Test: Presubmits (including the two new tests)
Change-Id: Ifb5b3fc89b5454d231966bfa4e61c22cd69834f3
2021-04-21 12:24:27 +02:00
Lukacs T. Berki
3f9416ea80 Rename cryptic symbols in symlink_forest.go .
Test: Presubmits.
Change-Id: Ib41e5ba001924d6fca72410bc0895747d6e55d37
2021-04-20 13:08:11 +02:00
Lukacs T. Berki
b353cca496 Create a synthetic Bazel workspace.
It's under out/soong/workspace and is a symlink forest that merges BUILD
files generated by bp2build and the source tree.

bazel-* symlinks and $OUTDIR are currently ignored.

Test: Presubmits.
Change-Id: If14df23bd0f6a6f2d7d1140f95213f2aee66d652
2021-04-20 10:00:02 +02:00
Rupert Shuttleworth
c143cc50dd Transform paths to headers in include dirs to take package boundaries into account.
This allows the following cc_library_static targets to build with bp2build:

- libc_freebsd
- libc_freebsd_large_stack
- libc_openbsd_ndk
- libc_gdtoa  (* but still fails for mixed builds)
- libc_aeabi
- libc_static_dispatch
- libc_dynamic_dispatch

This also allows a number of other cc_library_static targets to progress further in their builds.

Test: Added unit test
Test: bp2build-sync.py write; bazel build //bionic/...

Change-Id: I71742565c16594448a41a6428a5c993171ec4cb4
2021-04-14 17:05:36 -04:00
Jingwen Chen
b4628eb03f bp2build: codegen singleton lists on one line.
This CL refactors the bp2build code generator to pretty print lists with
a single element on one line, instead of taking up three lines, which
can make BUILD files unnecessarily long. A single line singleton list is
also more commonly used in BUILD files.

Test: TH
Change-Id: Ic9e44741bbb070c8f45925466b9ccdd0608498b2
2021-04-09 09:42:57 +00:00
Jingwen Chen
6393098ecf bp2build: cc_library converter for //bionic/libdl:libdl_android
This CL contains the converter for libdl_android, a cc_library that
expands into a cc_shared_library and a regular cc_library.

Test: TH
Test: bp2build; bazel test //build/bazel/tests/...

Change-Id: If70641a538211b0d6b2aac0e4d0d06912318304d
2021-04-09 09:42:53 +00:00
Rupert Shuttleworth
c58d3d214e Be more explicit about headers in cc_library_static targets.
Test: bp2build-sync.py write; bazel build //bionic/...
Change-Id: I253b55f4d3cbe76805691b32e761016950871601
2021-04-08 09:43:04 +00:00
Rupert Shuttleworth
64a90286c4 Merge "Support arch variations for export_system_include_dirs in cc_library_headers bp2build converter." 2021-04-08 00:54:47 +00:00
Rupert Shuttleworth
b815168474 Support arch variations for export_system_include_dirs in cc_library_headers bp2build converter.
Test: Added unit test
Test: bp2build-sync.py write; bazel build //bionic/... works for more cc_library_static targets (in a parent CL)
Change-Id: Ib487216a4bcbc52958ff948722dae347b0d8b606
2021-04-07 16:05:44 +00:00