Commit Graph

108 Commits

Author SHA1 Message Date
Jingwen Chen
0cc7196bc7 bp2build: remove libc_nomalloc from denylist.
Test: TH

Fixes: b/186825031
Change-Id: I596d6ea611fffb582e9161643148baf9a2f3be6a
2021-05-20 01:34:42 +00:00
Christopher Parsons
f949ec20c0 Merge "Support middleman actions in mixed builds" 2021-05-18 22:31:19 +00:00
Chris Parsons
c4fb133380 Support middleman actions in mixed builds
This allows support of generated hdrs / srcs in mixed builds.

Test: Manually verified that libc_bionic_ndk passes compilation (failed
previously due to missing generated heaer)
Test: bp2build and mixed_libc CI scripts
Test: New aquery test

Change-Id: I88e359a4bd9eba383c207d5cf812272725ff0a3d
2021-05-18 14:07:44 -04:00
Jingwen Chen
535429299a Remove libjemalloc5 and libc_jemalloc_wrapper from denylist.
See b/188503688 for more info on why libjemalloc5 is in the
bp2buildCcLibraryStaticOnlyList.

Bug: 188503688

Fixes: 187012490
Fixes: 186828626

Test: TH
Change-Id: I2b70cf99b2efa81f3f35ab0b61716eb0f6f9ebd6
2021-05-18 10:43:26 +00:00
Chris Parsons
484e50aa7c bp2build: support generated sources and hdrs
There are two pieces to make this work:
1. Local include paths must include $(BINDIR)-relative paths, to support
generated headers in those directories.
2. The srcs that bp2build outputs for BUILD targets must include labels
for targets given in generated_hdrs and generated_srcs.

Support for exported_generated_hdrs intentionally deferred.

This allows us to remove several targets from the bp2build denylist.
Some are moved to the mixed build denylist, because genreated headers are
still unsupported in mixed builds.
Test: bp2build.sh CI script

Change-Id: Ib4f9dac20f6445487b8dad53b91eac01f437a590
2021-05-17 14:30:07 -04:00
Rupert Shuttleworth
0096079bbc Make bazel build //external/... work by only allowing "approved" existing BUILD files in the Bazel workspace.
Test: bazel build //bionic/... //external/... //frameworks/... //system/...

Test: ./build/bazel/scripts/run_presubmits.sh

Change-Id: I91865ca87c6535053e7a14d2526ff3ce0991bfea
2021-05-17 06:34:53 -04:00
Jingwen Chen
75be1cae8f bp2build: build //external/scudo/...
Test: TH
Fixes: 187158841
Change-Id: I73c1d8fe075d2534c2389973b9381405d9389044
2021-05-13 06:25:49 +00:00
Jingwen Chen
bf61afb7f7 bp2build: export some cc toolchain flags into Starlark.
This CL exports common/global/device/host clang/ld/ldd flags
from their Ninja variable initialization locations in
cc/config/global.go and cc/config/clang.go to make Bazel's cc_toolchain
and Soong's cc actions more consistent with each other.

This does not handle env-dependent or arch-specific toolchain flags
yet (logic in compiler.go and linker.go).

Test: TH
Bug: 187086342
Bug: 187084737
Bug: 186628704
Bug: 187857770
Change-Id: Ie403d7cd23f35160897b9dd902c799cbf1bd7f0c
2021-05-13 06:25:47 +00:00
Jingwen Chen
3950cd6ed4 bp2build: remove libc_tzcode from denylist.
This now works with --features no_copts_tokenization. With tokenization,
the escaped empty string '\"\"' became an actual empty string '' on the
command line, setting -DWILDABBR to the wrong value.

However, no_copts_tokenization unveiled other problems with Android.bp
flags, such as spaces in existing cflags like
https://cs.android.com/android/platform/superproject/+/master:bionic/libc/Android.bp;l=288;drc=a0a4a6c2967b5b3c02c951ea1145f32ed5564ab9
- this trips up Bazel's copts when generated literally, so the
fix (other than splitting on space and making Soong accept strings that
don't start with dashes as cflags) is to make bp2build split cflags on
spaces before generating them as copts.

Test: bp2build, build bionic/...
Fixes: 186822591
Change-Id: Icf10bd20f6fb81db0b719ca0555fc70c75b91a79
2021-05-12 10:46:15 +00:00
Treehugger Robot
ee4e7fe76f Merge "bp2build: build static version of libstdc++." 2021-05-04 08:46:37 +00:00
Jingwen Chen
179856a69d bp2build: build static version of libstdc++.
This CL adds the ability to filter cc_library modules to only generate
their static variant of their shared variant isn't ready.

For example, the libstdc++ library is buildable as a static library,
which libc depends on. However, the shared variant of libstdc++ depends
on libc, which has to come later.

This CL introduces that abstraction to break up bp2build conversion into
more atomic steps to help with conversion.

Test: TH (bazel build //bionic/... incl. libstdc++'s static variant)
Bug: 186489250
Bug: 186822597

Change-Id: I3e2fe748e4e3d3b656760da4807f342d67c8f45f
2021-05-04 02:06:41 +00:00
Rupert Shuttleworth
52e6672613 Add fmtlib_ndk back to the deny list because it fails in mixed builds postsubmit.
But add it to the mixed builds deny list instead of the bp2build deny list.

Test: Waiting for another postsubmit to run.
Change-Id: Ief51356758304b3f13c95df401e310d4f5e236f7
2021-05-03 10:05:55 -04:00
Rupert Shuttleworth
fb955387db Update info on some deny-list entries.
(Some info has been relocated to blockers in Buganizer)

Also remove libjemalloc5 and fmtlib_ndk from the deny list as they don't seem to need to be there at the moment.

Test: bazel build //bionic/...
Test: build/bazel/scripts/run_presubmits.sh

Change-Id: I87e9dd3d4a83291efbcc15d6b99187b57d45ad18
2021-05-03 04:47:03 -04:00
Chris Parsons
c424b76f76 Support multilib properties in bp2build
This combines properties among "multilib" and "arch" stanzas in selects
generated by bp2build.

With this fix, libc_gdtoa may be removed from the denylist.

This change also refactors a portion of arch.go, specifically bp2build's
arch mutator, adding a number of comments along the way, to hopefully
make this code clearer for future readers.

Test: mixed_libc.sh
Change-Id: If2beea672957cfb1af6760406ba507181ec38f77
2021-04-30 13:21:36 -04:00
Jingwen Chen
790324e69c bp2build: add bugs and update bp2build denylist.
This CL updates the error messages and reorganizes the bottom half of
the bp2build denylist.

Test: TH
Change-Id: Ib15d019ba9cd006a78474477af34c66b5e418348
2021-04-30 09:43:20 +00:00
Rupert Shuttleworth
47aa58420b Update reasons for modules being on the deny list (and add bug links).
Test: bp2build; bazel build //bionic/...
Test: build/bazel/scripts/run_presubmits.sh

Change-Id: I852146b9df57689c022cb4e6c5b9e4a764866822
2021-04-30 04:28:49 -04:00
Jingwen Chen
d01da05860 Group some modules together under a single bug for tracking purposes.
Test: TH
Change-Id: I3f75bbd5c1455ab42d589d884a705bf09ef0b825
2021-04-30 03:33:15 +00:00
Jingwen Chen
53681ef590 bp2build: make libdl build.
This CL adds support for the static and shared props, forwarding
whole_static_libs and srcs to the underlying shared library's
static_deps and static library's srcs respectively.

Test: TH
Fixes: 186609667
Change-Id: I84f838361c7758b256b2a163af894c502de03109
2021-04-29 08:59:02 +00:00
Jingwen Chen
282881683f bp2build: convert //external/arm-optimized-routines/...
Also update modules in denylist that depend on arm-optimized-routines libraries.

Bug: 186388919
Test: bazel build //external/arm-optimized-routines/...
Test: TH
Change-Id: If449b30288a4f852e5fcf31d923eda6cec1eb043
2021-04-28 22:11:44 +00:00
Treehugger Robot
12bcc130ea Merge "Update bp2build allow/denylisting for system" 2021-04-28 15:53:42 +00:00
Lukács T. Berki
331917a4c6 Merge "Remove libc_syscalls from the denylist." 2021-04-28 13:38:45 +00:00
Liz Kammer
a3b5dcf9f2 Update bp2build allow/denylisting for system
Test: build/bazel/ci/bp2build.sh
Test: build/bazel/ci/mixed_libc.sh
Change-Id: I21b3e32f7a9b9235ea87d68cff425077c8a46ff9
2021-04-28 09:01:50 -04:00
Liz Kammer
8e72286b68 Merge "Make GetDirectDep implementation match comment" 2021-04-28 12:34:40 +00:00
Lukacs T. Berki
667bb27363 Remove libc_syscalls from the denylist.
It looks like eakammer@'s aosp/1684105 fixed it.

Test: Presubmits.
Change-Id: I166344bbf82ec4e4273cf30b7bb369dcb8ec4442
2021-04-28 11:54:46 +02:00
Lukacs T. Berki
497f17d227 Allowlist libpropertyinfoparser.
Or at least as much of it as needed for Bionic.

Bug: 186388932
Test: Presubmits.
Change-Id: Ic984f50a4869244cbdb6946dd9cca4d9e08445b7
2021-04-28 08:21:37 +02:00
Rupert Shuttleworth
6036497a3c Remove libdl_static from the deny list in light of http://aosp/1688405
Unfortunately libdl has to stay on the deny list due to an unrelated clang error in CI.

Test: bazel build //bionic/... now works with libdl and libdl_static both removed from the deny list.

Test: ./build/bazel/ci/bp2build.sh passes locally

Change-Id: I1b3c3dd01bc535c73c2e88c81b7998fa7906b4c5
2021-04-27 20:25:21 -04:00
Liz Kammer
2b50ce669c Make GetDirectDep implementation match comment
The comment on the interface of GetDirectDep states if there are
multiple deps with different tags, the first will be returned; however,
the current behavior is to panic if there are multiple deps. The
behavior now:
* a single dep, return the module and tag
* a single module with different tags: return module with first tag
* multiple modules: panic
* no module: return nil, nil

Bug: 186488405
Test: ~/aosp/build/bazel/ci/bp2build.sh
Test: m nothing
Change-Id: Id1e7315e7874b4a683ad7357ed2793822315821f
2021-04-27 15:53:51 -04:00
Chris Parsons
f60ecf081e Mixed builds: propagate includes from cc_* targets
This allows us to remove libasync_safe from the mixed builds denylist.

Test: mixed_libc.sh CI script
Change-Id: Ibafd231284864078bf30340f919d39e5098843ce
2021-04-27 15:04:37 -04:00
Lukacs T. Berki
b5ac5af618 Reorganize and cull cc_library_static denylist.
Take two: this time, put libasync_safe on the mixed build denylist
because it seems to break things.

Test: Presubmits + testing the offending action using a Ninja
invocation. "m droid dist" also running.

Change-Id: I7912b6670980776d981d874dab157b0268693ab2
2021-04-27 15:00:45 +02:00
Jingwen Chen
e32e9e07d7 bp2build: arch-specific exclude_srcs and srcs.
OS-specific exclude_srcs and srcs will be done in a follow-up, due to
complexities from merging multiple select statements together.

Test: TH
Bug: 186153868
Change-Id: I01b881e9a5a7cd41b4a507f8be7e9e65eab37bdc
2021-04-27 11:02:07 +00:00
Liz Kammer
cc86ac88a8 Revert "Reorganize and cull cc_library_static denylist."
This reverts commit cba7b0c43c.

Reason for revert: mixed_droid failing: libbacktrace fails due to libasync_safe, linkerconfig (and others) due to duplicate symbols

Change-Id: Ib6dd6bbb6f76048a7abd0970d457cf75921beae6
2021-04-26 22:17:48 +00:00
Lukacs T. Berki
cba7b0c43c Reorganize and cull cc_library_static denylist.
Test: Presubmits.
Change-Id: Ia82d5b6975fe53888798db0378635ec36b43603e
2021-04-26 18:54:28 +02:00
Liz Kammer
dbe1ef2149 Merge "Add bp2build arch-specific paths mutator" 2021-04-23 12:50:05 +00:00
Lukacs T. Berki
f8645b93b4 Update error messages.
For libm and libseccomp_policy in particular.

Test: Presubmits.
Change-Id: I778a466b7736e25aa4486335976847e8c5cf645f
2021-04-23 09:07:28 +02:00
Rupert Shuttleworth
5baca302df Add libc_openbsd to mixed builds deny list.
Test: USE_BAZEL_ANALYSIS=1 m linkerconfig
Change-Id: I59386d7d70a9a3c0cabe1b641284dfbd131d532d
2021-04-22 13:41:37 -04:00
Rupert Shuttleworth
e5c719e124 Allow more cc_library_static targets to build.
- libc_openbsd_large_stack
- libc_openbsd

Also update the lib_dns comment as it makes more progress now (still fails).

Test: bp2build; bazel build //bionic/...
Test: USE_BAZEL_ANALYSIS=1 m libc

Change-Id: I54f104d359779420d181f195ecbfc9e1269e30be
2021-04-22 01:46:06 -04:00
Liz Kammer
4562a3b218 Add bp2build arch-specific paths mutator
Adds deps for properties tagged `android:"path"` within arch, multilib,
and target properties.

Test: build/bazel/ci/bp2build.sh
Test: m nothing
Bug: 185217298
Change-Id: I0230da399d2c4e984b837f69523fa09eadba3ff1
2021-04-21 18:21:50 -04:00
Rupert Shuttleworth
2a4fc3ecdc Generate BUILD files for every directory that has an Android.bp file.
Test: Added an integration test
Test: bazel build --package_path=out/soong/workspace //bionic/...

Change-Id: Ie34bd23ab3c5428e6c9c9919e5fb6fcb4e709adc
2021-04-21 07:15:50 -04:00
Chris Parsons
bab4d7e3bc Add a denylist for specifically mixed builds
This allows us to, for certain modules, enable bp2build, but disable
mixed builds.
This facilitates easier iteration on these two features, as modules can
be incrementally "fixed" for bp2build, even if they are not ready for
mixed builds integration.

Test: USE_BAZEL_ANALYSIS=1 m libc
Test: Manually verified that note_* modules exist in bionic/libc/BUILD
Test: bazel build //bionic/....
Change-Id: I3462c8398680140cfc953443d96196143b8ad0a3
2021-04-19 12:30:02 -04:00
Liz Kammer
cefa3c75be Denylist two targets that fail for non-x86_64
Test: demo.sh full
Fixes: 185582610

Change-Id: I620029eaaa7dc46d2434c382867b08eacbd9eb91
2021-04-16 11:51:06 -04:00
Rupert Shuttleworth
c143cc50dd Transform paths to headers in include dirs to take package boundaries into account.
This allows the following cc_library_static targets to build with bp2build:

- libc_freebsd
- libc_freebsd_large_stack
- libc_openbsd_ndk
- libc_gdtoa  (* but still fails for mixed builds)
- libc_aeabi
- libc_static_dispatch
- libc_dynamic_dispatch

This also allows a number of other cc_library_static targets to progress further in their builds.

Test: Added unit test
Test: bp2build-sync.py write; bazel build //bionic/...

Change-Id: I71742565c16594448a41a6428a5c993171ec4cb4
2021-04-14 17:05:36 -04:00
Rupert Shuttleworth
43b04926b2 Add reasons for cc_library_static modules to be on the deny list.
Test: bp2build-sync.py write; bazel build //bionic/...
Change-Id: I8fc7036136adbcf16dc31d9691a8d18b99e3c3ae
2021-04-13 08:54:20 -04:00
Jingwen Chen
0a92ed74ad bp2build/mixed builds denylist note_memtag_heap_async and sync temporarily.
This adds the two modules to the bp2build/mixed builds denylist while we
investigate b/185079815.

Test: TH
Change-Id: I6d67c5c24502ac48bdc90a005ddef86c46eab0b7
2021-04-12 05:37:42 +00:00
Jingwen Chen
6393098ecf bp2build: cc_library converter for //bionic/libdl:libdl_android
This CL contains the converter for libdl_android, a cc_library that
expands into a cc_shared_library and a regular cc_library.

Test: TH
Test: bp2build; bazel test //build/bazel/tests/...

Change-Id: If70641a538211b0d6b2aac0e4d0d06912318304d
2021-04-09 09:42:53 +00:00
Chris Parsons
eefc9e6a62 Fix note_memtag bionic libraries in mixed builds
Required fixes:
  - Set MakeLinkType for libraries, even in mixed build mode.
  - Set snapshot header information to empty list, which passes
        validation logic for depending modules

Fixing these libraries also requires a Starlark change to
cc_library_static.bzl, which will be submitted separately.

Additionally, this adds better error messaging in the event that
output files are missing from a mixed-build library.

Test: USE_BAZEL_ANALYSIS=1 m libc
Test: USE_BAZEL_ANALYSIS=1 m runtime-module-sdk
Change-Id: Iad2c4d46359986fb0a43263292a15ed45fabbac7
2021-04-05 15:27:55 -04:00
Christopher Parsons
012ae1f4e0 Merge "Use correct includes for mixed build gen headers" 2021-04-05 19:03:47 +00:00
Chris Parsons
e59af4e1c0 Use correct includes for mixed build gen headers
Previously, genrules which were used as generated_headers only
propagated the old Soong gen directory as an include path, even if the
genrule was managed by Bazel. With this chance, the gen directory of the
Baze output files are used in mixed-build mode.

Test: USE_BAZEL_ANALYSIS=1 m libc
Test: m libc
Change-Id: I3913c4e5b0354421ba1fe43d1f6ef4d1e729a058
2021-04-02 18:27:21 -04:00
Jingwen Chen
91220d7334 Add os/target configurable selects for label list attributes.
This CL is pretty large, so I recommend starting with reading the newly
added tests for the expected behavior.

This change works in conjunction with the linked CLs in the Gerrit topic.
Those CLs add support for new platform() definitions for OS targets
specified in Soong's arch.go, which are configurable through
Android.bp's `target {}` property. It works similary to previous CLs
adding support for the `arch {}` property.

These configurable props are keyed by the OS: android, linux_bionic,
windows, and so on. They map to `select` statements in label list
attributes, which this CL enables for cc_library_headers' header_libs
and export_header_lib_headers props.

This enables //bionic/libc:libc_headers to be generated correctly, from:

    cc_library_headers {
        name: "libc_headers",
        target: {
            android: {
                header_libs: ["libc_headers_arch"],
                export_header_lib_headers: ["libc_headers_arch"],
            },
            linux_bionic: {
                header_libs: ["libc_headers_arch"],
                export_header_lib_headers: ["libc_headers_arch"],
            },
        },
        // omitted props
    }

to:

    cc_library_headers(
        name = "libc_headers",
        deps = [] + select({
            "//build/bazel/platforms/os:android": [
                ":libc_headers_arch",
            ],
            "//build/bazel/platforms/os:linux_bionic": [
                ":libc_headers_arch",
            ],
            "//conditions:default": [],
        }),
    )

Test: TH
Test: Verify generated //bionic/libc:libc_headers
Fixes: 183597786

Change-Id: I01016cc2cc9a71449f02300d747f01decebf3f6e
2021-04-02 08:17:34 +00:00
Rupert Shuttleworth
760fb185be Annotate bp2buildModuleDoNotConvertList with reasons for modules being blocked.
Test: bp2build write; bazel build //bionic/...
Change-Id: I6065c945dbb8747c85aa554eefb355afe58d2078
2021-04-01 04:38:08 +00:00
Chris Parsons
b164b381b2 Register bp2build module types even outside generation
Without this fix, mixed builds are silently disabled, as a recent change
disabled mixed builds for all module types which are not listed in this
map. This map should be populated for both bp2build generation and for
mixed builds.

Test: lunch aosp_flame && USE_BAZEL_ANALYSIS=1 m libc
Change-Id: Ib2a3fb2de2e7a5204ef01355522ae569e0424dc7
2021-03-31 12:26:01 -04:00