Commit Graph

38316 Commits

Author SHA1 Message Date
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
0da7ce6f39 Document testing & build_conversion purposes
Formatter also relocated some imports

Test: Existing tests pass
Change-Id: I48b1cb65400e463a6b4e91fd9fe76e13d8d81cd6
2021-08-23 22:04:53 +00:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
ce0a07e249 Rename runPythonTestCase
To runBp2BuildTestCaseSimple because it's not Python-specific
but rather Bp2Build-specific

Name is suboptimal but given current functionality and use
we couldn't come up with a better one.

Test: Existing tests pass
Change-Id: I641cd051af741836b21ec6a7d6639516fb13028f
2021-08-23 22:02:20 +00:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
1c92aeff4e Share runBp2BuildTestCase & runPythonTestCase
Pull along bp2buildTestCase as well
Because none are specific to their current hosts

Move `errored` up to the "generic" block vs.
the "custom" block

Test: Existing tests pass
Change-Id: Ie8d5322a5554ebd88f0c142a4c541791687be326
2021-08-23 19:20:29 +00:00
Colin Cross
b3c68004af Merge "Set ANDROID_HOST_MUSL for musl builds" 2021-08-23 18:47:40 +00:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
8f33dcc9e3 Merge "Fix inverted error message" 2021-08-20 22:20:23 +00:00
Treehugger Robot
899b98d508 Merge "rust: doc: Zip up docs when done" 2021-08-20 22:11:17 +00:00
Treehugger Robot
13b3b785ec Merge "rust: Hook up InstallIn functions + Product" 2021-08-20 21:03:18 +00:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
d75507fff7 Fix inverted error message
The code checks to see if the `child` is not a Python library,
but then when reporting the error it uses the module name as
the dependency name and the child (dependency) as the module name

Test: Existing tests pass
Change-Id: Ied606342291312d8485a8fd2ddcc580ad24ae82f
2021-08-20 21:02:43 +00:00
Matthew Maurer
ca68c49621 rust: doc: Zip up docs when done
Fetching artifacts from the build server doesn't go well with large
numbers of files in nested directories.

Produce an additional rustdoc.zip artifact to make this easier.

This change also prevents ${DIST} from receiving the unpacked rustdoc
site. This can be changed if b/188822051 is fixed.

Bug: 162741284
Test: m rustdoc; check resulting zip contents manually
Change-Id: I80b6a8fa6e274d2d8c3419d8734251afd4d7dba7
2021-08-20 13:08:16 -07:00
Matthew Maurer
9f59e8db27 rust: Hook up InstallIn functions + Product
InstallIn functions must be hooked up not only to install to special
images, but also to receive special handling by android/arch.go such
that some images are implicitly multilib: first.

Also hook up more product details.

Bug: 178565008
Bug: 165791368
Test: lunch aosp_arm64; m
Change-Id: I1980d5aa9d55f78c222b98d60a404cd6ea5abbfb
2021-08-20 12:14:54 -07:00
Treehugger Robot
99d5a0f5c9 Merge "Revert "Revert "Apply pylint to cc/symbolfile""" 2021-08-19 20:51:40 +00:00
Spandan Das
3f5659f268 Revert "Revert "Apply pylint to cc/symbolfile""
This reverts commit ea28f4cae7.

Reason for revert: Postsubmit failure was unrelated, and was fixed by this revert CL aosp/1802327

Change-Id: I2693b17c2261cfc3dce0ed69baeb2d1f45f784f8
2021-08-19 19:31:54 +00:00
Lukács T. Berki
5470d24ab7 Merge "Use combined.ninja for soong_docs null build test." 2021-08-19 18:04:00 +00:00
Lukacs T. Berki
1a86bd2dc4 Use combined.ninja for soong_docs null build test.
This requires some explanation: turns out, Ninja uses the $builddir
variable declared in the top-level Ninja file to figure out where the
command log should go. This resulted in a loss of incrementality between
the manual Ninja invocation in test_null_build_after_docs because they
used different Ninja files and therefore different values of $builddir.

The reason why this was not a problem before is that the output files of
the main Ninja file and the bootstrap one were separate, but with that
changing, this will start to matter.

Test: Presubmits.
Change-Id: I28959cc7d659e698c7694a58326c8731eac060eb
2021-08-19 16:29:47 +02:00
Treehugger Robot
9ab1e685ee Merge "Revert "Apply pylint to cc/symbolfile"" 2021-08-19 11:43:17 +00:00
Ulyana Trafimovich
abcce6f1c3 Merge "Propagate <uses-library> dependencies from java_library to apps." 2021-08-19 09:14:33 +00:00
Junwei Wang
ea28f4cae7 Revert "Apply pylint to cc/symbolfile"
This reverts commit 73de70264a.

Reason for revert: Broken build 7651218 on aosp-master on aosp_x86_64-userdebug -- b/197190129

Change-Id: I18387e5d2765fecb292127d09e3bbe7cf19b5efd
2021-08-19 08:10:05 +00:00
Treehugger Robot
1c700090e9 Merge "Apply pylint to build/soong/bloaty" 2021-08-19 07:16:04 +00:00
Lukács T. Berki
1e255cf16d Merge "Do globbing-related paperwork explicitly." 2021-08-19 06:17:32 +00:00
Treehugger Robot
4132ef45bb Merge "Collect info on non-cc sources in a cc module." 2021-08-19 05:55:43 +00:00
Matthew Maurer
a8b1ab0370 Merge changes I9682b978,I35465715,I9c5aa5f3,I1c76e620
* changes:
  Support Rust in Ramdisk
  Support Rust in native-bridge
  Support Rust in Product
  Support Rust in Recovery
2021-08-19 02:55:46 +00:00
Liz Kammer
380dbb9327 Merge "Don't create a new module for bp2build conversion." 2021-08-18 19:28:20 +00:00
Treehugger Robot
99f323425a Merge "Support product-copy-files-by-pattern macro" 2021-08-18 19:09:19 +00:00
Spandan Das
16c2b8c3d3 Apply pylint to build/soong/bloaty
1. Run black --line-length 80 -S build/soong/bloaty
to fix formatting
2. Annotate # pylint: disable=import-error to skip checks for imports.
The imports are provided by Soong during m <target>

Test: m bloaty_merger_test
Test: pylint --rcfile tools/repohooks/tools/pylintrc
build/soong/bloaty/bloaty_merger.py
build/soong/bloaty/bloaty_merger_test.py
Bug: 195738175

Change-Id: I4579a80203de41d48992424f264dd1cdbafc854c
2021-08-18 17:46:46 +00:00
Liz Kammer
5641096e7f Merge "Handle arch-specific/not properties the same." 2021-08-18 15:59:49 +00:00
Ulya Trafimovich
e4432874a9 Propagate <uses-library> dependencies from java_library to apps.
Automatically add `uses_libs`/`optional_uses_libs` properties specified
in `java_library` modules to the apps that transitively depend on these
libraries.

Note that a library may choose to specify its <uses-library> deps as
either optional or required. If this is not the right choice for an app,
it should be able to override it by specifying its own `uses_libs`/
`optional_uses_libs` properties.

Add Soong tests to cover the new cases.

Bug: 196377222
Test: lunch aosp_cf_x86_64_phone-userdebug && m
Change-Id: I601f3e2830ed9699b78d911966ee70812d32b4f6
2021-08-18 16:57:11 +01:00
Jay Patel
eaf5e1b3ec Merge "Allow PRODUCT_CFI_INCLUDE_PATHS to work with blueprints also" 2021-08-18 15:18:41 +00:00
Lukacs T. Berki
809d2ed548 Do globbing-related paperwork explicitly.
In particular, the writing of build-globs.ninja and the collection of
glob list files for build.ninja.d is now done outside of the Blueprint
invocation.

This is useful because RunBlueprint() is sometimes called multiple times
in a single soong_build invocation and because now glob processing is
not hidden by the machinery of Blueprint.

Test: Presubmits.
Change-Id: I6eb19b357901e452e6d1e7ce055f8a3bab4a74f4
2021-08-18 17:11:56 +02:00
Martin Stjernholm
3931b8c70d Merge "Remove fallback workaround for APEX variants which is now unnecessary." 2021-08-18 08:37:06 +00:00
Sasha Smundak
0445308bc3 Support product-copy-files-by-pattern macro
Bug: 193566316
Test: internal
Change-Id: I5ffdbaee231669166aaa3f40f84ecac8f571f981
2021-08-17 18:15:03 -07:00
Aurimas Liutikas
91436f469a Merge "Propagate <uses-library> dependencies from android_library to apps." 2021-08-17 22:49:51 +00:00
Treehugger Robot
68a240badb Merge "Apply pylint to cc/symbolfile" 2021-08-17 22:44:37 +00:00
Matthew Maurer
c6868383f4 Support Rust in Ramdisk
Bug: 178565008
Bug: 165791368
Test: Build and link a Rust library into a ramdisk binary
Change-Id: I9682b978936624133e5a62e94caace0e8958fd0f
2021-08-17 15:08:49 -07:00
Matthew Maurer
a61e31f66a Support Rust in native-bridge
Bug: 178565008
Bug: 165791368
Test: Build and link a Rust library into a native-bridge binary
Change-Id: I3546571530529203d9dbfd62777f20de18c6bd51
2021-08-17 15:08:49 -07:00
Matthew Maurer
52af5b052b Support Rust in Product
Bug: 178565008
Bug: 165791368
Test: Build and link a Rust library into a product binary
Change-Id: I9c5aa5f3a1f323af9aa2aee804635045f1b91bd4
2021-08-17 15:08:49 -07:00
Matthew Maurer
460ee9429e Support Rust in Recovery
Bug: 178565008
Bug: 165791368
Test: Build and link a Rust library into a recovery binary
Change-Id: I1c76e6204019443c567082730a4cf680f4a2a74a
2021-08-17 15:08:49 -07:00
Liz Kammer
74ec1686aa Collect info on non-cc sources in a cc module.
Cc modules in Soong can contain non-cc srcs, collect information on
whether they exist in a module. This is not entirely precise as globs
are not evaluated and srcs from other modules (e.g. filegroups,
generated sources) are not handled; however, it should give a decent
lower bound on how often each type occurs.

Bug: 194938896
Test: SOONG_DUMP_JSON_MODULE_GRAPH=/tmp/json_srcs m nothing and verify
      outputs
Change-Id: I7ad52e863727cf5fa4e8d41cf04d27c86e395c6d
2021-08-17 17:29:51 -04:00
Liz Kammer
2ada09a546 Don't create a new module for bp2build conversion.
A performance improvement for bp2build as Blueprint/Soong no longer have
the overhead of additional modules. The creation of these modules
results in:
* traversal of additional modules for each subsequent mutator
* synchronization over a go channel to collect newly created modules:
https://cs.android.com/android/platform/superproject/+/master:build/blueprint/context.go;l=2594,2600;drc=1602226f23181b8c3fbfcaf3358f0297e839d7d3

We avoid both of these by storing the information directly in the
underlying module.

Also as a fringe benefit, removes some necessary boilerplate for
conversion.

For benchmarks, reduces runtime ~1% for 1% converted, ~24% for 100%
converted. See more: go/benchmarks-for-https:-r.android.com-1792714

Test: ran benchmarks/tests in bp2build
Test: build/bazel/ci/bp2build.sh
Change-Id: Ie9273b8cbab5bc6edac1728067ce184382feb211
2021-08-17 15:57:09 -04:00
Liz Kammer
135bf55281 Handle arch-specific/not properties the same.
Refactor arch-handling code to return a "no-config" axis which allows
handling non-arch configed properties the same way as arch-configed
properties.

Test: build/bazel/ci/bp2build.sh
Change-Id: I485b35fd91c28501fe2055234dc9b278488bf4b5
2021-08-17 13:06:54 -04:00
Ulya Trafimovich
42c7f0d173 Propagate <uses-library> dependencies from android_library to apps.
Automatically add `uses_libs`/`optional_uses_libs` properties specified
in `android_library` modules to the apps that transitively depend on
these libraries.

Note that a library may choose to specify its <uses-library> deps as
either optional or required. If this is not the right choice for an app,
it should be able to override it by specifying its own `uses_libs`/
`optional_uses_libs` properties.

Bug: 196377222
Test: lunch aosp_cf_x86_64_phone-userdebug && m
Change-Id: I1e36c56719c0023dcd25fa6aeb92016471702b9f
2021-08-17 16:58:20 +01:00
Lukács T. Berki
0dd067d309 Merge changes from topics "remove-blueprint-main", "remove-some-minibp", "remove-srcdir"
* changes:
  Remove Srcdir() from BootstrapConfig.
  Define command line args in soong_build .
  Move bpglob to .bootstrap from .minibootstrap.
2021-08-17 14:55:09 +00:00
Treehugger Robot
860f42c119 Merge "Most of the TARGET_COPY_OUT_xxx variables are not constant" 2021-08-17 13:05:31 +00:00
Colin Cross
7e7b028b88 Set ANDROID_HOST_MUSL for musl builds
Unlike glibc, musl does not #define __MUSL__ on the assumption that
building against musl will be done using a configure script that should
be testing for individual features rather than assuming them based on
using musl.  We don't use configure-based builds, so add a local
ANDROID_HOST_MUSL macro that will be defined for all host musl builds.

Bug: 190084016
Test: m USE_HOST_MUSL=true adb
Change-Id: I44e3ffd1d727fd0ea8b69c3b074fef7e92e2be41
2021-08-16 14:42:53 -07:00
Sasha Smundak
02183cfc53 Most of the TARGET_COPY_OUT_xxx variables are not constant
Bug: 193566316
Test: internal
Change-Id: I484a88de933904a2ac2e36084509838d7a85262c
2021-08-16 13:39:48 -07:00
Spandan Das
73de70264a Apply pylint to cc/symbolfile
PEP8 recommends using the not operator over comparision to empty string

Test: pytest build/soong/cc/symbolfile/test_symbolfile.py
Test: pylint --rcfile tools/repohooks/tools/pylintrc
build/soong/cc/symbolfile
Bug: 195738175

Change-Id: I4ed2bc0680beccc19a63d8b3f383983d4dff1baf
2021-08-16 18:59:57 +00:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
37abc3c343 Merge "Expose Soong's default clang versions to Bazel" 2021-08-16 18:08:03 +00:00
Lukacs T. Berki
f7e36d80ff Remove Srcdir() from BootstrapConfig.
It was always ".".

Test: Presubmits.
Change-Id: I5381002b3f2986122f1b335a88119cead0a86d75
2021-08-16 17:05:09 +02:00
Lukacs T. Berki
f9008075d9 Define command line args in soong_build .
They used to be defined in Blueprint but since it doesn't have a
separate existence anymore, we can consolidate all command line
arguments here.

Note that Blueprint is called directly from soong_ui, but that happens
by a simple Go function call and not by invoking a separate binary, so
command line arguments are not needed there.

Test: Presubmits.
Change-Id: Ifa2c101ddbe424b76fc5a508d3d41c329e7353ca
2021-08-16 15:29:02 +02:00
Lukacs T. Berki
9d7cf6a336 Move bpglob to .bootstrap from .minibootstrap.
Test: Presubmits.
Change-Id: If6315e497cdc0e6c0f348eb9d4b596ebd13d6303
2021-08-16 14:08:57 +02:00
Lukacs T. Berki
56ebaf35b3 Separate the files of bp2build and soong_build.
The following files are forked:

- build-globs.ninja
- The glob list files
- .d files for the glob list files
- The output file (build.ninja or the bp2build marker file)

This makes bp2build and soong_build not overwrite each other's files
they need for proper incrementality.

Test: Presubmits.

Change-Id: I69d192cbd36ecd9677f46f3fa095dfce6f872227
2021-08-16 09:15:28 +02:00