Commit Graph

50358 Commits

Author SHA1 Message Date
Treehugger Robot
6876d5aa5f Merge "Dedupe libraries in both static and whole static" 2022-03-26 00:32:01 +00:00
Treehugger Robot
1c2d87ba83 Merge "Add metrics collection to soong_build" 2022-03-26 00:21:21 +00:00
Liz Kammer
898e076d6f Add values to json module graph
Increases size of json module graph but it still remains smaller than
action information.

Test: m json-module-graph
Change-Id: I215e92afb0f5d3d688e5e590517392d0cc534455
2022-03-25 18:30:49 -04:00
Chris Parsons
715b08fd82 Add metrics collection to soong_build
Metrics are added to a materialized protobuf soong_build_metrics.pb

Recorded events have scoped names and recorded runtimes.
For example, mixedbuilds includes the following events:

mixedbuilds.prepare
mixedbuilds.analyze.write_files
mixedbuild     (this is a total of all mixedbuild events)

Test: Manually verified materialized protos for mixedbuild, bp2build,
and legacy build.

Change-Id: I2e72d4502c0373bee0279bb1320b2e4dd586129c
2022-03-25 17:19:44 -04:00
Mitch Phillips
c6ea944e18 Merge "[HWASan] Enable zero-initialization." 2022-03-25 20:53:02 +00:00
Colin Cross
f61766e987 Use turbine instead of kapt for kotlin annotation processors
Follow Bazel by using turbine instead of kapt to run annotation
processors.  This still requires using kapt to generate java stubs
of kotlin soruces, then uses turbine to run annotation processors
on the java stubs and any java sources to generate sources and
resources, and passes the annotation processor generated sources
to kotlinc and javac.

Bug: 225013372
Test: m checkbuild
Test: TestKapt
Change-Id: I9c6fc496a9fba64658bb062538bc5f7b9478b07a
2022-03-25 10:28:50 -07:00
Colin Cross
411647e8f4 Don't use ziptime or --temp_dir for turbine
Turbine sets dates to 2010 and doesn't need ziptime any more, and
ignores the --temp_dir flag.

Bug: 225013372
Test: m checkbuild
Change-Id: I6122f27d873ca671050e4dc0d65641bb52e90e99
2022-03-25 10:26:47 -07:00
Spandan Das
183ed1044c Merge "Do not propagate strict_updatability linting to conscrypt and framework-media java_sdk_libraries" 2022-03-25 15:05:06 +00:00
Trevor Radcliffe
dadc037ef3 Merge "Allow appending artifact in dist with product name" 2022-03-25 14:34:23 +00:00
Ivan Lozano
872d579a22 rust: rust_proc_macro host snapshot support.
Adds support for capturing rust_proc_macros as part of the host
snapshot. Proc macros target the host and can be thought of as compiler
plugins. Because of this, they don't have vendor image variants and
can't be easily captured as part of the vendor snapshot. Instead we
capture them as part of the host snapshot.

This adds a rust_prebuilt_proc_macro module type.

Bug: 204304380
Test: m HOST_FAKE_SNAPSHOT_ENABLE=true host-fake-snapshot dist
Test: python3 development/vendor_snapshot/update.py --image=host
  --install-dir=vendor/vendor_name/ 31 --local out/dist
Test: Checked Android.bp for rust_prebuilt_proc_macro modules.
Change-Id: I4a8c4d9c41b7ca361b5b97d3f74973918c2a5fe3
2022-03-25 09:25:07 -04:00
Yi Kong
846a25912d Merge "Update to clang-r450784" 2022-03-25 05:49:48 +00:00
Yi Kong
75ce5d3320 Update to clang-r450784
Test: presubmit
Bug: 219872355
Change-Id: I3cd5f42d887b07929906e08d7d9f1a677b8a4403
2022-03-25 04:30:53 +00:00
Spandan Das
2cf278e75b Do not propagate strict_updatability linting to conscrypt
and framework-media java_sdk_libraries

Tmp fix for
https://android-build.googleplex.com/builds/quarterdeck?branch=aosp-master&target=errorprone&lkgb=8352129&lkbb=8353967&fkbb=8352538

Test: m
out/soong/.intermediates/external/conscrypt/conscrypt.module.public.api.stubs.system/android_common/lint/lint-baseline.xml
Test: (internal CI target) https://android-build.googleplex.com/builds/abtd/run/L67700000953708519
Bug: 182349282
Bug: 226637002
Bug: 226608044

Change-Id: I17e540877200f59bfd51e69e57d8bcca6cf3850f
2022-03-25 02:31:20 +00:00
Jooyung Han
b3a3a2893b Add a tool to list contents of .img file
list_image <img> lists the contents of the file.

For example:
  $ m microdroid
  $ list_image <path to microdroid.img>
  ./
  ./apex/
  ./bin
  ..

Bug: 195425111
Bug: 225121718
Test: list_image .img
Change-Id: I7c4fca184751ba20066fd25fa4c366a955dbabf6
2022-03-25 02:44:36 +09:00
Spandan Das
ad40fffa64 Merge changes I4c373b29,I9ccda6fc,I1b390b0e,I30151217
* changes:
  Do not propagate strict updatability linting to libcore/
  Create allowlist to skip strict updatability lint check
  Propagate strict_updatability_linting to transitive deps of updatable apexes
  Export LintDepSetsIntf from java package
2022-03-24 16:38:20 +00:00
Trevor Radcliffe
90727f4576 Allow appending artifact in dist with product name
Some modules need to have their artifacts copied to dist with the
product name appended. This CL enables that functionality in a
boolean Soong property called append_artifact_with_product.

Fixes: 224561567
Test: Unit tests and build relevant target/modules
Change-Id: I4b824d4001679cebf0a9059be2d090d33a310933
2022-03-24 14:46:18 +00:00
Andrei-Valentin Onea
b47e89f2d1 Merge "Revert "Prevent unspecified values in soong_config_string_variable"" 2022-03-24 14:11:18 +00:00
Andrei-Valentin Onea
38944c70c4 Revert "Prevent unspecified values in soong_config_string_variable"
This reverts commit 542905a280.

Reason for revert: This check is breaking multiple builds, creating fixes to minimize re-land work.

Change-Id: I6431bd0864f49781910dad787e1ba9375b9931b3
2022-03-24 13:52:58 +00:00
Treehugger Robot
1da556928a Merge "Prevent unspecified values in soong_config_string_variable" 2022-03-24 00:44:47 +00:00
Mitch Phillips
597605603a [HWASan] Enable zero-initialization.
Previously, we use to fill memory with 0xbe bytes. This caused a lot of
problems that necessitated disablement. For example, 0xbe-filled mutexes
are apparently locked, and there were a few instances of
uninitialized-mutex use.

Given that zero-fill is now the default behaviour, enable zero-init in
HWASan as well.

For now, only fill the first page. It would be preferable to fill the
whole allocation, but I don't want to spin for too many cycles filling
huge secondary pages. In future, we might change the behaviour to have
an explicit "zero initialize" option that completely fills the primarily
allocations, and knows it's unnecessary for the secondary.

Bug: 226078464
Test: Boot w/ HWASan (done by presubmit robot)

Change-Id: I7de3a7f9fa2fdeb5116e5bf6586babe4d06fcb91
2022-03-22 18:38:34 +00:00
Cole Faust
ea9db58de9 Add the handle to artifact path requirement functions
Bug: 188079133
Test: go test
Change-Id: I320f4f14fb47414cb14ebb404efab4bceb1fa1e4
2022-03-22 11:17:16 -07:00
Colin Cross
057beb2146 Merge "Collect deps from r8" 2022-03-22 16:40:30 +00:00
Colin Cross
22e6a6f69a Collect deps from r8
Use the new --deps-file argument to collect a deps file from r8, which
will correctly track files included by proguard flags files.

Fixes: 221956283
Test: manual
Change-Id: I92341948311c2c994cc0ae032dc19e09b2e80550
2022-03-22 16:40:13 +00:00
Sam Delmerico
f241e72b57 Merge "add module type allowlist to bp2build" 2022-03-22 16:35:49 +00:00
Treehugger Robot
9b0af33cc8 Merge "Handle arch/cpu variants in mixed builds" 2022-03-22 15:52:51 +00:00
Treehugger Robot
c4ad11ec28 Merge "Prevent kotlinc from reading /tmp/build.txt" 2022-03-22 07:28:05 +00:00
Treehugger Robot
5beafc1900 Merge "Replace $(call my-dir) with a string literal" 2022-03-22 03:24:47 +00:00
Treehugger Robot
c9e38327dd Merge "Parse variable references with #s" 2022-03-22 03:15:47 +00:00
Treehugger Robot
bc1217b4e7 Merge "Revert "225341614: Enforce ExtraText in platform builds"" 2022-03-21 21:40:12 +00:00
Treehugger Robot
2f39f7dacd Merge "Don't pass static libs to r8" 2022-03-21 19:21:02 +00:00
Tor Norbye
73dca5c905 Revert "225341614: Enforce ExtraText in platform builds"
This reverts commit 4c471458ec.

Reason for revert: There were failures in some build targets that
must not have run during presubmit checks (for example,
aosp-master/errorprone@8332220)

Change-Id: I220fd70df663ba49d876fb3ebf841fa9f3f06b97
2022-03-21 18:56:14 +00:00
Tor Norbye
6014299148 Merge "225341614: Enforce ExtraText in platform builds" 2022-03-21 17:58:32 +00:00
Liz Kammer
0940b89937 Handle arch/cpu variants in mixed builds
Test: mixed_libc.sh
Change-Id: Ib5084060199b8e1a5955bd51bc3fd793bb83a1eb
2022-03-21 11:13:49 -04:00
Tor Norbye
4c471458ec 225341614: Enforce ExtraText in platform builds
Bug: 225341614
Test: N/A
Change-Id: Ic2ed5006643f3646315f16fe77552398f38c0265
2022-03-21 08:12:25 -07:00
Liz Kammer
cc2c1efc9a Dedupe libraries in both static and whole static
In Soong, libraries can appear in both static and whole static libs;
however, in Bazel, this results in errors due to duplicate libraries.

Test: bp2build.sh
Change-Id: I9dda4f7a7812340d28a6160d0e25f421c9db6612
2022-03-21 09:03:29 -04:00
Treehugger Robot
13642d4735 Merge "Do not dump metrics if reproxy was never started" 2022-03-21 07:05:41 +00:00
Treehugger Robot
ca8e93e79d Merge "Add google_prod_creds to the list of auth mechanisms" 2022-03-21 07:01:31 +00:00
Yu Liu
ba4638f529 Merge "Support tailoring clang --target flag based on min sdk version." 2022-03-19 15:52:15 +00:00
Cole Faust
f5adedce03 Replace $(call my-dir) with a string literal
This is so that we can set LOCAL_PATH to the result
of my-dir, as LOCAL_PATH can only be set to a string
literal of the exact value of LOCAL_PATH.

It's probably also the correct choice to start phasing
out LOCAL_PATH.

Bug: 214405650
Test: go test
Change-Id: Ia97d7fedf4ce62643921d90a176e65edd4e2fce6
2022-03-18 14:05:06 -07:00
Colin Cross
c2d504873f Prevent kotlinc from reading /tmp/build.txt
Pass -Didea.plugins.compatible.build=999.SNAPSHOT to kotlinc and kapt
to prevent it from reading /tmp/build.txt and failing with an obscure
exception:
exception: java.lang.ExceptionInInitializerError
	at com.intellij.openapi.util.BuildNumber.currentVersion(BuildNumber.java:297)
...

Bug: 222162908
Test: m metalava with /tmp/build.txt present
Change-Id: I5faced4a2e94561ce3aa37bfaacf5bd7d988a4f0
2022-03-18 20:48:51 +00:00
Colin Cross
9bb9bfb349 Don't pass static libs to r8
r8 gets the static libs in the program jar, it shouldn't also get
them as library jars.  Keep a separate classpath for dexing that
includes libs but not static_libs.

Bug: 222468116
Test: m checkbuild
Test: TestD8
Test: TestR8
Change-Id: Icca3393f496cbcadcc633f3b156761e6c145f975
2022-03-18 20:48:27 +00:00
Treehugger Robot
3f57c6a2c8 Merge "Adds min_sdk_version to device_kernel_headers headers lib." 2022-03-18 18:52:27 +00:00
Paul Duffin
2cb89166df Merge "Make bootclasspath_fragment hidden API package checks exhaustive" 2022-03-18 18:42:37 +00:00
Nicolas Geoffray
1a9df2604c Merge "Pass --preloaded-classes to boot image compilation." 2022-03-18 13:41:59 +00:00
Jingwen Chen
c35fbdcd90 Merge changes from topics "e2fsdroid", "e2fsprogs-bazel"
* changes:
  Allowlist //external/e2fsprogs/...
  Build e2fsdroid with Bazel.
2022-03-18 11:34:31 +00:00
Paul Duffin
846beb7f54 Make bootclasspath_fragment hidden API package checks exhaustive
Previously, the bootclasspath_fragment's hidden_api.split_packages and
hidden_api.package_prefixes properties did not specify an exhaustive
set of packages that were provided by the fragment. They excluded
packages which were either not split or which could not be used as a
package prefix because it would match sub-packages provided by other
bootclasspath modules.

This change adds the hidden_api.single_packages list to specify those
additional packages and then uses that information to verify that any
bootclasspath_fragment that specifies at least one of split_packages,
single_packages or package_prefixes properties only contains classes
from a package that matches one of those properties. That will
prevent a module from accidentally including unexpected classes, such
as might happen when statically including a common utility library.

It also adds coverage specific versions of the properties as additional
packages are added to the art-bootclasspath-fragment when building
coverage builds.

Bug: 194063708
Test: atest signature_patterns_test
      m out/soong/hiddenapi/hiddenapi-flags.csv
      m EMMA_INSTRUMENT=true EMMA_INSTRUMENT_FRAMEWORK=true out/soong/hiddenapi/hiddenapi-flags.csv
      # Breaks without corresponding change to add android.system to
      # the art-bootclasspath-fragment.
      /usr/bin/pylint --rcfile $ANDROID_BUILD_TOP/tools/repohooks/tools/pylintrc scripts/hiddenapi/signature_patterns*.py
      pyformat -s 4 --force_quote_type single -i scripts/hiddenapi/signature_patterns*.py
Change-Id: Iddf6c59cd4dc8c36dde7943a9840ccef5794b320
2022-03-18 10:54:51 +00:00
Kousik Kumar
a1d8fa9961 Do not dump metrics if reproxy was never started
Dumping of RBE metrics should only happen when reproxy was started.
Reproxy is started only when Soong is run in make / build mode and not
in dumpvars mode.

Doc: https://docs.google.com/document/d/1Iqhu5QgozB_rA9fucMHO6X4U6A8e8qFtZyub9j83ICQ/edit#
Test: Ran a build with / without RBE. No metrics dumped during lunch
without RBE and metrics dumping worked as expected with RBE.
Bug: b/197213143

Change-Id: Ic3403331e1f4ac2c7c72fd9169aa7b82a7545c75
2022-03-18 02:53:36 -04:00
Kousik Kumar
93d192cc6a Add google_prod_creds to the list of auth mechanisms
This is to ensure that we don't default to using
application_default_creds when it is not set and rather
"RBE_use_google_prod_creds" variable is set. This is necessary to
rollout RBE gcert auth.

Bug: b/224446621
Change-Id: I51e3f1aefadf282240b8f861efafde6b9b61d305
2022-03-18 01:39:56 -04:00
Yu Liu
fc603167f4 Support tailoring clang --target flag based on min sdk version.
Bug: 215748260
Test: Tested using aquery to verify the --target flag; add added unit

Change-Id: Ifb13509db7d1e110316aa44d55ec9cccaa4b83d1
2022-03-18 01:16:52 +00:00
Cole Faust
e309a91ca8 Parse variable references with #s
Inside a variable reference, a # does not start
a comment.

Fixes: 218742602
Test: go test
Change-Id: I16cf04c74a8aa30482fd9293175f893e4efb60f1
2022-03-17 17:15:12 -07:00