Commit Graph

298 Commits

Author SHA1 Message Date
Colin Cross
f059b2f2fc Remove obsolete java version environment variables
jdk9 and jdk11 are no longer used, remove ANDROID_JAVA9_HOME and
ANDROID_JAVA11_HOME.

Bug: 221270882
Test: treehugger
Change-Id: Iad566bff21055c5e2acd62642ccd579507e21fef
2023-11-01 20:58:30 +00:00
Colin Cross
2650f35faa Merge "Use correct path for soong_build_metrics.pb when dist is enabled" into main 2023-11-01 16:54:14 +00:00
LaMont Jones
c416ddd1cc Merge "Add support for PRODUCT_RELEASE_CONFIG_MAPS" into main 2023-11-01 16:01:08 +00:00
Colin Cross
b67b061baa Use correct path for soong_build_metrics.pb when dist is enabled
When dist is enabled soong_build_metrics.pb is written to $DIST_DIR/logs,
not $OUT_DIR.

Test: m dist nothing, examine out/dist/logs/bulid.trace.gz
Change-Id: Ia41de6c838ecbcf0acca1e7ec7e68ba24dce2b32
2023-10-31 23:52:38 +00:00
LaMont Jones
772e21ba27 Add support for PRODUCT_RELEASE_CONFIG_MAPS
This allows product config to determine what build flags are present in
a product's build, as well as their value.

As product config moves to a declarative specification, this can be
handled with less unnecessary work.

Bug: b/302593603
Test: manual
Change-Id: If6ae19e3da8471eaba8d6e048769f8ac6b244155
2023-10-31 18:47:18 +00:00
Colin Cross
ab766b40e8 Merge "Copy soong_build metrics into build.trace.gz" into main 2023-10-30 19:40:31 +00:00
Colin Cross
aa9a273b40 Copy soong_build metrics into build.trace.gz
Make soong_ui read soong_build_metrics.bp to extract the event timings
and propagate them to Tracer, which will put them in build.trace.gz.
This provides much better visibility into what parts of the build are
contributing to the overly large analysis time.

Test: m nothing and examine build.trace.gz
Change-Id: I473727f1ec044b0d973f2cb4e3eaca96bfca94f6
2023-10-27 19:36:47 -07:00
MarkDacek
39825ead17 Remove Bazel metrics/finalize-bazel-metrics.
Test: m nothing
Change-Id: Idd4610aca77f0a5deeadc3b895869879ded6c16e
2023-10-26 20:14:36 +00:00
Chris Parsons
bf8ab858da disable bazel by default
Test: m nothing, verified bp2build was not run
Test: presubmits
Change-Id: I7476194f8424fa4f199fb94e1cbd0b3c84339fd9
2023-10-26 17:22:27 +00:00
Colin Cross
106d6ef459 Disable Soong tests by default on Linux
Disable running the Soong tests when building on Linux.  The tests
are still enabled when running on Darwin to ensure we have test on
Darwin in CI.  Checkbuilds still depend on the tests, which will
provide test coverage in CI on Linux.

Bug: 269296618
Test: aninja -t path checkbuild out/host/linux-x86/bin/go/soong-java/test/test.passed
Test: m nothing
Change-Id: Ib8359ff6d2a92eeb2ac9a9e95ecd65abc4e40061
2023-10-24 11:00:59 -07:00
Sebastian Pickl
1c4188c9da Revert "Default from-text stub generation in build"
This reverts commit 182b56b870.

Reason for revert: prime suspect for breaking boot tests b/307495247, b/307411752

Bug:307495247
Change-Id: Iea05703b767d2699ca3cf69377eb44b1d21697ad
2023-10-24 11:20:06 +00:00
Jihoon Kang
182b56b870 Default from-text stub generation in build
This change defaults Java stubs to be generated from API text files
during build. Using the `--build-from-source-stubs` flag, users can
toggle between the feature.

Test: m nothing && verify ninja path exists between android_stubs_current and android_stubs_current.from-text, and does not exist between android_stubs_current.from-source, m nothing --build-from-source-stub && verify the opposite
Bug: 274805756
Change-Id: I28834f92c1b1311e3fe0a71a6ea9e8ec2e278d7e
2023-10-20 23:11:19 +00:00
Cole Faust
06ea531ab3 gofmt soong
Test: Presubmits
Change-Id: Ia76c35ba51685eca29df51738000eacd3f43ce20
2023-10-18 17:42:59 -07:00
Jihoon Kang
e100a884e3 Merge changes from topic "enable-hidden-api" into main
* changes:
  Enable hiddenapi check for from-text stub build
  Utilize module lib and test api superset module in hiddenapi
2023-10-12 17:33:35 +00:00
Jihoon Kang
dff2aea212 Enable hiddenapi check for from-text stub build
This change removes the UNSAFE_DISABLE_HIDDENAPI_FLAGS env var setting
during from-text stub build, and enables hiddenapi list during from-text
stub build.

Test: ENABLE_HIDDENAPI_FLAGS=true m --build-from-text-stub
Bug: 275570206
Change-Id: Ic8cd60e376b978ccc658ff43a44d082eb2759fa5
2023-10-11 17:46:29 +00:00
Ramy Medhat
bc06176350 Set download_tmp_dir explicitly to be a directory under the output directory.
This directory is where outputs are download first before being moved
to their final destination to ensure atomicity.

Change-Id: Ic224bf21c07566de00d292d02f1c0f7f727dcb08
2023-10-10 22:33:46 +00:00
Cole Faust
583dfb426f Generate build_number.txt only once
build_number.txt was generated in runKati, and runKati is run multiple
times throughout the build. If a build number is not set, then one will
be generated using the current timestamp, meaning that multiple
different build numbers will be used in different phases of the build.

Instead, generate build_number.txt during SetupOutDir(), so that we
only have 1 build number.

This is a resubmission with a change to ensure out/soong is created
before writing the build number file.

Bug: 297269187
Test: edit a bzl file to force bazel to rerun, change the code so that use_fixed_timestamp=true during partition builds, m installclean, m bazel_sandwich, and observe that the bazel/make partitions are byte-for-byte identical
Change-Id: I47abcca166c701bb66a6a7731aecad5818279244
2023-09-28 17:56:43 -07:00
Roy Luo
f852b72a22 Revert "Generate build_number.txt only once"
This reverts commit ab6e524709.

Reason for revert: DroidMonitor-triggered revert due to breakage https://android-build.corp.google.com/builds/quarterdeck?branch=aosp-main&target=ndk&lkgb=10879153&lkbb=10879224&fkbb=10879224, bug b/302608661.

Change-Id: I07d47a92ea9a9f5d329e813859316decfd75cde1
BUG: 302608661
2023-09-29 00:28:32 +00:00
Cole Faust
ab6e524709 Generate build_number.txt only once
build_number.txt was generated in runKati, and runKati is run multiple
times throughout the build. If a build number is not set, then one will
be generated using the current timestamp, meaning that multiple
different build numbers will be used in different phases of the build.

Instead, generate build_number.txt during SetupOutDir(), so that we
only have 1 build number.

Bug: 297269187
Test: edit a bzl file to force bazel to rerun, change the code so that use_fixed_timestamp=true during partition builds, m installclean, m bazel_sandwich, and observe that the bazel/make partitions are byte-for-byte identical
Change-Id: I5bf3bb6c78e7529749ca9275f67db3f2f9e66af2
2023-09-28 14:11:43 -07:00
Chris Parsons
73f411b72c Delete ApiBp2build soong_build mode
This feature is obsolete.

This makes a large number of codepaths "dead code" (such as
module-specific implementations of ApiBp2build functionality). These
will be deleted in a followup CL.

Bug: 284029211
Test: Presubmits
Change-Id: Ib53b99f1fe8c24380d219caf44e9bb3b96724fa0
2023-09-05 17:09:51 -07:00
Treehugger Robot
3537d8028a Merge "Disable zipbomb detection inside the build" into main 2023-08-09 22:04:03 +00:00
Kousik Kumar
97e9b0cb8e Merge "Revert "Revert "Remove env config fetcher code""" into main 2023-08-08 15:59:56 +00:00
Kousik Kumar
8036b163ef Revert "Revert "Remove env config fetcher code""
This reverts commit d93c67f64c.

Reason for revert: USE_RBE has again been made the default now, so we should no longer need this code.

Change-Id: I4163c61eed90163e763c29e07dd6edfc9c41b9b9
2023-08-04 19:46:16 +00:00
Colin Cross
fe5ed4d6e2 Disable zipbomb detection inside the build
Some versions of Fedora include patches to unzip to enable zipbomb
detection that incorrectly handle zip64 and data descriptors and fail
on large zip files produced by soong_zip.  Disable zipbomb detection
inside the build.

Reported upstream in https://bugzilla.redhat.com/show_bug.cgi?id=2227130.

Bug: 286885495
Test: builds
Change-Id: I8e4438720bbb17a073ff3b5535f01c2827485838
2023-07-28 09:31:03 -07:00
Kousik Kumar
6d1e348cc3 Disable RBE env variable when we cannot support RBE
Enabling the RBE variable can cause issues around pool parallelism when
we set the env variable, but don' run the build with RBE due to auth issues.
So, I am explicitly unsetting the variables in those cases.

Bug: b/292224253
Tested: https://b.corp.google.com/issues/292224253#comment26
Change-Id: I19718b4ee6c058ba1b11d3df260421bbf8c9567e
2023-07-24 17:06:42 +00:00
Jingwen Chen
d7ccde11d6 Don't use RBE for bp2build/queryview/api_bp2build/json_module_graph,
even with USE_RBE=true.

These are alternate modes of soong_build that don't use rbe/reclient.

This lets us to remove USE_RBE=false from the bp2build run.

Test: presubmits
Change-Id: I439f9cf7e92ec85ca56baec5f62a83ee49b510d4
2023-06-30 09:32:32 +00:00
Kousik Kumar
67ad434ffc Ignore USE_RBE when on glinux laptops
When there's no stubby and we are asked to use google prod creds, that's invalid configuration, so don't use RBE

Bug: b/283828386
Change-Id: I1564b4f70e46fb90c87a0432c46616caa1614aac
2023-06-23 18:43:54 +00:00
Andus Yu
f3da8ee95b Merge "Do not check ssh credentials when running gcertstatus" 2023-06-22 19:38:33 +00:00
andusyu
0b3dc03f9d Do not check ssh credentials when running gcertstatus
reproxy only requires LOAS credentials for authentication. Append the
-nocheck_ssh option to gcertstatus so we don't check ssh credentials.

Test: RBE successfully started after running gcertdestroy --nodestroy_loas2
Bug: b/287297140
Change-Id: I413931b10d8e8d9ae2f8acd7ebfe37d8b3bba455
2023-06-21 17:30:21 -04:00
MarkDacek
396491eee4 Add bes-id to soong. Also renames the proto field.
This is for integration with BES for bazel test metrics.
Bug: 287102416
Test: b build libcore:all --invocation_id=`uuidgen`
Test: build/bazel/scripts/analyze_build (to verify it's set)

Change-Id: I541b2d65bfa85718fc916582e0540384ae3810d9
2023-06-21 14:48:17 +00:00
Treehugger Robot
d34e4c72f3 Merge "Increase config fetcher timeout to 20s" 2023-06-15 18:37:53 +00:00
Chris Parsons
21f80277c3 Remove bazel dev mode
This mode is no longer in use, and is dead code.

Fixes: 282809863
Test: Presubmits
Change-Id: I45ab7d7c9dee153f1cbc66262c2cf9f1e9487a28
2023-06-15 14:14:19 +00:00
Kiyoung Kim
eaa55a8ce1 Rename soong.environment.used with product name
From the previous work on renaming build.ninja, it was found that there
are extra dependencies from build.ninja which can be varied by
TARGET_PRODUCT : which is soong.environment.used.<tag>. This change
renames soong.environment.used to have target product between 'used' and
'<tag>' if available.

Bug: 277029044
Test: Test confirmed that build.ninja is not being re-generated
Change-Id: I987b6bd1a8b4f06dac52537e4178d8556251d254
2023-06-08 01:07:47 +00:00
Kousik Kumar
a3a2af6b64 Increase config fetcher timeout to 20s
This is to make sure timeout doesn't show up for users from APAC / EMEA regions

Bug: b/283828386
Change-Id: Ie04352553494634070e4c5e83a68b3004fbdf88c
2023-06-06 17:29:11 -04:00
Kousik Kumar
ac985b9c59 Merge "Revert "Remove env config fetcher code"" 2023-06-06 21:26:36 +00:00
Jeongik Cha
a87506f5e8 Use HINT_FROM_SOONG if ninja_log doesn't exist
In non-incremental build, there is no ninja_log. For this case, use
HINT_FROM_SOONG as an alternative solution.

Bug: 273947040
Test: 1.m after removing out/.ninja_log
      2.check if non-incremental CI build uses HINT_FROM_SOOONG
      3.check if incremental CI build uses NINJA_LOG
      4.check if there is no regression in CUJ
Change-Id: I00cd216df096cb2288eeab233729acefb0d1b73c
2023-06-02 15:50:02 +01:00
Treehugger Robot
9cae81483a Merge "implement USE_BAZEL_VERSION for m" 2023-06-01 22:06:17 +00:00
Chris Parsons
c83398ffb4 implement USE_BAZEL_VERSION for m
This allows USE_BAZEL_VERSION to be set for m builds, which will use
Bazelisk for any Bazel invocations during those builds.

This should be used only for manual debugging, typically to either test
new Bazel features, verify Bazel compatibility with Android, or culprit
find new Bazel breakages.

Test: Manually run builds with USE_BAZEL_VERSION, toggled off and on to
ensure the build was rerun. Tested with a broken commit, a working
commit, and 'last_green' special term

Change-Id: I8b475dca5c8d4bd849ee3724a8c3aca9b631bcb8
2023-06-01 17:42:18 +00:00
Jeongik Cha
c018d84a00 Merge "Enable NINJA_LOG as weight list source" 2023-06-01 00:40:42 +00:00
Kousik Kumar
d93c67f64c Revert "Remove env config fetcher code"
This reverts commit 6324647bcc.

Reason for revert: This change negatively impacts users running builds on gLinux laptops. Reverting to limit impact and reland with appropriate fix.

Bug: b/284985772
Change-Id: I04a3d0107bb3ea680ba6641240620736a2ef5f17
2023-05-30 20:32:08 +00:00
Kousik Kumar
678e680a75 Merge "Remove env config fetcher code" 2023-05-30 17:28:10 +00:00
Saagar Jha
204beb184a Merge "Replace prodcertstatus with gcertstatus" 2023-05-30 11:12:43 +00:00
Saagar Jha
f513b23ac4 Replace prodcertstatus with gcertstatus
Prodaccess is apparently deprecated now.

Test: Build with/without credentials and check that it works

Change-Id: Iaadaebe84fe9531b2d7d5f0d265751462bb827d8
2023-05-26 00:23:50 +00:00
Kousik Kumar
6324647bcc Remove env config fetcher code
This code is no longer needed since we are now setting USE_RBE to true
for 100% of users (except two users, who I'll reach out to over email).
So fetching the configuration is no longer needed since sometimes this
can result in RBE config flipping between on and off (depending on
whether you have gcert or not).

Tested: Ran the build with latest sync and ensured RBE was used for the
build.

Bug: b/283828386
Change-Id: I3cda78607b46a0e161b7841560dc587a44d91f27
2023-05-25 15:23:29 +00:00
Kiyoung Kim
a37d9baa96 Rename build.ninja with product name
Current build.ninja does not contain any product name, while other ninja
files (such as combined ninja) do. This change adds product name to the
build.ninja so it can be separated over multiple lunch targets

Bug: 277029044
Test: build succeeded and checked if out/soong/build.ninja has been
renamed

Change-Id: I16dc71f829fd76f01b98da0d509a8e0ef6f62fa9
2023-05-25 10:09:45 +09:00
Jeongik Cha
f2ecf7659e Enable NINJA_LOG as weight list source
Bug: 273947040
Test: m
Change-Id: I1a803f5c1da75235395fc594e4f9d17f48c55db8
2023-05-19 14:03:45 +09:00
Chris Parsons
fddeaff4a6 Revert "Turn on HINT_FROM_SOONG by default"
This reverts commit f5c872f36b.

Revert reason: The original commit seems to have a performance regression for a number of benchmarked performance CUJs.

Bug: 283143307

Test: Benchmarking results. See attached bug.
Change-Id: Ib9463c8d30a5ba61640993424696a84e2e03040a
2023-05-17 22:30:34 +00:00
Treehugger Robot
abc2b2adb3 Merge "Turn on HINT_FROM_SOONG by default" 2023-05-15 14:36:05 +00:00
MarkDacek
d33c2fd5a2 Add --bazel-exit-code parameter to soong_ui.
This is for use in tracking bazel exit codes from b invocations.

Bug: 279754118
Test: b build libcore:all (with prints to verify)
Change-Id: If36b5b73ff93cd15647e5c1c1f676e95137210ff
2023-05-04 21:27:34 +00:00
Anton Hansson
c8cea365f3 Merge "Remove --skip-kati" 2023-04-27 11:26:50 +00:00