Commit Graph

1083 Commits

Author SHA1 Message Date
Cole Faust
eda96ba50f Remove Android.bp file input to primaryBuilders
The soong primary builder action lists all the Android.bp files
as dependencies via a depfile. The top-level one doesn't need to be
listed as an input explicitly. I think it was there from the early
days of soong when soong started at the top-level Android.bp file and
traversed downwards through references via the subdirs or
optional_subdirs variables.

Bug: 364749114
Test: Presubmits
Change-Id: I72eabe4d4179696bfa258c8c4c56351a132e0870
2024-09-05 15:26:32 -07:00
Taylor Santiago
8b0bed7187 Reapply "Clean environment variables to account for sandbox work directory."
with minor edits.

This reverts commit 9543d19b4b.

Bug: 363037195
Change-Id: I4cb10b312b7f468185bfad12a16f9b4b64e7a58a
2024-09-04 22:55:02 +00:00
PODISHETTY KUMAR (xWF)
cdd70d268a Merge "Revert "Clean environment variables to account for sandbox work directory."" into main 2024-09-02 06:57:34 +00:00
PODISHETTY KUMAR (xWF)
9543d19b4b Revert "Clean environment variables to account for sandbox work directory."
This reverts commit 0af8ea14fc.

Reason for revert: <Droidmonitor created revert due to b/363848580. Will be verifying through ABTD before submission.>

Change-Id: Iafba28897ad27df67ef8ae8904454196c482216d
2024-09-02 03:54:36 +00:00
Taylor Santiago
3cd732bcd1 Merge "Clean environment variables to account for sandbox work directory." into main 2024-08-31 04:04:22 +00:00
Taylor Santiago
0af8ea14fc Clean environment variables to account for sandbox work directory.
Unset HOME to prevent username leak.

Bug: 363037195

Change-Id: I7a8694f746d58de8f6e41adb6ad52abbaf955ef4
2024-08-29 20:12:35 +00:00
Wei Li
91a1e9f894 Merge "Add "packages/" to Android.mk denylist." into main 2024-08-26 21:24:56 +00:00
Wei Li
ffb99497a7 Add "packages/" to Android.mk denylist.
Bug: 318428689
Change-Id: I229fb645cdf03881ed709216dd6743d456440403
Test: CIs
2024-08-23 22:54:40 +00:00
Qing Shen
713c54289c Revert "Revert "Add a coverage suffix to avoid Ninja file regene..."
Revert submission 3235973-revert-3207217-emma-flip-time-saver-BYWASVIMTD

Reason for revert: Rolling forward with a fix

Adding a coverage suffix for product_config.go too:

FAILED: ninja: 'out/soong/soong.sdv_core_cf.extra.variables', needed by 'out/soong/.intermediates/build/soong/product_config/product_config.json', missing and no known rule to make it
03:42:23 ninja failed with: exit status 1

Reverted changes: /q/submissionid:3235973-revert-3207217-emma-flip-time-saver-BYWASVIMTD

Change-Id: I74d0e3fda09b1367f3dbb939d4f47c4d09b3d51f
2024-08-23 21:41:52 +00:00
Cole Faust
4e58bba1fc Use musl build of n2
We're seeing bugs on the build server only, and previous similar bugs
were caused by issues in the older version of glibc the build server
uses, so try using the musl build of n2 to see if it resolves the
issues.

Bug: 318434287
Test: Presubmits
Change-Id: I416cd77f4dfb4ca04f6dd5b36fa73585a2904b55
2024-08-22 14:27:03 -07:00
Hugo Drumond Jacob
079871cd55 Merge "Revert "Add a coverage suffix to avoid Ninja file regeneration."" into main 2024-08-21 17:20:34 +00:00
Hugo Drumond Jacob
134fa5df00 Revert "Add a coverage suffix to avoid Ninja file regeneration."
Revert submission 3207217-emma-flip-time-saver

Reason for revert: b/361236607

Reverted changes: /q/submissionid:3207217-emma-flip-time-saver

Change-Id: I7b4dcd3f6e73e302a2c3ef7ed5c02f6345619463
2024-08-21 13:07:06 +00:00
Qing Shen
ff545caecd Merge "Add a coverage suffix to avoid Ninja file regeneration." into main 2024-08-20 01:16:53 +00:00
LaMont Jones
b2ab527798 Pass through RUST_LOG
If the user has RUST_LOG set in the environment, pass that through.

Bug: 352600283
Test: manual
Change-Id: Ia6bfa4be791a8e72591fc729de4e4270df150a1c
2024-08-14 10:08:50 -07:00
Qing Shen
9e05d1c5cd Add a coverage suffix to avoid Ninja file regeneration.
This CL uses the environment variable, EMMA_INSTRUMENT to determine whether or not coverage is enabled. If coverage is enabled, it adds a bunch of suffixes to the files generated in out/soong/ folders.

This change reduces the build time by avoiding ninja regeneration when users simply switches from atest to atest --experimental-coverage, or vice versa, and no Android.bp changes are made.

Bug: 331444846
Test: Locally run m libc, and then EMMA_INSTRUMENT m libc twice, no ninja regenerations required. Compared the hash of out/target/product/vsoc_x86_64/ between the before-change repo and after-change repo. Only diff is the after-change directory contains an extra file `out/target/product/vsoc_x86_64/.installable_files` while the original directory has only `out/target/product/vsoc_x86_64/.installable_files.previous`, both files have the same hash.
Change-Id: I972aec20a9d97b223cc833ee245016ac4b0f09a2
2024-08-13 02:04:53 +00:00
Taylor Santiago
2611851112 Individually mount children of root with ABFS enabled and respect
OUT_DIR configuration.

Mounting '/' directly as read-only results in failed bindmounts at /src
if that directory does not already exist due to failure to mkdir.

Change-Id: I7f11c45e82374d142a86fcb2f03e53ddacffffb8
2024-08-12 12:06:29 -07:00
Taylor Santiago
60fa202bb5 Merge "Use consistent directory for source and output inside the build sandbox for ABFS." into main 2024-07-31 17:55:41 +00:00
Taylor Santiago
ca30e08b9d Use consistent directory for source and output inside the build sandbox
for ABFS.

A consistent source directory addresses problematic full path inputs and
outputs, which can manifest as cache misses or build failures after an ABFS
patch application.

Bug: 347704053
Change-Id: Ib2297bb0b904b1b490e83c22fd468eee928e53fe
2024-07-30 13:52:31 -07:00
Yaowen Mei
d4da266b35 Fix RBE tmp dir
The RBE tmp dir holds all the re-client logs and socket files.

Currently, it is being created as the `out/soong/.temp/` dir, which is auto deleted by Soong. This is causing problems because while a build is running, user cannot open a second terminal and run `lunch`.

This CL fixes the issue by creating the RBE tmp dir in the `out/soong/rbe/` directory. The next RBE build will clean this folder.


Bug: 349664018
Change-Id: Ib238e48f5b19755b016263b45f02b1be3e8efb35
2024-07-30 14:21:04 +00:00
Christopher Ferris
cdd872a5be Merge "Update tests to expect RBE not supported on linux." into main 2024-07-26 17:58:26 +00:00
Wei Li
9f13555225 Merge "Generate SBOM of products in Soong." into main 2024-07-26 00:45:43 +00:00
Christopher Ferris
1db36e3888 Update tests to expect RBE not supported on linux.
Bug: 354280127

Test: Passes on linux
Test: Treehugger for the rest.
Change-Id: Ic1546a153e7e761a2132409743650cbf0a0b1ba6
2024-07-25 16:52:31 -07:00
Taylor Santiago
dd65d3e672 Merge "Add ABFS (Android Build Filesystem) configuration option to Soong" into main 2024-07-25 21:04:06 +00:00
Wei Li
b85a178b07 Generate SBOM of products in Soong.
Bug: 324465531
Test: CIs
Test: m soong-sbom
Change-Id: If76776851d49282829a79bfb1c33f05b8f57de31
2024-07-24 18:23:13 -07:00
Treehugger Robot
c7eb3bdae7 Merge "Highlight build failures in soong output" into main 2024-07-18 20:13:15 +00:00
Colin Cross
ea1b3b78b8 Merge changes from topic "elide_empty_variants" into main
* changes:
  Convert sdk mutator to TransitionMutator
  Convert link mutator to TransitionMutator
  Convert version mutator to TransitionMutator
  Convert python_version mutator to TransitionMutator
  Convert rust_libraries and rust_stdlinkage mutators to TransitionMutators
2024-07-18 17:37:38 +00:00
Treehugger Robot
865b877027 Merge "Generate .build-id directory tree after every build." into main 2024-07-17 23:13:24 +00:00
Colin Cross
8a49a3dd76 Convert rust_libraries and rust_stdlinkage mutators to TransitionMutators
Replace rust.LibraryMutator and rust.LibstdMutator with
TransitionMutators.

Bug: 319288033
Flag: EXEMPT refactor
Test: all soong tests pass
Test: no change to build.ninja
Change-Id: Ia24a582119d39889279d7b93bac9259685153619
2024-07-17 15:50:36 -07:00
Taylor Santiago
3c16e61c57 Add ABFS (Android Build Filesystem) configuration option to Soong
Change-Id: Ibbe81004adc67b1212d7e619a35f7425bfee0d72
2024-07-17 14:50:25 -07:00
cherokeeMeta
f7119b57ac Highlight build failures in soong output
Summary: When build failures occur, it can be hard to find where the error message begins. We now highlight "FAILURE" in red to make it easier to see what exactly has failed in the output.

Test: execute automated tests with m nothing --no-skip-soong-tests, can also manually test with a build failure and observe red text in the output

Change-Id: Iad3b94cc1c385f0afdebdf12c44843db04ff2bdc
Signed-off-by: Cherokee Toole <cherokee@meta.com>
2024-07-16 18:18:33 +00:00
Cole Faust
bee030d36b Allow n2 as a replacement for ninja in builds
If `SOONG_USE_N2=true` is set in the environment, then n2 is used in
place of ninja.  Some ninja features are not available in n2 at this
time, but this enables bringup efforts to happen in parallel.

Bug: 352368206
Test: manual
Change-Id: I8455cb24eb640a4651782ee76e48a7d3a9932b93
2024-07-10 12:35:32 -07:00
Nathan Egge
978c934dd9 Add LEX to the list of env variables to unset.
Some linux environments export LEX and this variable must be unset to
 ensure Soong and Make LEX values match.

Change-Id: I8ff2b62a49e0108575ff88b1a88a83d650fc3768
2024-07-08 20:53:46 +00:00
Spandan Das
dd274aba5f Merge "Revert^2 "Enforce that output files are created in primary ninja execution"" into main 2024-07-01 23:28:56 +00:00
Spandan Das
28a6f19866 Revert^2 "Enforce that output files are created in primary ninja execution"
This reverts commit fddc3dc8d3.

Reason for revert: https://r.android.com/3148741 makes this compatible with art gtests

Change-Id: Ia85c60fdcfa3142fc464a35476170c88c172989c
2024-07-01 21:00:25 +00:00
Treehugger Robot
38a97b2c47 Merge "Revert "Enforce that output files are created in primary ninja execution"" into main 2024-06-26 20:25:07 +00:00
Santiago Aboy Solanes
fddc3dc8d3 Revert "Enforce that output files are created in primary ninja execution"
This reverts commit 8d8253f525.

Reason for revert: Breaks ART gtests e.g. https://android-build.corp.google.com/build_explorer/build_details/12016159/art-gtest/

Change-Id: Ie8e1a28c7d0a21ee6727954127a6ab00ed00e781
2024-06-26 14:52:52 +00:00
Treehugger Robot
e8d5c8a5b2 Merge "Enforce that output files are created in primary ninja execution" into main 2024-06-26 02:18:55 +00:00
Treehugger Robot
51fc67bd20 Merge "Cleanup more bazel code" into main 2024-06-25 19:24:12 +00:00
Cole Faust
487b144fe0 Cleanup more bazel code
Bug: 315353489
Test: m nothing --no-skip-soong-tests
Change-Id: I6c6b4fcd559f4a45b1a96c70cbb5d6d1a615ccff
2024-06-25 11:02:38 -07:00
Joe Onorato
c870a75fe1 Merge "RBE is only supported when the host is linux, so don't try on others." into main 2024-06-25 17:58:35 +00:00
Joe Onorato
86f50e752e RBE is only supported when the host is linux, so don't try on others.
Bug: 343589123
Test: m and verify
Change-Id: I872d36a40c980f58c5de2598c3fb5dcfcadabf4f
2024-06-24 14:28:25 -07:00
Spandan Das
8d8253f525 Enforce that output files are created in primary ninja execution
Missing output files in ninja execution will be treated as errors.
Products can bypass this using the newly introduced
BUILD_BROKEN_MISSING_OUTPUTS flag.

Test: m nothing # verified missingoutfile appears in out/soong.log
Test: checkbuild passes on presubmits

Change-Id: I4eebcd08aa57fc6ccf1688c32e0d1fe06d66ab9a
2024-06-24 19:26:45 +00:00
Yu Liu
88f4e331cc Merge "Experimental code to support build action caching." into main 2024-06-19 01:57:12 +00:00
Yu Liu
fa29764f9f Experimental code to support build action caching.
Bug: 335718784
Test: build locally
Change-Id: Icc1f1fb15f9fe305e95dd51e2e7aff1e9cbf340c
2024-06-19 00:09:32 +00:00
Inseob Kim
58c802f8e3 Add product_config module
product_config module will export productVariables struct (defined in
android/variable.go). Other modules can depend on product_Config module
to read product variables, without needing to update Soong.

Bug: 346214955
Test: build and see output of product_config module
Change-Id: I06e1d2716dc2a8b90984bb46c5fc901cd1ca5020
2024-06-18 09:56:45 +09:00
Treehugger Robot
1571ad9605 Merge "Add following directories to Android.mk denylist to block new Android.mk files." into main 2024-06-13 23:43:25 +00:00
Cole Faust
1ae7b774eb Remove timestamp from local build number
The build_info.prop file was converted to soong, which then caused it to
have a stale build number due to not adding the build number file as a
dependency to avoid rebuilds. I added it as a dependency to fix the
staleness, but it always rebuilds locally now.

I think the way it worked before, it would get stale build numbers,
except that it was cleared with installclean which was always run on CI.
Now that it's a soong module and generated in out/soong/.intermediates
it's not cleared by installclean. I could make a system to register
files that should be installclean'd from a soong module, but I'd
eventually like to eliminate the need to run installclean entirely.

So as an alternative, just make the build number not change every build
when doing local builds, by removing the timestamp from it.

Bug: 346757289
Test: m repeatedly and observe no rebuilds the second time (of the system image, the build flags infrastructure seems to do some small rebuilding)
Change-Id: I0207feb739523dde3e89d1e5c4822865f641c313
2024-06-13 10:27:20 -07:00
Wei Li
8778d26c9d Add following directories to Android.mk denylist to block new Android.mk files.
bionic/
development/
device/sample/
prebuilts/
sdk/
test/
trusty/

Bug: 318428689
Change-Id: I327bf354df064a5249edc3e9c846199cde0ce00b
Test: CIs
2024-06-12 21:13:24 +00:00
Spandan Das
02f1e342cd Merge "Drop PRODUCT_INCLUDE_TAGS from banner vars" into main 2024-06-07 16:01:21 +00:00
Treehugger Robot
620b0c46d9 Merge "Regard out/soong/release-config as a source directory" into main 2024-06-07 05:12:31 +00:00