Globs can fail when a directory is replaced with a file:
Error checking globs: stat foo: not a directory
Manually check for syscall.ENOTDIR, which is not covered by
fs.ErrNotExist.
Ignore-AOSP-First: "Landing internal first for critical ap4a move"
Bug: 369548825
Test: manual
Flag: EXEMPT bugfix
(cherry picked from https://googleplex-android-review.googlesource.com/q/commit:c3cd8e3f3c93773f0051608b94a0358554a95c98)
Merged-In: I35f65725b70fc0abb22aff46298d053d98d03b00
Change-Id: I35f65725b70fc0abb22aff46298d053d98d03b00
This permits easier testing and comparison between different ninja
implementations.
Bug: 362926979
Bug: 318434287
Test: manual
Change-Id: Ic058338d0a2fd8c988794a0781dcc93c0f4c03e3
This CL fix the bug that shouldCleanupRBELogsDir() never return true.
The way how shouldCleanupRBELogsDir() use to work is it will return true only if `RBE_proxy_log_dir` flag is not set.
But CI build always got this flag from gcl file, and developer build always got this flag set from ui/build/rbe.go (http://shortn/_K604iWNYkd). So there is never a case this shouldCleanupRBELogsDir() return true. That is to say, we never clean up
RBELogsDir by ourselves.
The reason why this is not a concern is that soong will automatically delete the out/soong/.temp folder every time when user run `m` or `lunch`, and the default RBELogsDir use to be `out/soong/.temp/rbe`. So soong is helping us to clean the logs before.
Previously, I merged this CL: https://r.android.com/3192211 to support running `lunch` in two terminals. In that CL, I moved the auto created RBELogsDir one level up to `out/soong/rbe`. This is causing a problem because RBELogsDir never get cleaned.
This CL will fix the shouldCleanupRBELogsDir() method, after this merged in, 1) if a user didn't set the `RBE_proxy_log_dir` flag, the logs in `out/soong/rbe` will be cleaned each time when running `m`, but `lunch` will not touch `out/soong/rbe`, so we still support running `lunch` in different terminals. 2) If user set `RBE_proxy_log_dir` flag to anything rather than `out/song/rbe`, the directory will not be cleaned by running `m`; 3) we have updated our doc everywhere that `out/soong/rbe` is re-client's default log dir, so I think it should be expected if user set `RBE_proxy_log_dir=out/soong/rbe`, then the logs will be deleted when next time `m` invoked, same as if they set `RBE_proxy_log_dir=out/soong/.temp/rbe` before.
Test: abtd run is green: http://shortn/_KqTvQelstP, local build with multiply lunch is not affected, and running m after a build can clean the out/soong/rbe directory.
Bug: 362798954
Change-Id: I38a7ad650fc59ad06716c5be7de6ecc61ead8eef
If you remove a folder that's part of a glob, soong will error out
currently. Instead treat it like an out-of-date dependency.
Bug: 364749114
Test: m nothing, rm glob folder, m nothing
Change-Id: I86ee755a0815b79192133223f8ef3ecd90669db1
This reverts commit b1fbc79206.
Removing the build broken flag to progress the python3 migration. It's been here for over a year and I haven't heard of any users.
Bug: 203436762
Test: Presubmits
Change-Id: Icfd2d25986abda934c60fd868ed5525dc384fbf3
Previously, globs worked by having soong_build rewrite a ninja file
that ran the globs, and then dependended on the results of that ninja
file. soong_build also pre-filled their outputs so that it wouldn't
be immediately rerun on the 2nd build.
However, the pre-filling of outputs worked for ninja, because although
it updated their timestamps, the soong ninja file was then touched
by soong_build after that, so the soong_build ninja file was newer
and ninja wouldn't rerun soong. But N2 reruns actions if their inputs'
mtimes change in any way, not just if they're newer. Similarly,
hashed-based ninja implementations could not enforce an order on
file contents, so they would have the same problem.
To fix this, lift the glob checking out of ninja and into soong_ui.
Soong_build will output a globs report file every time it's run, and
every time soong_ui is run it will check the globs file, and if any
globs change, update an input to soong_build. soong_ui is essentially
doing what was done in ninja with bpglob actions before.
Bug: 364749114
Test: m nothing, m nothing again doesn't reanalyze, create a new file under a glob directory, m nothing again reanalyzes
Change-Id: I0dbc5ec58c89b869b59cd0602b82215c4972d799
The zeros were kept in place of the timestamp in case people were
parsing it. Let's try to remove the zeros now and see what breaks.
Test: Presubmits
Change-Id: Ic52168f6be05a74323d8e47935de5c6a5cf15516
The soong primary builder action lists all the Android.bp files
as dependencies via a depfile. The top-level one doesn't need to be
listed as an input explicitly. I think it was there from the early
days of soong when soong started at the top-level Android.bp file and
traversed downwards through references via the subdirs or
optional_subdirs variables.
Bug: 364749114
Test: Presubmits
Change-Id: I72eabe4d4179696bfa258c8c4c56351a132e0870
This reverts commit 0af8ea14fc.
Reason for revert: <Droidmonitor created revert due to b/363848580. Will be verifying through ABTD before submission.>
Change-Id: Iafba28897ad27df67ef8ae8904454196c482216d
Revert submission 3235973-revert-3207217-emma-flip-time-saver-BYWASVIMTD
Reason for revert: Rolling forward with a fix
Adding a coverage suffix for product_config.go too:
FAILED: ninja: 'out/soong/soong.sdv_core_cf.extra.variables', needed by 'out/soong/.intermediates/build/soong/product_config/product_config.json', missing and no known rule to make it
03:42:23 ninja failed with: exit status 1
Reverted changes: /q/submissionid:3235973-revert-3207217-emma-flip-time-saver-BYWASVIMTD
Change-Id: I74d0e3fda09b1367f3dbb939d4f47c4d09b3d51f
We're seeing bugs on the build server only, and previous similar bugs
were caused by issues in the older version of glibc the build server
uses, so try using the musl build of n2 to see if it resolves the
issues.
Bug: 318434287
Test: Presubmits
Change-Id: I416cd77f4dfb4ca04f6dd5b36fa73585a2904b55
This CL uses the environment variable, EMMA_INSTRUMENT to determine whether or not coverage is enabled. If coverage is enabled, it adds a bunch of suffixes to the files generated in out/soong/ folders.
This change reduces the build time by avoiding ninja regeneration when users simply switches from atest to atest --experimental-coverage, or vice versa, and no Android.bp changes are made.
Bug: 331444846
Test: Locally run m libc, and then EMMA_INSTRUMENT m libc twice, no ninja regenerations required. Compared the hash of out/target/product/vsoc_x86_64/ between the before-change repo and after-change repo. Only diff is the after-change directory contains an extra file `out/target/product/vsoc_x86_64/.installable_files` while the original directory has only `out/target/product/vsoc_x86_64/.installable_files.previous`, both files have the same hash.
Change-Id: I972aec20a9d97b223cc833ee245016ac4b0f09a2
OUT_DIR configuration.
Mounting '/' directly as read-only results in failed bindmounts at /src
if that directory does not already exist due to failure to mkdir.
Change-Id: I7f11c45e82374d142a86fcb2f03e53ddacffffb8
for ABFS.
A consistent source directory addresses problematic full path inputs and
outputs, which can manifest as cache misses or build failures after an ABFS
patch application.
Bug: 347704053
Change-Id: Ib2297bb0b904b1b490e83c22fd468eee928e53fe
The RBE tmp dir holds all the re-client logs and socket files.
Currently, it is being created as the `out/soong/.temp/` dir, which is auto deleted by Soong. This is causing problems because while a build is running, user cannot open a second terminal and run `lunch`.
This CL fixes the issue by creating the RBE tmp dir in the `out/soong/rbe/` directory. The next RBE build will clean this folder.
Bug: 349664018
Change-Id: Ib238e48f5b19755b016263b45f02b1be3e8efb35
* changes:
Convert sdk mutator to TransitionMutator
Convert link mutator to TransitionMutator
Convert version mutator to TransitionMutator
Convert python_version mutator to TransitionMutator
Convert rust_libraries and rust_stdlinkage mutators to TransitionMutators
Replace rust.LibraryMutator and rust.LibstdMutator with
TransitionMutators.
Bug: 319288033
Flag: EXEMPT refactor
Test: all soong tests pass
Test: no change to build.ninja
Change-Id: Ia24a582119d39889279d7b93bac9259685153619
Summary: When build failures occur, it can be hard to find where the error message begins. We now highlight "FAILURE" in red to make it easier to see what exactly has failed in the output.
Test: execute automated tests with m nothing --no-skip-soong-tests, can also manually test with a build failure and observe red text in the output
Change-Id: Iad3b94cc1c385f0afdebdf12c44843db04ff2bdc
Signed-off-by: Cherokee Toole <cherokee@meta.com>
If `SOONG_USE_N2=true` is set in the environment, then n2 is used in
place of ninja. Some ninja features are not available in n2 at this
time, but this enables bringup efforts to happen in parallel.
Bug: 352368206
Test: manual
Change-Id: I8455cb24eb640a4651782ee76e48a7d3a9932b93
Some linux environments export LEX and this variable must be unset to
ensure Soong and Make LEX values match.
Change-Id: I8ff2b62a49e0108575ff88b1a88a83d650fc3768
This reverts commit fddc3dc8d3.
Reason for revert: https://r.android.com/3148741 makes this compatible with art gtests
Change-Id: Ia85c60fdcfa3142fc464a35476170c88c172989c
