It's a common mistake in make to have leading/trailing whitespace
on values, so strip it out before assigning to variables.
Bug: 282824346
Test: Presubmits
Change-Id: If4d3b86795c05ba32006af3af8031ca083a4c54b
- rblf_cli and rblf_env
- -c and -f
This is in preparation for making rbcrun able to function as a more
general purpose starlark interpreter.
Bug: 280685526
Test: go test, ./out/rbc ./build/make/tests/run.rbc, ./build/bazel/ci/rbc_dashboard.py --quick aosp_arm64
Change-Id: Ifff9ce7b4369422f39c5003bb85a168c78bde7cf
In make, soong_config_set uses an $(eval) to set it's value, expanding
the value of the soong config variable _before_ evaluating the value.
Because of this, make will strip trailing whitespace like it does on
regular assignments.
Make rbcrun match this behavior.
Test: ./out/rbcrun ./build/make/tests/run.rbc
Change-Id: I907e85cdf50f6fac54331c0d1044f0d53bec22ed
words() attempts to join all the elements of a list
and then resplit them to more closely match make.
But sometimes, like when calling words() on a product
variable, not all of the elements are strings. In
that case, just return the list unchanged.
Bug: 267407943
Test: ./out/rbcrun ./build/make/tests/run.rbc
Change-Id: I738d0c86c8935f446807cc79623f796e8cae3c01
Inherit references (@inherit in make, 1-tuples in starlark)
usually appear in list product variables, where they're
evaluated before printing. But sometimes they can be copied
into a regular global variable, in which case they won't
be evaluated before printing. The 1-tuples were failing to
print, so adapt them to their make representation before
printing.
Fixes: 264554449
Test: ./out/rbcrun ./build/make/tests/run.rbc
Change-Id: I5ac6eb996c25ee5e77aa26ed5c4b6b6cc31819d1
Make's clear-var-list causes the variables to exist as empty strings.
Mimic that functionality in starlark for the variable diff and ?=.
Bug: 262303006
Test: ./out/rbcrun ./build/make/tests/run.rbc
Change-Id: I5f9c8cf342d875b1022c5c74906e59fa68fcd6c9
This is what mk2rbc assumes, and previously that
assumption was incorrect when passing a list to
mkstrip.
Bug: 226974242
Test: Presubmits
Change-Id: I7b1596662df7c966404aab2a99d7c2e53d917bd6
Kati's $(wildcard) results are guaranteed to be sorted,
but go's filepath.Glob() is not.
Remove the file existence functions because they can
just be replaced with wildcards.
Bug: 226974242
Test: go test
Change-Id: I02fb6292b932bc28cd856ec3c7cb9ed9e96ca630
Stdout gets redirected to a makefile that is
ingested by the rest of the build system.
The trace outputs were being mixed up with
the rest of the output, making them hard to
read. In addition, some of the trace output
wasn't commented out, so it was defining
make rules.
Fixes: 230495885
Test: Manually
Change-Id: I02f9038cd89598fc1c1561bf4bf2997ac090d9e9
realpath doesn't return a path if the file doesn't exist,
but $(abspath) in make does.
Bug: 229132189
Test: ./out/rbcrun ./build/make/tests/run.rbc
Change-Id: Ief7f634024cc52a9e8c5e478666b15512512f0d8
We pass the filename without the extension to
rblf.inherit(). Removing the extension changes the
sort order when one file's name is a prefix of another:
```
>>> sorted(["base", "base-secondary"])
['base', 'base-secondary']
>>> sorted(["base.mk", "base-secondary.mk"])
['base-secondary.mk', 'base.mk']
```
Correct the sort order so that global variables get
their correct ordering.
Bug: 229132189
Test: ./out/rbcrun ./build/make/tests/run.rbc
Change-Id: I22367eb49b33956b71ac1b966fe78c1308b94257
This is to minimize the differences between the make
and starlark product configs for the quick ci test.
Bug: 229132189
Test: Manually
Change-Id: I3286ba64e9d8fc9d1b169a7ed18f5a565a117a8d
This is something that didn't show up as a regression
when testing ninja files, but does show up when testing
make variables.
Bug: 229132189
Test: New --quick mode in rbc_dashboard.py
Change-Id: I6c1219cdc5ee1d8b3cc39e8f944e99fc380d4668
config.mk represents essentially the entire product/board
configuration. In order to develop a "quick" rbc regression
test, dump all the make variables at the end of config.mk.
We can then compare these variable dumps instead of ninja
files, because the ninja files take much longer to generate.
Bug: 229132189
Test: Manually
Change-Id: I4e8371be446b7e511aba22dff58530a6d9ff072f
When an intermediate product is evaluated, it needs to
act as if it's the only product being evaulated. However,
currently, if it inherited a makefile that was also being
inherited by the overall top level product via a different
path, it would not get the values from that makefile.
Copy the configs dictionary before evaluating each product
that needs artifact path requirements, and create seperate
postfix orders for all of them that don't contain any products
that they don't inherit from.
Bug: 221312707
Test: ./out/rbcrun ./build/make/tests/run.rbc
Change-Id: I235ad78d587a2e315ba446b5e126d8f6d0fbbea7
Artifact path requirements requires setting some new variables,
and also dumping the state of the product config variables
after evaluating partial products that set artifact path requirements.
Bug: 188079133
Test: Removed the code that disabled artifacts path requirements from the rbc dashboard script,
and verified that it was still green for aosp_arm64
Change-Id: I228e35285d788f4c83aa695c0f28b7c7db02544c
List variables needed to be percolated in the order of inherit() calls.
children.keys() was in that order, due to starlark dictionaries being
iterable in the order of insertion, but the previous cl broke that
behavior by sorting them. Instead, only sort the children for
single value variables.
Fixes: 226206409
Fixes: 228044099
Test: ./out/rbcrun ./build/make/tests/run.rbc and testing aosp_arm64
Change-Id: I5b91514e87b158b615e4d4ec7868fccb0248379b
This reverts commit 670c587c09.
Reason for revert: Breaks RBC product config for more products than it fixes: b/228044099
Change-Id: I614663311f67b3631a0cee427453c550fc426c8b
To increase its feature parity with our notdir
implementation and gnu make.
Bug: 227245326
Test: Manually
Change-Id: Ied975f934d7af963e0099054c416f0d364aec8f9
The children to inherit from were not sorted. This
wasn't a problem for list variables, because all those
inheritances get merged together based on the @inherit/
values in the list variables. But for single value variables
it made a difference.
Bug: 226206409
Test: ./out/rbcrun ./build/make/tests/run.rbc
Change-Id: Ib56bbb91a79fe8c7cb780c253f5bd8d6c0e87765
As a first step to making .rbc files compatible with bazel,
remove regex support since bazel doesn't have it.
Fixes: 227384703
Test: ./out/rbcrun ./build/make/tests/run.rbc
Change-Id: I8b946c20cc42897a47a5516a167732f4e16b6158
mk2rbc was already emitting conversions of the dir
function, but its implementation didn't exist.
Bug: 227245326
Test: Manually
Change-Id: I3348c022a0ff84e6d3a5be6b18e45c439f1db802
Single-value product variables resolve to the first
instance of that variable in prefix order. What we
have right now is taking the value from the current
makefile if it's set, but if it's not, we take it
from the last child, not the first.
Fixes: 226206409
Test: Manually
Change-Id: Id23f7c269ff9a352bf0b67cb57156b72449f978e
The make version, find-copy-subdir-files, has very
similiar functionality: it adds `| sed "s://:/:g"`
to its results.
Fixes: 223221122
Test: Manually
Change-Id: Iaac96f760d46304aed27a7d31bb4e8bd930c2c55
This is clearer and allows us to more easily
add new fields to the struct. One usecase for
more fields is the artifact path requirements.
Bug: 221312707
Test: Verified starlark product config still works for aosp_arm64
Change-Id: I4c4307e916f3b3fc8ca303b10eac3c7c73b76ac6
Starlark configuration output is a makefile containing variable
assignments. Inheritance trace is also written to stdout. Trace lines
need to be comments, otherwise stdout cannot be digested by Make/Ckati.
Test: manual
Change-Id: I3c1ad6554ef4356f9ceff1ff3279b2eac962e9ac
Passing variables via a makefile instead of
rblf_cli / rblf_env allows us to give them correct
types while converting the makefile to starlark,
as opposed to the variables always being strings
when given via rblf_cli / rblf_env.
This also allows us to remove some hand-converted
starlark code.
Bug: 201700692
Test: ./out/soong/rbcrun ./build/make/tests/run.rbc
Change-Id: I58c4f20b29171c14e5ae759beb26a849426f6961
dict(**h[0]) was only copying the top level dictionary
object, but not any other dictionaries/lists inside of
it. This was causing us to not find all the changes to
variables the board configuration was making, and so
some variables were not included in the output.
Bug: 201700692
Test: ./build/bazel/ci/rbc_regression_test.sh mainline_system_x86_64-userdebug
Test: ./out/soong/rbcrun ./build/make/tests/run.rbc
Change-Id: I4467eead7e597f6f49119a8c4832126f39d646e7
The board config can actually change a few product config
variables that are not made read-only.
Bug: 201700692
Test: ./build/bazel/ci/rbc_regression_test.sh -b aosp_cf_x86_64_phone
Change-Id: Ib2ac04ac2c5a56d32bca51fc8189e44e180075ee
