There were a few issues with the output packaging process that were
found during testing of the general-tests optimization.
First and foremost is that the packaging commands were trying to be
created before the build ran, when the outputs don't exist. I've changed
the logic to just collect the methods themselves which will then be run
during build plan execution after the build has completed.
A few other smaller issues include fixing the path to the soong_zip
binary, incorrect execution of the soong dumpvars command, and not
building the shared libs zip.
Test: atest build_test_suites_test; atest optimized_targets_test
Bug: 358215235
Change-Id: I8a3f54738f8bb5d871aadf7423844076c38b54a6
Implement the output packaging step in the GeneralTestsOptimizer. This
step takes all the outputs built by the general-tests target (in the
case where it was optimized) and packages them into the necessary zips
generated by the target normally.
Test: atest build_test_suites_test; atest optimized_targets_test
Bug: 358215235
Change-Id: I5d27eef4e37137cc9b6e235f52f3856ba0b30460
Refactor package_outputs in the TestOptimizer so it just returns a list
of soong_zip commands to be run by build_test_suites.
Since we already have a tested implementation for running subprocesses
in build_test_suites.py there's no reason to reimplement it in
optimized_targets.py. Because any packaging will ultimately use
soong_zip to package its final outputs change the code to just do
whatever prep it needs to and return a list of soong_zip commands.
This way the code is simpler to test without requiring subprocesses and
no reimplementation of subprocess running code is necessary.
Test: atest build_test_suites_test; atest optimized_targets_test
Bug: 358215235
Change-Id: I3025aefeeb7186f537266a72d8422211ca9835ba
Implement functionality in GeneralTestsOptimizer to find what targets to
build.
This logic is fairly complex and involves checking for test
configs that download general-tests.zip. Then the configs are checked to
see if they're proper test mapping tests (if they use the
'test-mapping-test-group' option). If they are, then TEST_MAPPING
modules are scanned to see if the list of changed files would cause any
test mapping modules to run. The tests are then further filtered by
test-mapping-test-groups used in the test configs.
In case that a test uses general-tests.zip but does not specify
'test-mapping-test-group' then all bets are off and general-tests.zip
is built in its entirety.
package_outputs is still unimplemented so this will need to be
implemented before the optimization can be enabled.
Test: atest build_test_suites_test && atest optimized_targets_test
Bug: 358215235
Change-Id: I6a7eebfd1b06b380799292eb2019ac17c9af5367
Add a BuildContext class to simplify parsing the build context dict, and
fix the parsing of enabledBuildFeatures, which was being parsed as a
list of strings when it's actually a list of dicts like:
[{'name': '<feature_name>'}]
Test: atest build_test_suites_test
Bug: 361605425
Change-Id: I6424c444daf1582e92313c39f43207cb274aa78f
Refactor the OptimizedBuildTarget base class to more simply handle
enabling/disabling build optimizations based on flags. Now all
optimizers (aside from the no-op NullOptimizer will have their enabling
built into the base class, and also necessitate their get_build_targets
and package_outputs functions to be defined in order to not error out.
Add the GeneralTestsOptimizer class as well, unimplemented for now.
Test: atest build_test_suites_test
Bug: 358215235
Change-Id: Ide22c64fc238d754db8d8e76484947401e54e680
Add functionality in build_test_suites.py to not build targets if
their outputs are not used in the test configurations saved in the build
context. If none of the tests reference the targets' outputs they will
not be built at all.
Note that the corresponding flags will need to be enabled for these
optimizations to take place.
Test: atest optimized_targets_test
Bug: 348489774
Change-Id: I8f0ac90e75552ae80073f13229b026c7f23476a6
