And a few minor clean-ups to the styling.
Bug: 131631303
Test: python -m unittest test_merge_target_files
Test: python3 -m unittest test_merge_target_files
Test: Use `python merge_target_files` to merge two target_files zips.
Test: Use `python3 merge_target_files` to merge two target_files zips.
Change-Id: I8502dfb243408f658d022e8d5e5fbb60066e4ff0
Split merge_target_files function into several steps, so we can increate
readability and add conditional flows for other *SSI mixed build with less effort
Test: m -j & atest passed
Change-Id: I558f9dd5bca31b132a09cb36d9dfcd30c92efbc9
This CL changes the condition for building super_empty.img from
PRODUCT_BUILD_SUPER_PARTITION to PRODUCT_USE_DYNAMIC_PARTITIONS, as a
follow-up to the change in [1].
With the CL in [1], it skips building super.img and super_empty.img both
when turning off PRODUCT_BUILD_SUPER_PARTITION. However, the latter
should be mandatory whenever dynamic partitions is enabled. Because
fastboot relies on this file to properly flash dynamic partitions. Plus,
the cost for building super_empty.img is much lower than the one for
super.img.
As part of the change, it'll write group info into target_files when
building with PRODUCT_BUILD_SUPER_PARTITION == false. It's the work for
target_files merging script to determine the values to be picked up. The
current logic in merge_target_files.py always uses the one from vendor
target_files. This CL adds a testcase to ensure the behavior.
[1] https://android-review.googlesource.com/c/platform/build/+/928756
Bug: 135752763
Test: `m dist` with a target that sets PRODUCT_BUILD_SUPER_PARTITION to
false. Check the built artifacts contain super_empty.img. Verify
that the build can be flashed properly.
Change-Id: I277f087eab45663a6c3b33333d16e9e576c1c25c
There are some places defining same file open function and use
common.LoadDictionaryFromLines. This commit creates
LoadDictionaryFromFile to reduce some code redundancy.
Test: m -j & atest passed
Change-Id: I6a3fa48693095937f8c79ce6f3c110b6862a1967
Previously, setting PRODUCT_BUILD_SUPER_PARTITION to false for a partial
build (with PRODUCT_USE_DYNAMIC_PARTITIONS == true) would fail to
include necessary keys in misc_info.txt that are required when merging
two partial builds to create a dynamic-partition-enabled mixed build.
This change ensures these necessary keys are included even when
PRODUCT_BUILD_SUPER_PARTITION is false. Setting
PRODUCT_BUILD_SUPER_PARTITION to false causes partial builds to skip
building super.img and super_empty.img, instead relying on these images
to come from the final merged build.
Bug: 134764140
Test: Building & booting a dynamic-partition-enabled mixed build, and
inspecting partial builds' logs / out folder to ensure that
super.img/super_empty.img were not created.
Change-Id: I99431a9a342e9b0617510e250597f3024ef39322
merge_target_files.py step.
Bug: 134681035
Test: Built a merged build and ensured that the new timestamp was
visible in the log.
Change-Id: Ia6bbda48c7f57afdb6482253eaf0b3b0ea067468
Also fixes small nit from previous change to write_sorted_data().
Bug: 132788610
Test: python -m unittest test_merge_target_files
Test: Creating and booting a merged build.
Change-Id: I3dc43a4fe55b86b436dec08feb5d70096d38de36
This is to prevent a user from accidentally including files from the
wrong build. For example, adding any SYSTEM/ line to other_item_list
while keeping SYSTEM/* in system_item_list would cause the other build
to introduce an extra or changed file in the system image.
Bug: 132730710
Test: python -m unittest test_merge_target_files
Change-Id: Ic1178cdc9b991114f293ff3f2b4e6054e06647c6
This enables mixed builds to use the file_contexts.bin from the system
build when regenerating images that come from system target files, and
similarly for file_contexts.bin from the other build when regenerating
images from other target files.
In monolithic (non-mixed) builds all image-specific selinux_fc props
point to the same file_contexts.
Bug: 132108151
Test: Built and booted mixed build devices.
Change-Id: Id51ed6d96ea6337879f1ab21d47c93c67bc25312
Bug: 131437873
Test: Built system-only and vendor builds for merge. Compared
resulting apkcertx and apexkeys text files to that of a monolithic
build,
Test: Created colliding entries in both apexkeys and apkcerts text
files and ensure the script exited with an appropriate error message.
Test: Created unit tests to cover both non-colliding and colliding
entries
Change-Id: I6e42ce682ffa9059344e8cd63ba3a720c1f93452
This file is used by OTA generation so it needs to appear in mixed
builds with the combined content from the system and other versions of
the file.
Test: python -m unittest test_merge_target_files
Test: Running merge_target_files on a dynamic-partition-enabled build
and observing the resulting target files.
Bug: 131889742
Change-Id: I4ddbebc087e430f6307d0bd5461121a374e58ea4
The old process_file_contexts_bin function did not properly generate a usable
file_contexts.bin to regenerate images, so instead use the file_contexts.bin
from the other partial target files package. When combining any one of several
other partial target files packages with a single system partial target files
package, this file will properly apply contexts as long as the same source is
used for the system partial target files.
Test: Verify that file contexts are properlty applied to vendor image.
Bug: 131584454
Change-Id: I16f8cc3b7f2eb7f09746f0ddcb2c1daf3fd19da6
By sorting the content of the final output merged target files package, the
merged target files package is more like the target files packages generated by
a build.
Test: Generate merged target files package, verify that content is sorted.
Change-Id: Ic0c198630ebd7692a3f3f9663d85e4b45229175c
This is used by merge_target_files to prevent an unnecessary unzip and
copy.
Test: Ran merge_target_files.py and booted using the img.zip.
Change-Id: I6fe0dd025b30b3f4965c9b22fb6943019bf5899b
This simplifies the use case for mixed build users. Instead of having to
remember to call img_from_target_files.py after this script, they can
use this flag to automatically create the IMG package.
Also includes an update to super_empty.img logic. The super_empty.img is
now always created for dynamic-partition builds. The flag now only
controls copying the super_empty.img to a user-provided location.
Bug: 129976345
Test: Ran merge_target_files.py using --output-img and
--output-super-empty and inspected the resulting img zip and
super_empty.img.
Change-Id: Ida602942bb7a6b4b94f4e225640af9104fc9360c
This simplifies the use case for mixed build users. Instead of having to
remember to call ota_from_target_files.py after this script, they can
use this flag to automatically create the OTA package.
Bug: 129976345
Test: Ran merge_target_files.py using --output-ota and inspected the
resulting zip.
Change-Id: Icc95943c24b8f83b3221e845a7d69a34c1edb4fc
Any mixed build that uses dynamic partitions will require a
super_empty.img image. This image must be created from the merged
misc_info.txt file, so adding this functionality here simplifies
the creation of this image for users (versus having to call
build_super_image.py manually after calling merge_target_files.py).
Bug: 129976345
Test: Ran merge_target_files.py on a dynamic partition enabled build
using the new --output-super-empty flag.
Change-Id: I73901f363d73c9fae1af1579faa2a908369dbbec
This provides the ability to run merge_target_files without the end goal
of a target files zip. This is useful for users that only want the IMAGES
folder, for example.
Bug: 130304869
Test: python -m unittest test_merge_target_files
Change-Id: If0412b8e1eb85fe09d7b689fd7f56ce84067faea
For non-A/B, you must supply the --system-item-list, --other-item-list, and
--system-misc-info-keys parameters approrpriate for merging two partial target
files. Additionally, you must supply the --rebuild-recovery option to correctly
generate the filesystem config and file contexts. With all of these parameters,
the script will generate a merged target files containing the correct recovery
files.
Also fix the --keep-tmp option to be consistent with the other options.
Bug: 122813742
Test: verify that merged target image boots and can perform OTA.
Change-Id: I5a942ac0cd9924fec419a686794a2340304594c8
To build a complete list of the dynamic partitions and partitions
groups, we need to merge the contribution from the system and other
target files.
Bug: 127687287
Test: Running merge_target_files.py and observing partition lists are
merged as expected.
Change-Id: I5bb9bd0e3179d48c9bfacdb3aca8253158f61cf6
This validation is to help ensure that any usage of custom merge config
files does not accidentally exclude any item that has been added to the
default config lists.
Bug: 124197349
Test: Run merge_target_files with custom merge config files.
Change-Id: I34c51cb75212368146a2944d37621f311060d24d
Also makes AddSystem check that an output_zip exists before attempting
to add the recovery patch to the output zip.
Bug: 128838154
Test: Running merge_target_files with --rebuild_recovery and verifying
it passes --rebuild_recovery to add_img_to_target_files.
Change-Id: I19347b2c0dabf29b7196045b18551b5d0687df2c
Bug: 124467065
Test: Running `python merge_target_files.py` using the three new flags
and observing that their contents are passed to the merge_target_files() function.
Change-Id: I4de46f041f5ae8bc8be2730313ce873a952bf78e
This change adds another utility function to common.py: UnzipToDir, which is
generally useful. Refactor merge_target_files.py to use it, and also refactor
other uses in common.py to use it.
Test: ota_from_target_files.py, validate_target_files.py, test_common.py
Bug: 124464492
Change-Id: Ia571070bceb7d3c8002304836bdf688485bf0dd9
Then refactor the code in merge_target_files.py to adapt to this semantic
change. This makes the code more consistent with existing releasetools code,
and it's easier to follow.
Test: Failure cases (verify exception), success cases (merged target generated)
Bug: 124521133
Change-Id: I56f04e360d8ff8ffcd6245359cdeb79f4565a9c4
The merge_target_files.py script needs fc_sort and sefcontext_compile, so
include these tools into otatools.zip via core/Makefile.
Modify tools/releasetools/merge_target_files.py to use the otatools common argv
processing to take advantage of the '--path' option so that we add point the
'--path' option to an extracted otatools.zip package to gain access to fc_sort,
sefcontext_compile, and soong_zip (previously included in otatools.zip).
Bug: 123600124
Test: extract otatools.zip, use --path option to point to it, verify result
Change-Id: I7d84525981b8741c6bdbcac9984256920fc7f417
This script takes as input two partial target files (one contains system bits,
and the other contains non-system, or other, bits). The script merges the
contents of the two partial target files packages to produce a complete target
files package.
Bug: 123430711
Test: Build two partial target files, merge, compare with full target files.
Test: Validate merged target files via validate_target_files.py.
Change-Id: Ic24acf43b86fc703fb4c970688b006291a1861f8
