Commit Graph

22 Commits

Author SHA1 Message Date
Chris Gross
fabf50a577 Add merging of apexkeys and apkcerts metadata
Bug: 131437873
Test: Built system-only and vendor builds for merge. Compared
resulting apkcertx and apexkeys text files to that of a monolithic
build,
Test: Created colliding entries in both apexkeys and apkcerts text
files and ensure the script exited with an appropriate error message.
Test: Created unit tests to cover both non-colliding and colliding
entries
Change-Id: I6e42ce682ffa9059344e8cd63ba3a720c1f93452
2019-05-20 18:35:22 +00:00
Daniel Norman
714bd12861 Fixes typo when calling process_dynamic_partitions_info_txt.
Test: Running merge_target_files.py
Change-Id: Ieddfbfe524824844fc12cc10347f667332149f7c
2019-05-08 16:20:02 -07:00
Daniel Norman
a61cde0747 Begins merging META/dynamic_partitions_info.txt.
This file is used by OTA generation so it needs to appear in mixed
builds with the combined content from the system and other versions of
the file.

Test: python -m unittest test_merge_target_files
Test: Running merge_target_files on a dynamic-partition-enabled build
and observing the resulting target files.
Bug: 131889742

Change-Id: I4ddbebc087e430f6307d0bd5461121a374e58ea4
2019-05-07 11:53:58 -07:00
Bill Peckham
736b223d5f Use file_contexts.bin from other build to regenerate images.
The old process_file_contexts_bin function did not properly generate a usable
file_contexts.bin to regenerate images, so instead use the file_contexts.bin
from the other partial target files package. When combining any one of several
other partial target files packages with a single system partial target files
package, this file will properly apply contexts as long as the same source is
used for the system partial target files.

Test: Verify that file contexts are properlty applied to vendor image.
Bug: 131584454
Change-Id: I16f8cc3b7f2eb7f09746f0ddcb2c1daf3fd19da6
2019-05-06 23:23:23 +00:00
Bill Peckham
540d91ae3f Fix missing import.
Test: Verify that script runs properly now that import is present.
Change-Id: I255bc6ca2ba1f2cc387001c6f4bf9690a90eb47c
2019-04-25 14:18:16 -07:00
Bill Peckham
9662cfb234 Sort content of merged target files package.
By sorting the content of the final output merged target files package, the
merged target files package is more like the target files packages generated by
a build.

Test: Generate merged target files package, verify that content is sorted.
Change-Id: Ic0c198630ebd7692a3f3f9663d85e4b45229175c
2019-04-24 17:59:01 -07:00
Daniel Norman
b8a2f9df51 Adds support to create img from target files directory instead of zip.
This is used by merge_target_files to prevent an unnecessary unzip and
copy.

Test: Ran merge_target_files.py and booted using the img.zip.
Change-Id: I6fe0dd025b30b3f4965c9b22fb6943019bf5899b
2019-04-24 20:53:22 +00:00
Daniel Norman
1bd2a1d946 Adds --output-img flag to enable building the IMG package.
This simplifies the use case for mixed build users. Instead of having to
remember to call img_from_target_files.py after this script, they can
use this flag to automatically create the IMG package.

Also includes an update to super_empty.img logic. The super_empty.img is
now always created for dynamic-partition builds. The flag now only
controls copying the super_empty.img to a user-provided location.

Bug: 129976345
Test: Ran merge_target_files.py using --output-img and
--output-super-empty and inspected the resulting img zip and
super_empty.img.

Change-Id: Ida602942bb7a6b4b94f4e225640af9104fc9360c
2019-04-18 16:37:29 -07:00
Daniel Norman
e5b134aa02 Runs python formatter and linter over merge_target_files.
Test: python -m unittest test_merge_target_files
Change-Id: I95c8611439a019cd21ba16a93327f35e2035a84d
2019-04-17 14:54:06 -07:00
Daniel Norman
3b64ce1437 Adds --output-ota flag to enable building the OTA package.
This simplifies the use case for mixed build users. Instead of having to
remember to call ota_from_target_files.py after this script, they can
use this flag to automatically create the OTA package.

Bug: 129976345
Test: Ran merge_target_files.py using --output-ota and inspected the
resulting zip.

Change-Id: Icc95943c24b8f83b3221e845a7d69a34c1edb4fc
2019-04-17 10:28:57 -07:00
Daniel Norman
f031825560 Adds --output-super-empty flag to enable building super_empty.img.
Any mixed build that uses dynamic partitions will require a
super_empty.img image. This image must be created from the merged
misc_info.txt file, so adding this functionality here simplifies
the creation of this image for users (versus having to call
build_super_image.py manually after calling merge_target_files.py).

Bug: 129976345
Test: Ran merge_target_files.py on a dynamic partition enabled build
using the new --output-super-empty flag.

Change-Id: I73901f363d73c9fae1af1579faa2a908369dbbec
2019-04-17 10:22:11 -07:00
Daniel Norman
fdb38817ea Adds output-dir and output-item-list for copying only certain files.
This provides the ability to run merge_target_files without the end goal
of a target files zip. This is useful for users that only want the IMAGES
folder, for example.

Bug: 130304869
Test: python -m unittest test_merge_target_files
Change-Id: If0412b8e1eb85fe09d7b689fd7f56ce84067faea
2019-04-17 10:17:50 -07:00
Bill Peckham
364c1ccaa5 Add support merging system and other for the non-A/B configuration.
For non-A/B, you must supply the --system-item-list, --other-item-list, and
--system-misc-info-keys parameters approrpriate for merging two partial target
files. Additionally, you must supply the --rebuild-recovery option to correctly
generate the filesystem config and file contexts. With all of these parameters,
the script will generate a merged target files containing the correct recovery
files.

Also fix the --keep-tmp option to be consistent with the other options.

Bug: 122813742
Test: verify that merged target image boots and can perform OTA.
Change-Id: I5a942ac0cd9924fec419a686794a2340304594c8
2019-04-01 14:55:33 -07:00
Daniel Norman
19b9fe9fdf Merges dynamic partition misc_info.txt keys from both system and other.
To build a complete list of the dynamic partitions and partitions
groups, we need to merge the contribution from the system and other
target files.

Bug: 127687287
Test: Running merge_target_files.py and observing partition lists are
merged as expected.
Change-Id: I5bb9bd0e3179d48c9bfacdb3aca8253158f61cf6
2019-03-22 14:46:01 -07:00
Daniel Norman
a4ab1d8490 Merge "Validates the config lists used by the script." 2019-03-19 23:56:25 +00:00
Daniel Norman
e596452f03 Validates the config lists used by the script.
This validation is to help ensure that any usage of custom merge config
files does not accidentally exclude any item that has been added to the
default config lists.

Bug: 124197349
Test: Run merge_target_files with custom merge config files.
Change-Id: I34c51cb75212368146a2944d37621f311060d24d
2019-03-19 17:47:49 +00:00
Daniel Norman
a4911da57d Adds optional --rebuild_recovery flag to merge_target_files.
Also makes AddSystem check that an output_zip exists before attempting
to add the recovery patch to the output zip.

Bug: 128838154
Test: Running merge_target_files with --rebuild_recovery and verifying
it passes --rebuild_recovery to add_img_to_target_files.
Change-Id: I19347b2c0dabf29b7196045b18551b5d0687df2c
2019-03-18 12:29:31 -07:00
Daniel Norman
2c99c5bbd2 Adds support for optional config files to configure the merge.
Bug: 124467065
Test: Running `python merge_target_files.py` using the three new flags
and observing that their contents are passed to the merge_target_files() function.
Change-Id: I4de46f041f5ae8bc8be2730313ce873a952bf78e
2019-03-12 16:51:18 +00:00
Bill Peckham
8ff3fbdd08 Extract common.UnzipToDir, invoke that from merge_target_files.py
This change adds another utility function to common.py: UnzipToDir, which is
generally useful. Refactor merge_target_files.py to use it, and also refactor
other uses in common.py to use it.

Test: ota_from_target_files.py, validate_target_files.py, test_common.py
Bug: 124464492
Change-Id: Ia571070bceb7d3c8002304836bdf688485bf0dd9
2019-02-22 10:57:43 -08:00
Bill Peckham
889b0c6b09 Convert common.RunAndWait method to raise an exception on subprocess failure.
Then refactor the code in merge_target_files.py to adapt to this semantic
change. This makes the code more consistent with existing releasetools code,
and it's easier to follow.

Test: Failure cases (verify exception), success cases (merged target generated)
Bug: 124521133
Change-Id: I56f04e360d8ff8ffcd6245359cdeb79f4565a9c4
2019-02-21 19:02:07 -08:00
Bill Peckham
f753e15791 Include merge_target_files.py deps in otatools.zip, use common argv processing.
The merge_target_files.py script needs fc_sort and sefcontext_compile, so
include these tools into otatools.zip via core/Makefile.

Modify tools/releasetools/merge_target_files.py to use the otatools common argv
processing to take advantage of the '--path' option so that we add point the
'--path' option to an extracted otatools.zip package to gain access to fc_sort,
sefcontext_compile, and soong_zip (previously included in otatools.zip).

Bug: 123600124
Test: extract otatools.zip, use --path option to point to it, verify result
Change-Id: I7d84525981b8741c6bdbcac9984256920fc7f417
2019-02-21 00:50:54 +00:00
Bill Peckham
e9eb5f96e0 Adding initial merge_target_files.py script to merge two target files packages.
This script takes as input two partial target files (one contains system bits,
and the other contains non-system, or other, bits). The script merges the
contents of the two partial target files packages to produce a complete target
files package.

Bug: 123430711
Test: Build two partial target files, merge, compare with full target files.
Test: Validate merged target files via validate_target_files.py.
Change-Id: Ic24acf43b86fc703fb4c970688b006291a1861f8
2019-02-15 13:25:16 -08:00