This verifies the init rc files in the merged result.
Bug: 163089173
Test: test_common.py
Test: Run merge_target_files.py to merge two target-files packages where
one has init_rc errors. Observe script failure.
Test: Run merge_target_files.py on two good target-files packages,
observe no failure.
Change-Id: I86c8e5a2bc07c2c1896ac40afd32bc1d055447ee
common.py is used by many python-based host tools. When a host tool
invokes another host tool, it's usually done with the name of the tool,
not the absolute path of the tool. For example, ["cmd", "arg"] instead
of ["out/soong/host/linux-x86/bin/cmd", "arg"].
Previously, the caller of the tool has to teach all the locations of the
sub-tools that the tool will internally use. But this isn't ideal;
1) It's against the abstraction. The existence of the sub tools and their
names are implementation detail of the containing tool. It's also
subject to change.
2) This isn't scalable. Sometimes a host tool invokes a large number of
host tools. Furthermore, the sub tools might invoke other sub-sub tools.
Then the location of the sub-sub-tools had to be known to the top-level
tool.
The idea here is to make use of the fact that
a) dependencies to the sub (and sub-sub) tools are already described in
the Android.bp and
b) the dependencies are guaranteed to be up-to-date and installed to the
host tool directory (out/soong/host/linux-x86/bin) by Soong.
Then by the time a host tool is invoked, all of its sub tools should
be found in the host tool directory. So, when "cmd" is about to be
invoked, common.py first tries to find it from the user-given paths. If
not found there, it falls back to search the directory where the current
tool is located at. Then finally falls back to the original name "cmd"
and expects it to be found under PATH.
Bug: 172414391
Test: m
Change-Id: Id7b44f9021be3bbf0631ddafe382ea3990f7ea74
This check is used when merging target files to ensure that a merged
build does not contain any APKs that share UID across builds.
Bug: 171431774
Test: test_common
Test: Use merge_target_files.py to merge two partial builds,
observe no failures for inputs without colliding APKs.
Test: Use merge_target_files.py to merge two partial builds,
observe failure for inputs that have an APK that shares a
UID across input partition groups.
Change-Id: I9dc57216882741ae46a99cfd7847f34702c75582
In aosp/1459042, init has updated it's logic to use
ro.build.version.release_or_codename for fingerprint calculation.
The ota script needs the same update for ota targeting to work
correctly.
As a best effort to support pre-S dev stage builds, use
ro.build.version.release for sdk version < 30.
Bug: 170968068
Bug: 158483506
Test: unit tests, generate an OTA for S build
Change-Id: I01ca8a3b7b8b58f94b10f90ed0d27e688a72b866
BOARD_EXCLUDE_KERNEL_FROM_RECOVERY_IMAGE removes kernel from the
recovery image.
Test: build recovery image and unpack_bootimg
Test: build target files and unpack_bootimg IMAGES/recovery.img
on device with GKI and dedicated recovery partition.
Bug: 156098440
Change-Id: I5f37d74ed954b26fe3dd778708b6ab9cfdc51f1e
These changes were necessary to begin merging a new cross-release target
whose vendor half is frozen.
- MergeDynamicPartitionInfoDicts
- Filters combined fields to remove duplicates
- Merges `super_block_devices` as well as other keys that were not
previously used by other targets consuming this tool.
- Introduces --allow-duplicate-apkapex-keys. This gives a warning rather
than fatal error on duplicate apk/apex keys. This flag is needed for
targets that cannot update a frozen half.
- (Formats merge_target_files.py)
Bug: 170683837
Test: Use merge_target_files to merge an S+R build, and boot.
Change-Id: Id5f787e730de8f8ef697e1f2f29ac6514221e58d
When opening an zip file through zipfile.ZipFile(), python2 by default
disables zip64 support. To support update files >4GB, we manually add
allowZip64 to override the setting.
Test: generate && serve an OTA
Change-Id: I9645e963ced830cc2d3a4b72bc63b9369a1cefe8
We recently modify the logic in install-recovery.sh to support
non-A/B updates on VAB launched devices. And the position of the
$(getprop ro.boot.slot_suffix) is misplaced for the recovery image.
For example:
"if ! applypatch --check EMMC$(getprop ro.boot.slot_suffix):/dev/block/recovery:SHA1"
should be:
"if ! applypatch --check EMMC:/dev/block/recovery$(getprop ro.boot.slot_suffix):SHA1"
Bug: 156979431
Test: rerun add_image_to_target_file, then validate_target_files
Change-Id: Id2d45e281151a46d66b37ea8fdb7b5cb4ae72e81
(cherry picked from commit e55f62c971)
Add common.SetHostToolLocation so that, when calling
various common.Run* functions, replace the first
argument with previously set paths. This is needed
when the script is executed in a Soong sandbox.
Bug: 161563386
Test: pass
Change-Id: If3b40b518fc7fe068677d39d604e3f6578a12ea3
The core android build platform has changed to add support for LZ4
compression for ramdisks, but the release tools were not. Fix this.
Bug: 156129966
Change-Id: I39680b91930d2d3cbd0cd565beb78e1ee699397e
The custom images are any images owned by OEMs and SoCs, oem images
mounted on /oem is an example. The oem images can be used to customize
devices for different carriers, like wallpaper, ringtones, and
carrier-specific apks. OEMs can generate multiple oem images, like
oem.img, oem-carrier1.img and oem-carrier2.img and flash different oem
images for different carriers. The oem images are only one case, OEMs
and SoCs can add more custom images and mount them to custom partitions.
This change enables custom images to be vbmeta.img chained partitions.
The following configuration in BoardConfig.mk is an exmaple. It has two
custom partitions: oem and test. They will be signed by different keys.
And they will be chained by vbmeta.img. The custom images here are
prebuilts, which can be built by `make custom_images` separately.
BOARD_AVB_<CUSTOM_PARTITION>_IMAGE_LIST should include all custom images
to apply AVB signing. And to every custom partition, one image whose
name is partition name must be added in its
BOARD_AVB_<CUSTOM_PARTITION>_IMAGE_LIST.
BOARD_CUSTOMIMAGES_PARTITION_LIST := oem test
BOARD_AVB_OEM_KEY_PATH := external/avb/test/data/testkey_rsa4096.pem
BOARD_AVB_OEM_ALGORITHM := SHA256_RSA4096
BOARD_AVB_OEM_ADD_HASHTREE_FOOTER_ARGS :=
BOARD_AVB_OEM_ROLLBACK_INDEX_LOCATION := 1
BOARD_AVB_OEM_PARTITION_SIZE := 5242880
BOARD_AVB_OEM_IMAGE_LIST := \
device/xxxx/yyyy/oem/oem.img \
device/xxxx/yyyy/oem/oem1.img
BOARD_AVB_TEST_KEY_PATH := external/avb/test/data/testkey_rsa2048.pem
BOARD_AVB_TEST_ALGORITHM := SHA256_RSA2048
BOARD_AVB_TEST_ADD_HASHTREE_FOOTER_ARGS :=
BOARD_AVB_TEST_ROLLBACK_INDEX_LOCATION := 2
BOARD_AVB_TEST_PARTITION_SIZE := 10485760
BOARD_AVB_TEST_IMAGE_LIST := \
device/xxxx/yyyy/test/test.img \
device/xxxx/yyyy/test/test1.img
To resign the custom images in the target zip file, the
avb_extra_custom_image_key, avb_extra_custom_image_algorithms and
avb_extra_custom_image_extra_args options are added to the
sign_target_files_apks tool too. The following test cases list some
examples about how to use them.
BUG: 154171021
Test: 1) "atest --host releasetools_test releasetools_py3_test -c"
2) Build images by 'make dist', sign and validate target files.
a) Test on dist w/ chained vbmeta_system and ome custom images
sign_target_files_apks -d certs \
--avb_extra_custom_image_key oem=oem_rsa4096.pem \
--avb_extra_custom_image_algorithm oem=SHA256_RSA4096 \
xxx-target_xxx.zip signed.zip
validate_target_files.py signed.zip
Flash image and boot up.
Verify the oem images and vbmeta images in OUT and target zips by
avbtool.
b) Test on dist w/ chained vbmeta_system and oem and test custom images
sign_target_files_apks -d certs \
--avb_extra_custom_image_key oem=oem_rsa4096.pem \
--avb_extra_custom_image_algorithm oem=SHA256_RSA4096 \
--avb_extra_custom_image_extra_args oem=--do_not_generate_fec \
--avb_extra_custom_image_key test=test_rsa4096.pem \
--avb_extra_custom_image_algorithm test=SHA256_RSA4096 \
xxx-target_xxx.zip signed.zip
validate_target_files.py signed.zip
Verify the oem, test images and vbmeta images in OUT and target zips
by avbtool.
c) Test on dist w/o chained partition.
sign_target_files_apks -d certs xxx-target_xxx.zip signed.zip
validate_target_files.py signed.zip
Flash image and boot up.
Verify the vbmeta images in OUT and target zips by avbtool.
Change-Id: Ifccfee5e8909697eef6ccda0cc352fa16a9f6db6
Whenever a device is retrieved from fstab, wrap it with
add_slot_suffix() if it has slotselect option.
Test: change fstab (changes boot image, which is a static partition),
change system partition (a dynamic partition),
generate incremental OTA with --force_non_ab and apply it on
cuttlefish
Bug: 153581609
Change-Id: Id3f8e4425b65176baf1b0ff1ee07ab3d820a3a7f
(cherry picked from commit ae6e0d5d28)
Merged-In: Id3f8e4425b65176baf1b0ff1ee07ab3d820a3a7f
Force generate a non-A/B update package when requested.
Bug: 154344887
Test: ota_from_target_files.py --force_non_ab ...
Test: apply it as well
Change-Id: I5e81eb161722e07ef50081b6a16685cbc9963ae2
(cherry picked from commit 7169f754cc)
Merged-In: I5e81eb161722e07ef50081b6a16685cbc9963ae2
The different boot images in aosp_arm64 have different partition size
requirements. Add support for defining a partition size associated with
each boot image variant.
This support is limited to aosp_arm64 currently, which is built with
recovery-as-boot.
A previous version of this change did not update the boot-debug image
AVB logic with the required macro for multiple boot partition size
support. This is now required because the aosp-arm64 boot.img is
configured as a chained AVB partition.
Bug: 156036850
Bug: 155049180
Change-Id: I66b57de91042bfd56ba54a3659843d8cf7873955
The oem partition allows system build.prop to import properties from it
by "import /oem/oem.prop xxxx". An _ImportParser() error was raised on
this case.
BUG: 154171021
Test: 1) "atest --host releasetools_test releasetools_py3_test -c"
2) On a device who has oem partition, "make dist" and sign its
target zip file.
Change-Id: I47875bf7a698390e11690150e6516a3064550ca0
The different boot images in aosp_arm64 have different partition size
requirements. Add support for defining a partition size associated with
each boot image variant.
This support is limited to aosp_arm64 currently, which is built with
recovery-as-boot.
Bug: 156036850
Bug: 155049180
Change-Id: I73b70a4d52c2662c3015899632bb8a2b424108d5
This reverts commit b21e48b499.
In practice, some partners use the 'import' statement to override
the device fingerprint at runtime. The runtime fingerprint will
later add to the metadata of OTA package, so that the OTA server
can deliver the package to corresponding devices correctly.
This CL supports parsing a subset of import statement that the init
process recognizes. And we loose the restriction based on how the
dynamic fingerprint is used in practice. Right now, we only searches
for the override of brand, name and device. And the placeholder
format should be ${placeholder}, with its value supplied by the
script caller.
As part of the implementation, we generate all the possible
combinations of the input boot variables. And recalculate the
fingerprint for each of the combination. Though we load the
build.prop multiple times, the logic is easier to follow. Also,
it's more convenient to enhance the logic if we only want to
allow some of the boot variables combination later.
Bug: 152167826
Change-Id: I4a9fa35c7ac037ff1cf4f9a4bdff602beac3894b
Test: unittests pass
This reverts commit 6022545272.
The build prop for a partition used to be a simple key:value
dictionary. But we need more fields to hold the alternative build
props overriden by the 'import' statement. Therefore, add a new
class as a wrapper for these props first.
Bug: 152167826
Change-Id: I5c952cd2a976ba1a09ddc66d56c2b8b55a61986b
Test: unittests pass
In practice, some partners use the 'import' statement to override
the device fingerprint at runtime. The runtime fingerprint will
later add to the metadata of OTA package, so that the OTA server
can deliver the package to corresponding devices correctly.
This CL supports parsing a subset of import statement that the init
process recognizes. And we loose the restriction based on how the
dynamic fingerprint is used in practice. Right now, we only searches
for the override of brand, name and device. And the placeholder
format should be ${placeholder}, with its value supplied by the
script caller.
As part of the implementation, we generate all the possible
combinations of the input boot variables. And recalculate the
fingerprint for each of the combination. Though we load the
build.prop multiple times, the logic is easier to follow. Also,
it's more convenient to enhance the logic if we only want to
allow some of the boot variables combination later.
Bug: 152167826
Test: unittests pass
Change-Id: Iac093a40dc6f873c5e1858efa44cb2bd6082508a
The build prop for a partition used to be a simple key:value
dictionary. But we need more fields to hold the alternative build
props overriden by the 'import' statement. Therefore, add a new
class as a wrapper for these props first.
Bug: 152167826
Test: unittests pass
Change-Id: I2fe7e93a2f4de8e55f5f8051b000b96b5efdc85a
The oem_dicts in BuildInfo is only available to ota_from_target_files
when it's called with "--oem_settings" input. However, aosp/1135332
starts to use BuildInfo in add_img_to_target_files w/o oem_dicts. An
assert is triggered due to it when oem_fingerprint_properties is in the
info_dict.
This change skip _oem_props reading if oem_dicts is None to allow to
this case.
BUG: 155360923
BUG: 154171021
Test: Sign images with oem_fingerprint_properties.
Change-Id: I6a73feecd9567fd4d85f4eab3d4e11c5df28fe39
For older target_files.zip, since "recovery_mkbootimg_args" may not be
set fallback to the earlier config -"mkbootimg_args" for recovery image.
This will make releasetools backward compatible, and usable for
older/already generated target-files package.
Change-Id: If40359130b8e6da48cb8c345d8cb079f457fe36f
When loading build info from a previous version of Android, the set of
allowed property sources should match those available in that version.
In this particular case, the product_services partition was a valid
property source in Android 10.
Bug: 155053195
Test: ran unit tests from test_common.py
Test: generated an incremental OTA which previously failed
Change-Id: Ic0b0a112656533eca78dee31517deff7e3c8d7cc
The arguments transparency_log_servers and transparency_log_pub_keys
have been merged. Update the caller in the OTA script accordingly.
Also disable the test to contact aftl server until we have
a public server.
Bug: 153940575
Test: check the argument
Change-Id: If6a7e7d644884d395c75c2fcdfd6aa7c2380d851
Support for generating multiple boot.img files is required in the
release tools to enable GKI distribution/signing.
Bug: 151094943
Change-Id: I536a286d3123f35918106a52c49b1148d746370f
Add support for building a non-v3 recovery image along with a
v3 boot.img and vendor_boot image.
Bug: 150825361
Change-Id: I80d4e71ff09b83dd1c7f9accb4de99a30f95f4bf
Since we might use ToT release tools to sign a package
generated by an older build, we make the new
`partition=` tag optional. This also means we need to be
careful to use non-greedy regex matching.
Bug: 153133823
Test: python3 -m unittest
Test: input with and without the new `partition=` tag
Test: new test_ReadApkCerts_WithWithoutOptionalFields
Change-Id: Ic57efd34e745ad302ae17150c6f2318f0b4524cb
Right now we assert that the aftltool should exist since the one
in aosp won't work due to grpc dependencies.
Bug: 147870995
Test: build
Change-Id: Iabb2c375167572a965493a7648fdc1abe287af67
