Commit Graph

437 Commits

Author SHA1 Message Date
Paul Duffin
00f6a090c2 Merge "Revert "support different boot partition sizes in aosp_arm64"" am: 29beb858ef am: 61b3971e66
Change-Id: I9bfa60c9d39199735676b776cde921d9628c7209
2020-05-14 09:57:56 +00:00
Paul Duffin
29beb858ef Merge "Revert "support different boot partition sizes in aosp_arm64"" 2020-05-14 09:23:49 +00:00
Paul Duffin
b5a334d0b3 Revert "support different boot partition sizes in aosp_arm64"
This reverts commit 11c4bcf406.

Reason for revert: Broke build b/156569314

Bug: 156569314
Change-Id: I384740320acf48d09b0544e45b5a13ff0a6a163d
2020-05-14 08:40:52 +00:00
Treehugger Robot
50639bd4cb Merge "support different boot partition sizes in aosp_arm64" am: ade028ea9e am: 6f978c6b7f
Change-Id: Icaf368a8bb753eff980bc053395ad09033dfd47d
2020-05-14 07:52:31 +00:00
Treehugger Robot
ade028ea9e Merge "support different boot partition sizes in aosp_arm64" 2020-05-14 07:05:31 +00:00
TreeHugger Robot
1f3678e84c Merge changes from topic "nonab_on_vab_r" into rvc-dev am: 4d8e9cc847
Change-Id: I18ae2cc04e6edad5498fdc0c2de6f5a163d10c82
2020-05-13 18:25:35 +00:00
Yifan Hong
010d320848 resolve merge conflicts of 7169f754cc to rvc-dev-plus-aosp
Change-Id: Ic1a44cfe929707e2fb76282dcd06f839a8159735
2020-05-13 17:44:04 +00:00
TreeHugger Robot
4d8e9cc847 Merge changes from topic "nonab_on_vab_r" into rvc-dev
* changes:
  Use add_slot_suffix function in edify script
  Support forcefully generating non-AB packages.
2020-05-13 17:26:44 +00:00
Steve Muckle
11c4bcf406 support different boot partition sizes in aosp_arm64
The different boot images in aosp_arm64 have different partition size
requirements. Add support for defining a partition size associated with
each boot image variant.

This support is limited to aosp_arm64 currently, which is built with
recovery-as-boot.

Bug: 156036850
Bug: 155049180
Change-Id: I73b70a4d52c2662c3015899632bb8a2b424108d5
2020-05-12 16:51:45 -07:00
Tianjie Xu
9afb221c9e Reland "Calculate the runtime fingerprint prefixes from build prop"
This reverts commit b21e48b499.

In practice, some partners use the 'import' statement to override
the device fingerprint at runtime. The runtime fingerprint will
later add to the metadata of OTA package, so that the OTA server
can deliver the package to corresponding devices correctly.

This CL supports parsing a subset of import statement that the init
process recognizes. And we loose the restriction based on how the
dynamic fingerprint is used in practice. Right now, we only searches
for the override of brand, name and device. And the placeholder
format should be ${placeholder}, with its value supplied by the
script caller.

As part of the implementation, we generate all the possible
combinations of the input boot variables. And recalculate the
fingerprint for each of the combination. Though we load the
build.prop multiple times, the logic is easier to follow. Also,
it's more convenient to enhance the logic if we only want to
allow some of the boot variables combination later.

Bug: 152167826
Change-Id: I4a9fa35c7ac037ff1cf4f9a4bdff602beac3894b
Test: unittests pass
2020-05-10 21:49:53 +00:00
Tianjie Xu
0fde41e711 Reland "Add a wrapper class PartitionBuildProp""
This reverts commit 6022545272.

The build prop for a partition used to be a simple key:value
dictionary. But we need more fields to hold the alternative build
props overriden by the 'import' statement. Therefore, add a new
class as a wrapper for these props first.

Bug: 152167826
Change-Id: I5c952cd2a976ba1a09ddc66d56c2b8b55a61986b
Test: unittests pass
2020-05-09 05:24:18 +00:00
Greg Kaiser
6022545272 Revert "Add a wrapper class PartitionBuildProp"
Revert submission 1297095

Bug: 156131275
Reason for revert: Broken build
Reverted Changes:
I2fe7e93a2:Add a wrapper class PartitionBuildProp
Iac093a40d:Calculate the runtime fingerprint prefixes from bu...

Change-Id: Ie846d23b9f5c3325d021236725826512be7a3c78
2020-05-09 00:30:33 +00:00
Greg Kaiser
b21e48b499 Revert "Calculate the runtime fingerprint prefixes from build prop"
Revert submission 1297095

Bug: 156131275
Reason for revert: Broken build
Reverted Changes:
I2fe7e93a2:Add a wrapper class PartitionBuildProp
Iac093a40d:Calculate the runtime fingerprint prefixes from bu...

Change-Id: I8b1262c6e94a1db349de39896e56f366c8d64d4c
2020-05-09 00:30:33 +00:00
Tianjie
0d2fcd50d7 Calculate the runtime fingerprint prefixes from build prop
In practice, some partners use the 'import' statement to override
the device fingerprint at runtime. The runtime fingerprint will
later add to the metadata of OTA package, so that the OTA server
can deliver the package to corresponding devices correctly.

This CL supports parsing a subset of import statement that the init
process recognizes. And we loose the restriction based on how the
dynamic fingerprint is used in practice. Right now, we only searches
for the override of brand, name and device. And the placeholder
format should be ${placeholder}, with its value supplied by the
script caller.

As part of the implementation, we generate all the possible
combinations of the input boot variables. And recalculate the
fingerprint for each of the combination. Though we load the
build.prop multiple times, the logic is easier to follow. Also,
it's more convenient to enhance the logic if we only want to
allow some of the boot variables combination later.

Bug: 152167826
Test: unittests pass
Change-Id: Iac093a40dc6f873c5e1858efa44cb2bd6082508a
2020-05-07 23:52:07 -07:00
Yifan Hong
ae6e0d5d28 Use add_slot_suffix function in edify script
Whenever a device is retrieved from fstab, wrap it with
add_slot_suffix() if it has slotselect option.

Test: change fstab (changes boot image, which is a static partition),
      change system partition (a dynamic partition),
      generate incremental OTA with --force_non_ab and apply it on
      cuttlefish
Bug: 153581609
Change-Id: Id3f8e4425b65176baf1b0ff1ee07ab3d820a3a7f
2020-05-07 16:55:39 -07:00
Yifan Hong
7169f754cc Support forcefully generating non-AB packages.
Force generate a non-A/B update package when requested.

Bug: 154344887
Test: ota_from_target_files.py --force_non_ab ...
Test: apply it as well

Change-Id: I5e81eb161722e07ef50081b6a16685cbc9963ae2
2020-05-07 12:48:17 -07:00
P.Adarsh Reddy
e4d5d568a3 releasetools: Fallback to "mkbootimg_args" if "recovery_mkbootimg_args" not found.
For older target_files.zip, since "recovery_mkbootimg_args" may not be
set fallback to the earlier config -"mkbootimg_args" for recovery image.
This will make releasetools backward compatible, and usable for
older/already generated target-files package.

Bug: 155708400
Change-Id: If40359130b8e6da48cb8c345d8cb079f457fe36f
Merged-In: If40359130b8e6da48cb8c345d8cb079f457fe36f
2020-05-07 09:15:57 -07:00
Tianjie
fd3883f159 Add a wrapper class PartitionBuildProp
The build prop for a partition used to be a simple key:value
dictionary. But we need more fields to hold the alternative build
props overriden by the 'import' statement. Therefore, add a new
class as a wrapper for these props first.

Bug: 152167826
Test: unittests pass
Change-Id: I2fe7e93a2f4de8e55f5f8051b000b96b5efdc85a
2020-05-06 22:09:44 -07:00
Treehugger Robot
ec8abc0eb3 Merge "releasetools: Fallback to "mkbootimg_args" if "recovery_mkbootimg_args" not found." 2020-05-06 07:32:12 +00:00
Hongguang Chen
3435403284 Skip _oem_props if oem_dicts is None.
The oem_dicts in BuildInfo is only available to ota_from_target_files
when it's called with "--oem_settings" input. However, aosp/1135332
starts to use BuildInfo in add_img_to_target_files w/o oem_dicts. An
assert is triggered due to it when oem_fingerprint_properties is in the
info_dict.
This change skips _oem_props reading if oem_dicts is None to allow this
case.

BUG: 155360923
BUG: 154171021
Test: Sign images with oem_fingerprint_properties.
Change-Id: I6a73feecd9567fd4d85f4eab3d4e11c5df28fe39
Merged-In: Idfd41c8b20b23d50e4895f24500f9a25625b7d98
2020-05-04 20:45:33 +00:00
Hongguang Chen
d7c160ffbf Skip _oem_props if oem_dicts is None.
The oem_dicts in BuildInfo is only available to ota_from_target_files
when it's called with "--oem_settings" input. However, aosp/1135332
starts to use BuildInfo in add_img_to_target_files w/o oem_dicts. An
assert is triggered due to it when oem_fingerprint_properties is in the
info_dict.
This change skip _oem_props reading if oem_dicts is None to allow to
this case.

BUG: 155360923
BUG: 154171021
Test: Sign images with oem_fingerprint_properties.
Change-Id: I6a73feecd9567fd4d85f4eab3d4e11c5df28fe39
2020-05-04 16:54:10 +00:00
P.Adarsh Reddy
d8e24eee70 releasetools: Fallback to "mkbootimg_args" if "recovery_mkbootimg_args" not found.
For older target_files.zip, since "recovery_mkbootimg_args" may not be
set fallback to the earlier config -"mkbootimg_args" for recovery image.
This will make releasetools backward compatible, and usable for
older/already generated target-files package.

Change-Id: If40359130b8e6da48cb8c345d8cb079f457fe36f
2020-05-04 14:30:09 +00:00
Steven Laver
dd33d75d39 releasetools: correct allowed property sources for incremental OTAs
When loading build info from a previous version of Android, the set of
allowed property sources should match those available in that version.
In this particular case, the product_services partition was a valid
property source in Android 10.

Bug: 155053195
Test: ran unit tests from test_common.py
Test: generated an incremental OTA which previously failed
Change-Id: Ic0b0a112656533eca78dee31517deff7e3c8d7cc
Merged-In: Ic0b0a112656533eca78dee31517deff7e3c8d7cc
2020-04-30 10:54:48 -07:00
Steven Laver
8e2086e6e3 releasetools: correct allowed property sources for incremental OTAs
When loading build info from a previous version of Android, the set of
allowed property sources should match those available in that version.
In this particular case, the product_services partition was a valid
property source in Android 10.

Bug: 155053195
Test: ran unit tests from test_common.py
Test: generated an incremental OTA which previously failed
Change-Id: Ic0b0a112656533eca78dee31517deff7e3c8d7cc
2020-04-29 22:30:42 +00:00
Steve Muckle
f83e3c3c3f support multiple boot.img files in release tools
Support for generating multiple boot.img files is required in the
release tools to enable GKI distribution/signing.

Bug: 151094943
Change-Id: I536a286d3123f35918106a52c49b1148d746370f
Merged-In: I536a286d3123f35918106a52c49b1148d746370f
2020-04-22 11:14:05 -07:00
Tianjie Xu
f53f313c36 Merge "Update the argument when signing aftl" 2020-04-21 22:39:31 +00:00
Tianjie
20dd8f20b1 Update the argument when signing aftl
The arguments transparency_log_servers and transparency_log_pub_keys
have been merged. Update the caller in the OTA script accordingly.

Also disable the test to contact aftl server until we have
a public server.

Bug: 153940575
Test: check the argument
Change-Id: If6a7e7d644884d395c75c2fcdfd6aa7c2380d851
2020-04-21 22:38:55 +00:00
Steve Muckle
9793cf6c8a support multiple boot.img files in release tools
Support for generating multiple boot.img files is required in the
release tools to enable GKI distribution/signing.

Bug: 151094943
Change-Id: I536a286d3123f35918106a52c49b1148d746370f
2020-04-16 17:54:25 -07:00
Steve Muckle
759d0c8cb2 Enable non-v3 recovery image with v3 boot image
Add support for building a non-v3 recovery image along with a
v3 boot.img and vendor_boot image.

Bug: 150825361
Change-Id: I80d4e71ff09b83dd1c7f9accb4de99a30f95f4bf
Merged-In: I80d4e71ff09b83dd1c7f9accb4de99a30f95f4bf
2020-04-14 15:40:01 -07:00
Steve Muckle
f84668e1da Enable non-v3 recovery image with v3 boot image
Add support for building a non-v3 recovery image along with a
v3 boot.img and vendor_boot image.

Bug: 150825361
Change-Id: I80d4e71ff09b83dd1c7f9accb4de99a30f95f4bf
2020-04-14 13:03:58 -07:00
Bill Peckham
96c9e6ed00 Make the partition= tag optional.
Since we might use ToT release tools to sign a package
generated by an older build, we make the new
`partition=` tag optional. This also means we need to be
careful to use non-greedy regex matching.

Bug: 153133823
Test: python3 -m unittest
Test: input with and without the new `partition=` tag
Test: new test_ReadApkCerts_WithWithoutOptionalFields
Change-Id: Ic57efd34e745ad302ae17150c6f2318f0b4524cb
Merged-In: Ic57efd34e745ad302ae17150c6f2318f0b4524cb
2020-04-03 21:23:04 -07:00
Bill Peckham
5c7b034a5c Make the partition= tag optional.
Since we might use ToT release tools to sign a package
generated by an older build, we make the new
`partition=` tag optional. This also means we need to be
careful to use non-greedy regex matching.

Bug: 153133823
Test: python3 -m unittest
Test: input with and without the new `partition=` tag
Test: new test_ReadApkCerts_WithWithoutOptionalFields
Change-Id: Ic57efd34e745ad302ae17150c6f2318f0b4524cb
2020-04-03 17:09:37 -07:00
Bill Peckham
8676f639f3 Build merged apexkeys.txt/apkcerts.txt by partition.
Propagate partition tag data to apexkeys.txt and
apkcerts.txt so that merge_target_files.py can build
merged versions of these files by filtering the
framework files for framework partitions and filtering
the vendor files for vendor partitions.

Bug: 138942268
Change-Id: Ic3226728e97dae84d38ec230ccc86d1b124bea94
Merged-In: Ic3226728e97dae84d38ec230ccc86d1b124bea94
2020-04-02 17:16:01 +00:00
Tianjie
0f3074566c Add aftltool is a signing parameter
Right now we assert that the aftltool should exist since the one
in aosp won't work due to grpc dependencies.

Bug: 147870995
Test: build
Change-Id: Iabb2c375167572a965493a7648fdc1abe287af67
2020-04-01 12:20:21 -07:00
Tianjie Xu
a3541df291 Merge "Add aftl inclusion proof" 2020-03-31 22:12:46 +00:00
Bill Peckham
19c3feb2d3 Build merged apexkeys.txt/apkcerts.txt by partition.
Propagate partition tag data to apexkeys.txt and
apkcerts.txt so that merge_target_files.py can build
merged versions of these files by filtering the
framework files for framework partitions and filtering
the vendor files for vendor partitions.

Bug: 138942268
Change-Id: Ic3226728e97dae84d38ec230ccc86d1b124bea94
2020-03-30 22:33:27 +00:00
Tianjie Xu
eaed60c1a1 Add aftl inclusion proof
The otatools should talk to the aftl server and append the inclusion
proofs when building the vbmeta image. We should only do this during the
signing process when the network is always available.

Also the inclusion proof doesn't impact the final vbmeta image size on
coral, where the final size is 8192 after padding. This is below the
physical image of 65536.

Bug: 147870995
Test: unit tests pass, run sign_target_file_apks
Change-Id: If84c6bf5198c9b05f5e0c16ae6335971915f47e3
2020-03-18 13:11:41 -07:00
Daniel Norman
ab5acefb6d RESTRICT AUTOMERGE Uses a per-partition fingerprint for building images and avb_salt.
This causes the output image files of a merged build to be identical
to the image files of the input partial builds, for each images in
PARTITIONS_WITH_CARE_MAP.

Test: python -m unittest test_common
Test: `m dist`; `unzip out/dist/target_files.zip IMAGES/\*`;
      `zip -d out/dist/target_files.zip IMAGES/\*`
      `add_img_to_target_files -a out/dist/target_files.zip`.
      Verify that the rebuilt images are identical to the deleted ones.
Test: Build a merged target (using merge_target_files.py). Verify that
      the partial target-files.zip IMAGES are identical to the merged
      target-files.zip IMAGES for PARTITIONS_WITH_CARE_MAP images.
Bug: 150405807
Change-Id: I5fdf5783c1aff9c14cf5408090389b1f65b69ca6
(cherry picked from commit d5fe862628)
2020-03-10 11:04:31 -07:00
Daniel Norman
d5fe862628 Uses a per-partition fingerprint for building images and avb_salt.
This causes the output image files of a merged build to be identical
to the image files of the input partial builds, for each images in
PARTITIONS_WITH_CARE_MAP.

Test: python -m unittest test_common
Test: `m dist`; `unzip out/dist/target_files.zip IMAGES/\*`;
      `zip -d out/dist/target_files.zip IMAGES/\*`
      `add_img_to_target_files -a out/dist/target_files.zip`.
      Verify that the rebuilt images are identical to the deleted ones.
Test: Build a merged target (using merge_target_files.py). Verify that
      the partial target-files.zip IMAGES are identical to the merged
      target-files.zip IMAGES for PARTITIONS_WITH_CARE_MAP images.
Bug: 150405807
Change-Id: I5fdf5783c1aff9c14cf5408090389b1f65b69ca6
2020-03-02 19:05:20 +00:00
Tianjie Xu
88a759d651 Resign apks contained in apex
Some apex payload images contain apk files. And these apks need to be
signed during the signing processed when sign_target_files_apks is
called. To support the signing, we can extract the payload and repack
the apex file with the (de)apexer tool. Add the signing support in the
apex_util.

Bug: 146508800
Test: unit tests pass, run sign_apex, sign_target_files_apks
Change-Id: If6d58975248709a144b07dbabf47c27916e5695e
2020-01-27 19:48:39 +00:00
Treehugger Robot
88ab531831 Merge "Go one directory up looking for releasetools path" 2020-01-10 12:31:32 +00:00
Robin Lee
34ea73940e Go one directory up looking for releasetools path
Change I366099c3dfd5fa4282745ef258a8cf35338e1e42 introduced a more
restricted search where we only look in out/host/x/bin instead of
out/host/x when looking for files.

Because signapk is in out/host/x/framework, and lib64 is in out/host/x,
we need to go one level up for this to work properly.

Fix: 147096688
Test: atest --host releasetools_test releasetools_py3_test
Change-Id: I597e1d95c09eb0e27b2faebb107218761901d9ac
2020-01-02 19:25:20 +00:00
P Adarsh Reddy
7e9b5c4204 Handle virtual A/B defines during dynamic_partitions_info merge.
While merging the Dynamic Partition info Dicts in the case of
split builds, pick the virtual-a/b flags like virtual_ab,
virtual_ab_retrofit from the vendor half.

Bug: 146811936
Test: Merged build contains the virtual a/b defines.
Change-Id: I0171b074094e47cfd12dc6af2565acdcd1b65163
2019-12-24 06:32:03 +00:00
Dan Austin
5290364e9b Add AFTL relevant options to OPTIONS in common.py.
Adds AFTL relevant options in preparation for avbtool changes. The
options are:
aftl_server: A string representing the URL of the transparency log.
aftl_key_path: The path to the public key associated with the
transparency log (in PEM format).
aftl_manufacturer_key_path: The path to the key associated with the
manufacturer for AFTL message signing.
aftl_signing_helper: The signing helper for use with AFTL message
signing.

Change-Id: I73d466f24b171c16dbdaa40c589779bed52be551
Test: Ran add_img_to_target_files with all AFTL options.
Bug: 138779249
2019-12-12 15:48:52 -08:00
David Anderson
0ec64ac0aa releasetools: Generalize base_fs_file handling.
Share the code to handle base_fs files rather than duplicate it for
system/vendor. At the same time, add support for system_ext and product
partitions.

Bug: 139201772
Test: manual test
Change-Id: Ib1b7c018c828901e24eefc01e3faead03e77bc2d
2019-12-11 22:32:00 +00:00
Treehugger Robot
e7c1f6314c Merge "Removes custom prefix/suffix from MergeDynamicPartitionInfoDicts." 2019-11-27 20:30:34 +00:00
Daniel Norman
55417148f9 Removes custom prefix/suffix from MergeDynamicPartitionInfoDicts.
All callers of this function now always pass the same values, so this
change hardcodes those values within the function body.

Fix: 145008064
Test: python -m unittest test_common
Test: build & boot a merged target that uses DAP

Change-Id: I0051c5ba507983231825edfcaf349e574efa451a
2019-11-26 11:09:17 -08:00
Yifan Hong
874a5aa95e Merge "Move check partition size logs to a file" 2019-11-26 00:32:52 +00:00
Donghoon Yu
92420db7dd releasetools: fix avbtool arguments.
The following error occurred when building dist.
Separate these arguments

error log:
avbtool: error: unrecognized arguments: --partition_name vendor_boot

Bug: 144820089

Change-Id: I3891ef3bd9a2f7d2cd1377228b9894b599dc1cc3
Signed-off-by: Donghoon Yu <hoony.yu@samsung.com>
2019-11-21 06:15:02 +00:00
Yifan Hong
309109351e Move check partition size logs to a file
Don't contaminate the build log.
Test: m check-all-partition-sizes -j (verbose logs stored to file)
Test: m check-all-partition-sizes-nodeps -j (see verbose logs)
Test: manually run the script with[out] -v and with[out] --logfile

Change-Id: I345a340deab3e29bb9cb05d4970a55d8758607a7
2019-11-05 14:06:27 -08:00