Commit Graph

3015 Commits

Author SHA1 Message Date
Treehugger Robot
1cc902690e Merge "Fix ext4 size estimates" into main 2024-01-29 20:06:18 +00:00
Yi-Yo Chiang
accdd3f087 Merge "Re-land: releasetools: Deprecate GKI build rules" into main 2024-01-25 05:29:11 +00:00
Yi-Yo Chiang
a4d5f4380b Re-land: releasetools: Deprecate GKI build rules
(Re-land of Ie882fccd864920289e48366e99a4ebd67e784d0d)

We no longer build GKIs from the platform tree.
These build commands were neither used nor maintained anymore, so clean
them up to reduce maintenance effort.
Keep the command line options as no-op, so existing scripts that still
specifies the deprecated options don't break.

Bug: 229701033
Test: presubmit
Change-Id: I0f7d05562dbc3eed29e902d6dc9a0f2e4083aaa6
2024-01-25 11:21:00 +08:00
Kelvin Zhang
158a5ebbc9 Fix ext4 size estimates
Currently, EXT4 image building is done in a 2 pass fashion:

First pass: build ext4 image with small margin/headroom, check the
number of block used in the output image

Second pass: Build ext4 image again with an estimated block count from
first pass

The two pass implementation is done to keep image size at a
minimum. Recently some failures in first pass invocation is observed due
to image being undersized. Since we will reduce the image size in 2nd
pass, it's OK to increase the image size estimate during first pass. Add
a headroom 10% of total file size to fix build failures.

Test: cherry pick r.android.com/q/topic:%22aosp_cf_system_x86_64%22 and
m aosp_cf_system_x86_64

Bug: 321003625
Change-Id: I254310c67a08b98b05d2c858d4ab59a48112a07b
2024-01-24 18:37:27 -08:00
Jooyung Han
750aad5c32 Add all apexes to apex_info
Previously, META/apex_info.pb contained only /system/apex apexes. Now,
it has all apexes from all possible partitions.

The main purpose of this file is to caculate the decompressed apex size
when applying OTA. Hence it should have all apexes, not just system
apexes.

Bug: 320228659
Test: m dist # check META/apex_info.pb
Change-Id: I3428dc502e4fe3336d1fc5ca941f1fbc332985cd
2024-01-23 05:52:51 +09:00
Treehugger Robot
3b7d46f4ca Merge "Fix signing failures on targets with >4GB target_files" into main 2024-01-19 20:06:07 +00:00
Kelvin Zhang
12ac11025e Fix signing failures on targets with >4GB target_files
An old python zipfile hack prevented zipfile module from decoding 64 bit
sizes correctly, remove the legacy hack to fix.

Test: check_target_files_signatures -v 5GB_target_files.zip
Bug: 319367048
Change-Id: I376c7b68f549ddf88680280d604548d1849cdfe2
2024-01-19 11:04:58 -08:00
Tim Tsai
7cd5b313f6 Merge "Fixed not mapping sdk_sandbox key to vendor key" into main 2024-01-17 17:08:07 +00:00
Treehugger Robot
0562b343d1 Merge "Remove unnecessary calling of PayloadGenerator" into main 2024-01-17 07:18:51 +00:00
Daniel Zheng
15a36a14d6 Update check to work with compression levels
vabc_compression_param can be a combination of algorithm + level.
Updating this check so that it works with both.

Test: ota_from_target_files
Change-Id: I2f13ca31c728d7c8607ac085c9b663691f8dfa02
2024-01-16 12:41:01 -08:00
Cloud You
0dbd877e5f Fixed not mapping sdk_sandbox key to vendor key
Some apps are signed with default sdk_sandbox key.
It results in failure of GtsPackageSignatureTest

Bug: 318798881
Test: run gts -m GtsPackageSignatureTest
Change-Id: I676f48eb2844866f4e61af60d78026a87303d15a
2024-01-10 15:12:39 +08:00
Daniel Zheng
a567eae9ad Support configuring compression factor
With the introduction of variable block sized compression. We want
ota_from_target_files to support a flag that confgures the max block
size for compression. This flag will be passed to delta_generator and
then used fro cow estimation + ota installation

Removing a function here as it looks like it's unused. We can modify the
specified values with ModifyTargetFilesDynamicPartitionInfo(). The other
function looks like it's just a wrapper.

Test: ota_from_target_files.

Change-Id: Ia17bf62d40f947ef1fbe543886f04d10acd5bcc9
2024-01-09 16:52:20 -08:00
Virajitha Daibala
779c960d9e Remove unnecessary calling of PayloadGenerator
PayloadGenerator object is created twice, introduced in:
1f93ce20c5

Remove this extra call.

Test: atest --host releasetools_test
Change-Id: I7701542f7f53c9295a44fc23f634d863ab4cfc52
2024-01-08 14:30:00 +01:00
Håkan Kvist
b77972402e Detect downgrade by checking build time for all partitions
Checking ro.build.date.utc to detect a downgrade is not enough in the
case where several target files are merged, as timestamps can differ
for each partition.

The solution is to check the build timestamp for each partition and
ro.build.date.utc.

With this change ota generation will be aborted during creation
instead of rejected when installing on device.

This also fixes the --override_timestamp so it is possible to generate
a package where timestamps are reversed, as was added in:
3e6161a3b3.

Issue: 315913966
Test: Manual, confirm that ota generation is aborted when timestamp
  on post vendor partitions is newer than on pre.
  Ota created successfully with --override_timestamp parameter and
  installation successful
Test: atest --host releasetools_test
Change-Id: I275e67a3840f4ef2263381c253231068e72f47d2
2023-12-20 08:45:26 +01:00
Kelvin Zhang
8793a72bfd Support overriding cow version during OTA generation
Added a --vabc_cow_version flag to override cow version.
This enables testing on COW v3 without switching the android platform to
v3.

Test: th
Bug: 313962438
Change-Id: I9ff3ae6b6c7c8ef8c1423af6f25e420f94558d35
2023-12-07 13:20:43 -08:00
Kelvin Zhang
84e14c2102 Update create_brick_ota doc
Serial number need to be a | separated list per
bootable/recovery/install/install.cpp

Test: th
Change-Id: I09d8d615fde4206ebe6c07a7314ff8f7a394f10c
2023-12-06 13:01:54 -08:00
Treehugger Robot
ca96bcb0c8 Merge "Sign OTA packages inside target_files during signing" into main 2023-12-05 21:46:55 +00:00
Yi-Yo Chiang
18650c7be0 releasetools: Remove deprecated BOARD_BUILD_SYSTEM_ROOT_IMAGE
This variable is always false or empty now, so clean up remaining
references to the variable.

Bug: 241346584
Test: Presubmit
Change-Id: I5dd3d356729d0bd2639633580bbb85f8b03582cf
2023-12-01 09:03:08 +00:00
Treehugger Robot
e5bfa38457 Merge "Revert "releasetools: Deprecate GKI build rules"" into main 2023-12-01 08:18:40 +00:00
Yi-Yo Chiang
92a517d75e Revert "releasetools: Deprecate GKI build rules"
This reverts commit faaede0f01.

Reason for revert: caused GSI sign failure b/311134118#comment23

Change-Id: Id794a8ab3369c907834a73cc88ea1640dc7492d1
2023-12-01 07:02:17 +00:00
Daniel Rosenberg
5514a5aaf6 Merge "Add support for different F2FS block sizes" into main 2023-11-30 00:24:46 +00:00
Dennis Song
4605c0ec76 Merge "Expose --avb_resolve_rollback_index_location_conflict" into main 2023-11-29 08:54:36 +00:00
Yi-Yo Chiang
6312ff59c1 Merge "releasetools: Deprecate GKI build rules" into main 2023-11-29 06:47:25 +00:00
Dennis Song
79e313972c Expose --avb_resolve_rollback_index_location_conflict
Expose this option at `merge_builds.py` so that this script
can also resolve the conflict AVB rollback index location
while building the vbmeta image.

Bug: 289368551
Test: TreeHugger
Change-Id: I404160ff5d7591cb6a81ed2d2267de1b0e6ac65f
2023-11-29 06:03:44 +00:00
Po Hu
ce867a65b5 Remove APEX from merged target-files package
It was a temp folder created for check_target_files_vintf.py.
But now it is wrongly archived by create_target_files_archive
in merge_target_files.py.
And then such package will cause CheckVintf exception in
ota_from_target_files.py because the APEX folder is created
again but already exists.
So use MakeTempDir to make it removed after check finished.

Test: merge_target_files & ota_from_target_files
Change-Id: Iab54afff2c346a582d5ee5228103fe091e016cd0
2023-11-29 08:54:44 +08:00
Kelvin Zhang
b84d2aa7d1 Sign OTA packages inside target_files during signing
Test: th
Bug: 293313353
Change-Id: Ifd5dd08153c5970dac8166808173f7dfbbb3411d
2023-11-28 10:08:38 -08:00
Yi-Yo Chiang
faaede0f01 releasetools: Deprecate GKI build rules
We no longer build GKIs from the platform tree.
These build commands were neither used nor maintained anymore, so clean
them up to reduce maintenance effort.

Bug: 229701033
Fix: 229701033
Test: presubmit
Change-Id: Ie882fccd864920289e48366e99a4ebd67e784d0d
2023-11-28 08:54:33 +00:00
Daniel Rosenberg
d9a520c5bd Add support for different F2FS block sizes
F2FS has been expanded to allow for different block sizes. Currently the
block size must be the same as page size, but this may change in the
future. This gives support for 16K while not restricting the ability to
choose a block size that isn't based on the page size later on.

Bug: 279820706
Test: Boot 16k F2FS device
Change-Id: I4e025d83a5e5ad03d89bc80b02fea390c2cbad69
2023-11-27 18:09:18 -08:00
Treehugger Robot
fdd0c0cdde Merge "Add --boot-image-dir-path arg" into main 2023-11-21 02:09:17 +00:00
Kelvin Zhang
68252b801a Move signing related options to a shared place
Move OTA signing flags to payload_signer.py so that we can re-use these
flags in multiple binaries.

Test: th
Bug: 293313353
Change-Id: I44f9910cee37c449397e174a5784f747ec9fb0d6
2023-11-14 12:41:58 -08:00
Kelvin Zhang
c68c6b9530 Allow ParseOptions to compose multiple option parsers easily
There are certain options which we need to share in multiple binaries,
for example, the signer options. Current options parsing function only
accepts 1 extra option handler, which is inflexible. Extend it to take a
list of extra option handlers.

Currently, to add a new CLI flag, caller must append the flag name to
`extra_long_opts`, then pass an extra option handler which can handle
that option. Define a new dataclass which contains both the CLI flag
name and the code to handle that flag for better composition.

Test: th
Bug: 293313353
Change-Id: I758db66dfd95934f5b2701454d97bfe7d37dc16d
2023-11-14 12:41:56 -08:00
Sundong Ahn
1930fd1f67 Add --boot-image-dir-path arg
The --boot-image-dir-path arg is added to use the boot.img in Android
Common Kernel (ACK) when merging.

Test: merge_target_files.py with --boot-image-dir-path and without
--boot-image-dir-path
Bug: 309745179

Change-Id: Ia164a9fc7e4761ad7efb871c0b6054195e9e4bda
2023-11-14 13:11:59 +09:00
Abhishek Nigam
1dfca46094 Revert "Move more non-AB code to separate files"
This reverts commit 513b86e5c2.

Change-Id: I6aae60642772a052404eb1773966b2e637864bbc
2023-11-08 22:29:18 +00:00
Abhishek Nigam
b148ac22f8 Revert "Fix non-AB ota generation failure"
This reverts commit 782b752889.

bug: 308932171
tested: locally generated and applied ota on nonab device
Change-Id: I67b26be01d6240b2cb332e13bc7c7959de128755
2023-11-08 22:29:18 +00:00
Kelvin Zhang
65c1162eb5 Generate incremental OTAs for 16K/4K boot option
Previously, the 16K/4K boot options OTAs are full OTAs, resulting in
file size of ~20MB each, and ~40M for both OTAs. To reduce the space
usage, use incremental OTAs instead.

Test: th
Bug: 302759296
Bug: 293313353
Change-Id: I61cc84c6c13f151dd6bc5ff37dd31daa5fb31abd
2023-11-06 12:37:49 -08:00
Kelvin Zhang
8c9166ae37 Convert partition size to int before CheckSize
Test: th
Change-Id: Ic312e40fdad99afd65b4fc259395b8952bb038c6
2023-11-01 10:14:13 -07:00
Kelvin Zhang
9f9ac4e8a0 Fix prebuilt boot image hermetic build
Break when we find a working salt, certain devices have boot image
containing both the kernel and ramdisk, resulting in the ramdisk being
used as salt. Since core/Makefile uses kernel as salt, update python
code to match.

Bug: 293313353
Test: th
Change-Id: I68b5a8c2f0747ddf68f775bc2a6c5db46eb7eb64
2023-11-01 10:14:13 -07:00
Kelvin Zhang
782b752889 Fix non-AB ota generation failure
Test: ota generation on pinot target
Bug: 308615740
Change-Id: I8b540516c0ba9e3acb23eebad8569a6dbf2ef9f3
2023-10-31 19:42:39 -07:00
Kelvin Zhang
513b86e5c2 Move more non-AB code to separate files
non-AB has different lifetime and development cycles from AB. To avoid
accidentally breaking each other when making changes, put non-AB code in
its own files.

Test: local non-AB OTA generation
Bug: 296485685

Merged-In: I0736ecef343834c746464666a454a8bfcd4a20ea

Change-Id: I0736ecef343834c746464666a454a8bfcd4a20ea
2023-10-30 23:43:15 +00:00
Kelvin Zhang
2ab6986dd9 Make non-ab code path compatible with extracted target files
Bug: 296485685
Test: ota_from_target_files -v -i non_ab.zip non_ab.zip ota.zip
Change-Id: I13ed54bfe7d14d2d1e87f45564da465a77bc2d1d
2023-10-27 11:01:27 -07:00
Kelvin Zhang
160762a276 Use deterministic salt for AVB footer of prebuilt boot img
When target specified a prebuilt boot.img, current build system will add
avb hash footer to it with a random salt. Use a deterministic salt
instead for more reproducible builds. To stay consistent with
non-prebuilt boot.img code path, we extract the kernel image from
prebuilt boot.img and uses sha256sum of kernel image as the salt.

Test: th
Bug: 293313353
Change-Id: I988999ddc4f18e0b8677b05a3165c847b6a11b52
2023-10-25 13:28:11 -07:00
Kelvin Zhang
9d74128194 Allow -v option to change log level
Setting logging level to 'NOTSET' would tell logging module to inherit
logging level from handlers. Since the verbosity of handlers is
controlled by -v option, this allows sign_apex.py to give verbose output
if -v is specified.

Test: th
Bug: 307191956
Change-Id: I9c1db107cf4eae80ab570d892b0465b3902298fd
2023-10-24 21:58:28 -07:00
Kelvin Zhang
d51332c82f Allow OEMs to bypass the on device SPL downgrade check
Certain OEMs bootloader implementation allow SPL downgrade. Allow on
device SPL downgrade check to be bypassed.

Test: th
Bug: 306271739
Change-Id: Iba991e12d36291e2d0547c94ef54c750d87f35fb
2023-10-23 10:10:33 -07:00
Thiébaud Weksteen
ed5abbdea4 Merge "Revert "Add sign_sepolicy_path for a binary to sign sepolicy.apex."" into main 2023-10-19 22:48:25 +00:00
Sandeep Dhavale
cc1a560241 Merge "Add support for specifying erofs blocksize" into main 2023-10-19 21:46:58 +00:00
Sandeep Dhavale
d7cc7e8f9a Add support for specifying erofs blocksize
This change allows us to specify erofs filesystem block size
which will be used to create erofs image.

Either it can be specified at board level using
BOARD_EROFS_BLOCKSIZE. Or per read only partition image
using BOARD_<PARTITION>IMAGE_EROFS_BLOCKSIZE.

Examples:
BOARD_EROFS_BLOCKSIZE := 16384 OR
BOARD_PRODUCTIMAGE_EROFS_BLOCKSIZE := 16384

Bug: 303316443
Test: m and file ${ANDROID_PRODUCT_OUT}/*.img

Change-Id: I9bf3d299ae02165d0bffc62f474a302408d8b9b9
Signed-off-by: Sandeep Dhavale <dhavale@google.com>
2023-10-18 08:23:36 -07:00
Thiébaud Weksteen
62865caf1b Revert "Add sign_sepolicy_path for a binary to sign sepolicy.apex."
This reverts commit e0a977affd.

Test: atest --host releasetools_test
Bug: 297794885
Change-Id: I951277e4aa3ae1f90474a1f7a036fc8693453c53
2023-10-18 11:41:25 +11:00
Wei Li
6437bae07c Add zip2zip as a data dependency of releasetools_common so it is built with releasetools_common and other python_binary_host modules depend on releasetools_common.
Bug: 303082945
Test: CIs
Test: "m dev_sign_bundle" on internal udc branch and check zip2zip is built too.
Change-Id: Iad5aa401de39516876714c854dc07aeba2f234e5
2023-10-17 21:05:53 +00:00
Kelvin Zhang
d7924a5311 Add missing AB partitions to img.zip
Add all AB partitions to img.zip, so that we can verify integrity of a
build by running avbtool on img.zip .

Bug: 274923679
Test: img_from_target_files on a pixel build. Unzip the generated img
zip, run avbtool verify_image --follow_chain_partitions --image
vbmeta.img, make sure result is successful.

Change-Id: I7d586a6f2ad87d8ebf87b788947055339b1121a2
2023-10-12 10:31:44 -07:00
Kelvin Zhang
7d1b29bd5d Run validation on user specified vabc_compression_param
During OTA generation, we check the vabc compression param specified by
target files, and override compression param to gz if the specified
compression method is unsupported. Run the same validation on
compression param passed by --vabc_compression_param as well.

Test: th
Change-Id: If855a2ee8c3c280c74efd5a67df229ab6c5709c5
2023-10-09 16:47:19 -07:00