Commit Graph

2800 Commits

Author SHA1 Message Date
Vincent Donnefort
f74012fd97 Merge "Fix missing dev nodes in init_boot.img ramdisk" am: 65af9993b3 am: 07738c42b2
Original change: https://android-review.googlesource.com/c/platform/build/+/2442241

Change-Id: I09c97d6d099179f7760a7ab17467d08b19710455
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-02-20 11:36:17 +00:00
Vincent Donnefort
6e861e9cf8 Fix missing dev nodes in init_boot.img ramdisk
Previously ("Add null and console dev nodes to the Android initramfs")
introduced into init_boot.img the /dev/console and /dev/null nodes.
Sadly, it missed changing the release tools which are also generating
images in their own way.

Bug: 254835242
Change-Id: I4d2893ac24c09cf0d117422cbe2bde66083e7118
2023-02-17 10:52:28 +00:00
Treehugger Robot
aa651b93b0 Merge "Use zip2zip to copy zipfiles" am: a59e5b7a29 am: 8c53b320ed
Original change: https://android-review.googlesource.com/c/platform/build/+/2439632

Change-Id: Iec87d556be8d028464c81d377d65241fca254057
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-02-16 23:55:10 +00:00
Kelvin Zhang
c8ff84b219 Use zip2zip to copy zipfiles
ZipDelete() works by copying every non-deleted entry to a new zipfile.
Current implementation uses python's zipfile module to perform entry
copying, which is inefficient, as every entry must be decompressed and
then re-compressed. Instead, use zip2zip which avoid re-compression.

Improvement: deleting META/dynamic_partitions_info.txt from
raven-target_files-9465001.zip improved from 500+ seconds to 13 seconds.

Change-Id: I0548255bc29380303314763f6d81e74bf3dbb76e
2023-02-15 16:55:48 -08:00
TreeHugger Robot
8bdaa29ab1 Merge "Support SHA minSdkVersion in APK in APEX" into tm-dev am: 240c63fba3 am: 73214a06a4 am: 7b9f430cec
Original change: https://googleplex-android-review.googlesource.com/c/platform/build/+/21337794

Change-Id: I9a9f1c7a09d71defc019cbb0ce877b2730ae181e
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-02-11 04:44:58 +00:00
Paul Duffin
a03f1265fb Support SHA minSdkVersion in APK in APEX
Bug: 266903788
Test: rm -fr out/dist
      UNBUNDLED_BUILD_SDKS_FROM_SOURCE=true UNBUNDLED_BUILD_TARGET_SDK_WITH_API_FINGERPRINT=true TARGET_BUILD_APPS="com.google.android.rkpd" vendor/google/build/mainline_modules_bundles.sh
      # Without this change it fails, with this change it passes at
      # which point run the following.
      mkdir contents
      cd contents
      unzip ../out/dist/dev_keys_signed/com.google.android.rkpd/com.google.android.rkpd.apks
      deapexer extract standalones/standalone-x86.apex apex-contents
      aapt2 dump badging apex-contents/priv-app/rkpdapp.google@*/rkpdapp.google.apk  | grep Version
      # The sdkVersion and targetSdkVersion should both be set to SHA
      # versions.
Change-Id: I9ca54affd30f0a6cf561f1274b991d39e7cf1bce
(cherry picked from commit 730d0c437a)
Merged-In: I9ca54affd30f0a6cf561f1274b991d39e7cf1bce
2023-02-08 17:28:29 +00:00
Dennis Shen
0dfb3aea75 Fix signing test failure
BUG: b/266137486
TEST: manual run target file signing pass with change
Change-Id: I4b8127937ba726560a69c79619a7f1a2882c9972
2023-01-31 13:28:09 +00:00
Hongguang Chen
1a7323390b Overwrite prop.default in all ramdisk fragments
default.prop can be linked to prop.default. To rewrite the fingerprints,
the original prop.default should be overwritten.

Bug: 264853953
Test: sign_target_files_apks, check artifacts
Change-Id: I8d22b8391dbfe13e74880ffad0619ef7fc4e20e0
2023-01-29 19:43:04 +00:00
Kelvin Zhang
e8ce38450f Disable imgdiff for non-AB OTA
imgdiff assumes that libz produces identical output across different
platforms, which is sometimes not the case. This causes incremental OTA
on non-AB targets to fail ocassionally. For stability, disable imgdiff.

Test: th
Bug: 265895599
Change-Id: I92d09a50762ec69a79caa37ed4d6e0db1ec8a47a
2023-01-23 20:43:09 -08:00
Kelvin Zhang
f80e886273 Fix non-AB OTA signing issue
After aosp/2201756 , caller must explicitly pass signing key as a
parameter to FinalizeMetadata . We forgot to update non_ab.py ,
therefore causing non-AB OTAs to be unsigned.

Test: th
Bug: 265895599
Change-Id: Ia54b1105f37b6574742b9b55d715fc9c61c223eb
2023-01-20 10:18:11 -08:00
Baligh Uddin
66dbfde748 Switch check_targetfiles_vintf to use the correct blkid binary.
BUG: 264426585
Test: TH
Change-Id: Iefe86e78c3e607ac27a5197d45fa832fa281c09d
2023-01-20 00:56:54 +00:00
Treehugger Robot
b80d9d6c33 Merge "Allow vendors to add custom vbmeta partitions" 2023-01-14 00:23:02 +00:00
Treehugger Robot
1ecedfc353 Merge "Allow otatools to deduce partial partition list from input" 2023-01-13 18:00:34 +00:00
Kelvin Zhang
b81b4e3621 Allow vendors to add custom vbmeta partitions
Currently we only have vbmeta_system and vbmeta_vendor. Add
BOARD_AVB_VBMETA_CUSTOM_PARTITIONS , which allows OEMs to define
their own vbmeta_*.img partitions. example: aosp/2381823

Bug: 263568805
Change-Id: Id671e2c3aee9ada90256381cce432927df03169b
2023-01-12 17:34:42 -08:00
Kelvin Zhang
30669e6c29 Ovewrite build props in all ramdisk fragments
Under some build configuration, there could be build prop stored under
path VENDOR_BOOT/RAMDISK_FRAGMENTS/recovery/RAMDISK/default.prop .
When signing, we must overwrite all build props, or device would display
incorrect build fingerprint.

Bug: 264853953
Change-Id: Id3b176b4ce4efa348ecfb3c9848f72273c5ccca8
2023-01-10 21:09:14 -08:00
Kelvin Zhang
9f0379bd60 Fix VB 1.0 failure due to openssl output format change
Openssl changed output format, the 'keyid:' prefix is removed, hence old
regex is unable to extract key id.

Bug: 262902909
Change-Id: Iea5b6fffed7c27855d87e35292f07a56686e4197
Merged-In: I446a0b16e482c43542a1c0e41b24e80eb9fbc8e6
Merged-In: Iea5b6fffed7c27855d87e35292f07a56686e4197
(cherry picked from commit b7d70a2cb1)
2023-01-05 12:02:13 +00:00
Kelvin Zhang
a10528c7a8 Allow otatools to deduce partial partition list from input
Different devices/builds might have different constraints on what
partitions should be updated together in a partial OTA. Add a convenient
feature to auto detect.

Test: generate ota with --partial , make sure output OTA contains only 4
partitions

Change-Id: Id79e4bc4c5fb9d1225cb2aa7c1823afccc0b61f2
2023-01-03 16:04:49 -08:00
Kelvin Zhang
37bc304a3d Allow build_image.py to build vbmeta partition
Cuttlefish needs to rebuild vbmeta at runtime, instead of duplicating
logic in cuttlefish tooling, re-use these logic in release tools.

Test: build_image $OUT misc_info.txt /tmp/vbmeta.img /tmp/otatols
Bug: 149866755
Change-Id: I2050f9e2dd7eed9b81a49a8442ec199147041d20
2022-12-20 10:59:58 -08:00
Jooyung Han
8af44a923f Refactor: remove --apex-info-file arg to checkvintf
Now, checkvintf finds input file from dirmaps. We don't need to pass
around --apex-info-file.

dump_apex_info also generates files under /apex without an explicit
--out_file argument.

This makes "make" know less about coupling between dump_apex_info
and checkvintf.

Bug: 260310732
Test: manually run check_target_files_vintf -v target-files
Change-Id: Ibfe31978e983e13df35c32521913d5cbc89dc1d1
2022-11-28 15:01:30 +09:00
Hongguang Chen
425f7d1cb3 Support chained init_boot partition signing
Bug: 256048561
Bug: 256237041
Test: sign_target_files_apks -d certs --avb_init_boot_algorithm \
        SHA256_RSA4096 --avb_init_boot_key init_boot_rsa4096.pem \
	xxx.zip signed.zip (Check signed.zip/META/misc_info.txt)
Change-Id: I65fc7913089ae318d90df55a533d3e2ebd93b029
(cherry picked from commit 0d6b727e03)
Merged-In: I65fc7913089ae318d90df55a533d3e2ebd93b029
(cherry picked from commit acf1b3afe2)
2022-11-15 22:30:54 +00:00
Treehugger Robot
9e0472e9c7 Merge "Fix recovery OTA error on merged targets" 2022-11-15 18:33:53 +00:00
Treehugger Robot
5cf8eef25e Merge "Fix 'expected str instance, bytes found' error when compute patches" 2022-11-14 19:14:22 +00:00
luoqiangwei1
3e3456bb72 Fix 'expected str instance, bytes found' error when compute patches
When building an incremental OTA package, if the compress_target=True
parameter is passed in when calculating a block, an exception will occur
in the subsequent join operation.

The reason is that Python 3 no longer allows str type join bytes type.

Bug: None
Test: Build Incremental OTA package

Change-Id: I6d556f9905b7ab75b70d3785334d71d5a6e5479b
Signed-off-by: luoqiangwei1 <luoqiangwei1@xiaomi.com>
2022-11-11 00:26:19 +08:00
Kelvin Zhang
fa40c04cba Fix recovery OTA error on merged targets
Currently, when merging target files, system side's update_engine
version info is copied into the merged target_files. This causes
recovery OTA failure, because update_engine_sideload daemon in recovery
is from vendor side. To fix it, use the older version of update_engine
config when merging target files.

Test: th
Bug: 257426678
Change-Id: I5d3304366749995c99a61ca0645a22ee047f1465
2022-11-09 14:43:17 -08:00
Dennis Shen
a8d1143beb Update deapexer call to explictly use blkid
To support erofs apex extract via deapexer, we need blkid to tell the filesystem type of underlying payload image. If it is ext4, debugfs_static will be used, if it is erofs, then we should use fsck.erofs. Thus we now need explicit blkid input.

Also, remove dependency on fsck.erofs for other deapexer calls. Only extract deapexer call needs blkid and fsck.erofs.

BUG: b/255963179, b/240288941
Change-Id: I8cea0f2def664f9cabf8b14c9a7ecc47bbddfbdd
2022-11-09 18:22:14 +00:00
Treehugger Robot
8f6474af5a Merge "Support chained init_boot partition signing" 2022-11-08 02:41:53 +00:00
Hongguang Chen
0d6b727e03 Support chained init_boot partition signing
Bug: 256048561
Bug: 256237041
Test: sign_target_files_apks -d certs --avb_init_boot_algorithm \
        SHA256_RSA4096 --avb_init_boot_key init_boot_rsa4096.pem \
	xxx.zip signed.zip (Check signed.zip/META/misc_info.txt)
Change-Id: I65fc7913089ae318d90df55a533d3e2ebd93b029
2022-11-07 15:48:23 -08:00
Dennis Shen
f58e548943 add EROFS apex support.
Signing code uses deapexer to expand payload image, deapexer currently
only supports ext4 payload image expansion using debugfs or
debugfs_static. To support EROFS, need to supply deapexer with
fsck.erofs native binary for EROFS payload expansion.

cherrypick of ag/20154950BUG: b/195515562, b/240288941
Change-Id: Ibf707989f2502bfcf112202a3a6fdb74b6a8dbbc
2022-11-07 20:40:41 +00:00
Dennis Shen
44b714b7ca Update deapexer extract call with more input
BUG: b/255963179, b/240288941
Change-Id: Id7f5eb20e4c6f3c3e793860389f4be48f4f3938f
2022-11-03 17:39:04 +00:00
Treehugger Robot
5667fe0a74 Merge "Allow RO build only for readonly mount points" 2022-10-31 22:58:32 +00:00
Treehugger Robot
a2faaa8c45 Merge "Remove all ZIP64LIMIT hack" 2022-10-27 22:44:22 +00:00
Treehugger Robot
1fca7c5e7f Merge "Enable zip64 support in build image script" 2022-10-27 22:31:00 +00:00
Jaegeuk Kim
551a2e66c3 Allow RO build only for readonly mount points
Should support RW for all other partitions.

Bug: 255454067
Change-Id: I9d87cf39195f8da0f6fe4153c703ba6a372aa2ea
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
2022-10-27 09:46:03 -07:00
Kelvin Zhang
37a4290909 Remove all ZIP64LIMIT hack
In the old days, we hacked values of ZIP64LIMIT to get around size
limitations of non-zip64 supported zip files. Now that we switched to
python3 + zip64, there's no point in keeping those hacks.

Test: th
Bug: 255683436
Change-Id: I913db33dad5503736c68a7a1f1321aa952019f60
2022-10-26 13:22:52 -07:00
Kelvin Zhang
1de1788dda Enable zip64 support in build image script
Images are getting bigger each year, we need zip64 support to store >4GB
files in .zip archive.

Test: th
Bug: 255683436
Change-Id: I9adbefa5ec72e357b02e46d6e629c054d466ddd7
2022-10-26 12:30:29 -07:00
Po Hu
4d2f64b6f0 Fix boot fail due to missing /apex in images
Do not move or rename any files in merge_target_files.py's temporary directory. They will be used to generate images further.

Bug: 254760332
Change-Id: I973a49b29234d49f5254a1403a0e236e69425d74
Test: check SYSTEM/apex in target-files.zip generated by merget_target_files.py
2022-10-26 02:34:12 +00:00
Treehugger Robot
775187f921 Merge "Fix ota build error when using boot variable files" 2022-10-21 20:43:21 +00:00
Kelvin Zhang
eb147e051e Fix ota build error when using boot variable files
When performing a deep copy of info dict, input_file is serialized as
filename. Make ExtractFromInputFile support reading from filename of a
ZipFile.

Test: build OTA with partner provided target_files
Bug: 253549364
Change-Id: I366a076c32d638d61b83a1df9bf864807cf5257c
2022-10-21 10:53:21 -07:00
GeQi
3fa9e32563 add a test case to validate deepcopy on PartitionBuildProps
Test: atest --host releasetools_test
Bug: 253549364
Change-Id: I187554ae33041ed090ee352167ef8deb51d754bc
2022-10-20 18:39:08 +08:00
Treehugger Robot
eb5f9e2bef Merge "check_target_files_vintf: Support flattened apexes" 2022-10-20 03:19:24 +00:00
Kelvin Zhang
5ef25191f7 Fix picle error on ota_from_target_files
When trying to make a deep copy of PartitionBuildProps, execution fails
with picle error because PartitionBuildProps stores a ZipFile object,
which stores a lock object, which doesn't support pickling.

To fix it, pickle ZipFile object as a str to its path.

Test: generate ota with boot variable files
Bug: 253549364
Change-Id: I92c5d78ce65045495203c85862922d33d886e9ea
2022-10-19 11:25:22 -07:00
Jooyung Han
eb1182115b check_target_files_vintf: Support flattened apexes
Flattened apexes are extracted and copytree()'d to /apex directory to
simulate "activation" on device.

Bug: 242314000
Bug: 239055765
Test: m otapackage # target with "flattened" apexe
Change-Id: I90cc37b5f64ebad1e15f6fc5fa245d0f5955095d

Change-Id: I7e8c3fdf8e4620344d23cee0de509e574206ad9b
2022-10-18 07:46:14 +09:00
Kelvin Zhang
2e1ff6e262 Remove unnecesasry use of ZipClose()
ZipClose() was introduced to work with large zip without switching to
ZIP64. It is a hacky patch. Now we have moved to python3 and zip64
completely, no need to keep using it.

Test: th
Bug: 246504616
Change-Id: I1ff15171bb26887b819f655e32817627ecad1132
2022-10-17 10:04:45 -07:00
Jooyung Han
62283b9649 Skip flattened apexes while check_target_files_vintf
Flattened apexes should be handled differently. For now just skip
flattened apexes to fix the build breakage.

Bug: 253669548
Bug: 253700117
Bug: 239055765
Test: m otapackage for targets with "flattened apexes"
Change-Id: I102a961fc1365f42881ca9ee6546e71246792d2a
2022-10-17 10:38:41 +09:00
Wei Li
ddfc1ebf14 Merge "Change function ZipDelete to use Python module zipinfo instead of command 'zip'." 2022-10-16 05:55:00 +00:00
Wei Li
8895f9ecea Change function ZipDelete to use Python module zipinfo instead of command 'zip'.
'zip' is a host command, and it can not be provided as a dependency when some release tools are used in Bazel rules.

Test: atest --host releasetools_test
Bug: 243748589

Change-Id: Ie5f42eadbd316ccd018b19194c466a908971af82
2022-10-14 18:31:56 -07:00
Rob Seymour
0a00cd27d2 Merge "Add APEX data to checkvintf for compatibility check." 2022-10-14 18:04:46 +00:00
Kelvin Zhang
1caead09f9 Use AVB footer to determine caremap
Care maps need to store the original image size, which excludes bytes
used by hash tree or FEC code.

We used to propagate original image size using the global OPTIONS
dictionary. This is bad coding practice, and also fragile because we
have to make sure what's stored in dictionary and what's on disk are
consistent. Instead, let's read the content of images on disk, and parse
the AVB footer. The AVB footer contains the ground truth original image
size.

Test: build OTA, make sure the care maps have valid range
Bug: 246504616
Change-Id: I9250b478ab34dda60578a6b8c23ae6d7a9385788
2022-10-10 13:12:50 -07:00
Kelvin Zhang
f294c87cfe Move GetCareMap from common.py to add_img_to_target_files.py
common.py is too big, break it down if possible

Test: th
Bug: 246504616
Change-Id: Ic1c1f11b53673419558137dffc2e233eeafff2c1
2022-10-07 10:42:00 -07:00
Rob Seymour
dc4e0f2ee0 Update check_target_files_vintf to check the last line of stdout.
More output is now produced on stdout by checkvintf.
Update logic to compare against the last line for incompatible.

Fixes the following tests:
test_check_target_files_vintf.CheckTargetFilesVintfTest#test_CheckVintf_kernel_incompat
test_check_target_files_vintf.CheckTargetFilesVintfTest#test_CheckVintf_matrix_incompat
test_check_target_files_vintf.CheckTargetFilesVintfTest#test_CheckVintf_sku_incompat

Bug: 251221029
Test: atest releasetools_test
Change-Id: I592acf9a3868e2a60e203b2201e7fadcbb0e7f04
2022-10-05 19:52:54 +00:00