211 Commits

Author SHA1 Message Date
Yumi Yukimura
c116d69cb1 releasetools: Pass non-sparse super_empty.img to update_dynamic_partitions()
* So that updater program can load dynamic partitions metadata from OTA
  when it fails to load from the device

Change-Id: I73687009a2cb243aba1bfec6f357c8ff25f27784
Signed-off-by: RITEFANG <1721985272@qq.com>
2025-02-05 20:41:13 +08:00
Kelvin Zhang
2427d9d402 Fix missing dir error for certain targets
Test: th
Bug: 364967828
Change-Id: I31f76533c3e3be63aa55f96be8fd697c4a629e65
2024-09-09 11:14:53 -07:00
Kelvin Zhang
1d9b3daad2 Make output of MakeVbmeta consistent
The order of items in `partitions` map is non-deterministic. To make
sure that builds are hermetic, sort the map before use.

Test: th
Bug: 332550989
Change-Id: I76c7037ef9f03904899339813819fcf985a7c893
2024-04-15 09:45:03 -07:00
Nariyasu Heseri
5ffefcc642 Ensure insertion order of Python dict
Add keys to the dict `partitions` in serial to have a deterministic
insertion order.
This guarantees the generated vbmeta.img is always the same on the same
input, fixing image consistency issues.

Reference:
https://android-review.googlesource.com/c/platform/build/+/2585397

Bug: 332550989
Test: Generate vbmeta several times, confirm identical result.
Change-Id: I39da5844045f497fdc89e6477a7e32ddacb70764
2024-04-12 07:57:37 +00:00
Jooyung Han
750aad5c32 Add all apexes to apex_info
Previously, META/apex_info.pb contained only /system/apex apexes. Now,
it has all apexes from all possible partitions.

The main purpose of this file is to caculate the decompressed apex size
when applying OTA. Hence it should have all apexes, not just system
apexes.

Bug: 320228659
Test: m dist # check META/apex_info.pb
Change-Id: I3428dc502e4fe3336d1fc5ca941f1fbc332985cd
2024-01-23 05:52:51 +09:00
Dennis Song
4aae62ee77 Resolve conflict AVB rollback index location
Add an `--avb-resolve-rollback-index-location-conflict`
option in merge_target_files. When this option is set,
the merge tool will resolve conflicting index locations
by assigning the smallest unused index location.

This is to support merging system and vendor target files
from two different targets. In this case, the two target
files may have conflict rollback index location because
they were built independently.

Test: atest releasetools_test
Test: validate_target_files *-target_files-*.zip
Test: merge_target_files &&
        add_img_to_target_files &&
        img_from_target_files &&
        flash device
Bug: 300604688
Change-Id: Ibd18ef2a9f3784157fe17966f5364c3c81c9bd9f
2023-10-04 06:45:15 +00:00
Treehugger Robot
d5474ac3da Merge "Add the option for custom_image to be AVB or NONAVB" into main 2023-08-01 19:08:06 +00:00
Ray-cy.lee
e97e0cb16b Add the option for custom_image to be AVB or NONAVB
Check BOARD_AVB_$(call to-upper,$(partition))_KEY_PATH to decide whether custom_image should sign AVB or not. If key path isn't set, the custom image will be excluded from AVB and copied to /IMAGES in target-files directly. This allows vendor to use custom_images flow packing unsigned image.
And to every non-avb custom partition, one image whose name is partition name must be added in its BOARD_<CUSTOM_PARTITION>_IMAGE_LIST.

BOARD_CUSTOMIMAGES_PARTITION_LIST := tvconfig
BOARD_TVCONFIG_IMAGE_LIST := \
    device/xxxx/yyyy/tvconfig.img \
    device/xxxx/yyyy/tvconfig_custom1.img

Test:
1) Build image, target-files, OTA package by m and m dist
2) Sign images by sign_target_files_apk.py

Fix: 285227850
Change-Id: I7477dafe023e4b168f0f08fb7aedd9e511a60e1b
2023-07-26 20:21:29 +08:00
Cole Faust
4f3dc8759f Remove BOARD_BPT* variables and the bpt parition table image
These variables appear to never be set.

Test: Presubmits
Change-Id: If4b89376f73204ad5780dff421f1216da255c3eb
2023-07-11 16:58:06 -07:00
Kelvin Zhang
c819b29f46 Always set a avb salt for hermetic build
When building images via `m` , build_image.py is invoked directly
without going through add_img_to_target_files. To ensure images built in
either way are identical, move uuid/salt computation to build_image.py,
so that the same uuid/salt will be used.

Bug: 281960439
Test: m installclean && m && m target-files-dir , maks sure images in
$OUT and $OUT/obj/PACKING/target_files_intermediates are identical

Change-Id: Icdab29df84f5a0ec7c080f99f9fdbdc3c9b10b90
2023-06-05 09:59:54 -07:00
Jeongik Cha
fa9090946e parallelize add_partition_calls only if output_zip is None
if output_zip isn't None, writing into zip file in parallel is not
thread-safe.

Bug: 281960217
Test: m dist
Change-Id: I10d68a4bb779cee244f40410ec95d38ca6040306
2023-05-12 15:14:58 +09:00
Jeongik Cha
03d014a03f parallelize add_partition_calls
it made total time equal to the longest image build

1m10s->30s in local build

Bug: 281960217
Test: m dist
Change-Id: I13d4f45d9b46b39292a014e3b4e1913365d89b7a
2023-05-12 08:43:44 +09:00
Treehugger Robot
67ca3c05c7 Merge "Fix duplicate super_empty.img" 2023-04-17 21:19:13 +00:00
Kelvin Zhang
aecdc18222 Fix duplicate super_empty.img
When calling add_img_to_target_files twice, super_empty.img will be
written twice, and resulting zip would contain duplicate entries.

Test: th
Bug: 278522974
Change-Id: I2419d0c4f8b04f876a8cc69a1517f59cab94d75d
2023-04-17 09:52:39 -07:00
Treehugger Robot
7b640806da Merge "Revert "Remove all ZIP64LIMIT hack"" 2023-04-15 01:48:58 +00:00
Kelvin Zhang
f92f7f046a Revert "Remove all ZIP64LIMIT hack"
This reverts commit 37a4290909.

Reason for revert: b/278156419

Change-Id: I67ea667619a9623be849d911993010ef0f0bfd88
2023-04-14 21:32:54 +00:00
Vincent Donnefort
6e861e9cf8 Fix missing dev nodes in init_boot.img ramdisk
Previously ("Add null and console dev nodes to the Android initramfs")
introduced into init_boot.img the /dev/console and /dev/null nodes.
Sadly, it missed changing the release tools which are also generating
images in their own way.

Bug: 254835242
Change-Id: I4d2893ac24c09cf0d117422cbe2bde66083e7118
2023-02-17 10:52:28 +00:00
Kelvin Zhang
b81b4e3621 Allow vendors to add custom vbmeta partitions
Currently we only have vbmeta_system and vbmeta_vendor. Add
BOARD_AVB_VBMETA_CUSTOM_PARTITIONS , which allows OEMs to define
their own vbmeta_*.img partitions. example: aosp/2381823

Bug: 263568805
Change-Id: Id671e2c3aee9ada90256381cce432927df03169b
2023-01-12 17:34:42 -08:00
Kelvin Zhang
37a4290909 Remove all ZIP64LIMIT hack
In the old days, we hacked values of ZIP64LIMIT to get around size
limitations of non-zip64 supported zip files. Now that we switched to
python3 + zip64, there's no point in keeping those hacks.

Test: th
Bug: 255683436
Change-Id: I913db33dad5503736c68a7a1f1321aa952019f60
2022-10-26 13:22:52 -07:00
Kelvin Zhang
1caead09f9 Use AVB footer to determine caremap
Care maps need to store the original image size, which excludes bytes
used by hash tree or FEC code.

We used to propagate original image size using the global OPTIONS
dictionary. This is bad coding practice, and also fragile because we
have to make sure what's stored in dictionary and what's on disk are
consistent. Instead, let's read the content of images on disk, and parse
the AVB footer. The AVB footer contains the ground truth original image
size.

Test: build OTA, make sure the care maps have valid range
Bug: 246504616
Change-Id: I9250b478ab34dda60578a6b8c23ae6d7a9385788
2022-10-10 13:12:50 -07:00
Kelvin Zhang
f294c87cfe Move GetCareMap from common.py to add_img_to_target_files.py
common.py is too big, break it down if possible

Test: th
Bug: 246504616
Change-Id: Ic1c1f11b53673419558137dffc2e233eeafff2c1
2022-10-07 10:42:00 -07:00
hungweichen
dd3fca09f6 Remove replace verity key args
We are removing VB support from release tools. This change aims to
remove the args related to replace verity key.

Bug: 242672222
Test: atest under build/make
Change-Id: I446a0b16e482c43542a1c0e41b24e80eb9fbc8e6
2022-08-24 06:37:12 +00:00
hungweichen
65ba375a8e Remove verity related props from build_image.py
We are removing VB related functionalities from release tools. This
change remove the verity related props in build_image.py, and also
remove VB1.0 bootimage builder, which will be unreachable without these
props.

Bug: 241044073
Test: atest under build/make
Change-Id: Ib3f1c100c2c6c9e63a64f49269619b76074a27fd
2022-08-19 05:57:56 +00:00
jiajia tang
e5ddfcd004 Fix potential issues if str has spaces
Uniform the split() function
str.split() will return a list split all spaces in str,
while str.split(' ') will return a list might contain ''
which might have potential issues.

Signed-off-by: jiajia tang <tangjiajia@xiaomi.com>
Change-Id: I0961659b140f800bdbe285f63bb4f02b8459ff8b
Signed-off-by: jiajia tang <tangjiajia@xiaomi.com>
2022-06-23 23:56:45 +00:00
Lucas Wei
03230250a3 vendor_kernel_boot: Update releasetools for vendor_kernel_boot
add_img_to_target_files.py will re-make image and build a new
signed vbmeta.img with new images in target-file.
We need to add vendor_kernel_boot into AVB signing list and make it
support that image.

Test: make dist -j110 and check vbmeta.img
Bug: 214409109
Signed-off-by: Lucas Wei <lucaswei@google.com>
Change-Id: Id07433f3dc33f95e2edd49de890f1e098cb9ef31
2022-04-21 00:27:27 +08:00
Lucas Wei
f57333f7fc Make build support vendor_kernel_boot image
Add vendor_kernel_boot image for vendors whose bootloader support
extra first stage booting kernel modules ramdisks. This benefit
kernel repo to build kernel-artifacts only image without Andorid
artifacts dependency.

Bug: 214409109
Signed-off-by: Lucas Wei <lucaswei@google.com>
Change-Id: If07218b86a7751b3d452a172610af960f5f9ec74
2022-03-16 23:13:19 +08:00
Treehugger Robot
ea586a8d71 Merge "Fix signing failure when no entry to convert to store" 2022-02-10 12:53:00 +00:00
Kelvin Zhang
70876143b5 Fix signing failure when no entry to convert to store
zip -d doesn't work when no entry names specified. So if the list of
entry is empty, just skip calling zip -d.

Bug: 218438888

Test: th
Change-Id: Ie4419b9d6fdc2780255c6f12f9f4d35e5c0a7b26
2022-02-09 16:07:29 -08:00
Iavor-Valentin Iftime
756b56125a Fix python3 errors: "TypeError: write() argument must be str, not bytes"
Bug: 186097910

Change-Id: I191c3230596026ee327c1403a6c72fbfa6fdb2d9
2022-02-09 16:02:33 +00:00
Kelvin Zhang
9d021e996b Fix small misuse of pass
The intention is to skip current iteration of for loop, so 'continue'
should be used instead of 'pass'

Test: th
Change-Id: Ic955beeaef74100981b351e2e7807c7fde538783
2022-02-07 16:38:09 -08:00
Kelvin Zhang
834f5d458f Store EROFS images uncompressed
Test: th
Change-Id: I53005bee139079afe25da231ca1d42f03fa010a5
2022-02-02 12:52:34 -08:00
Ramji Jiyani
13a4137929 core & tools: system_dlkm: add dynamic partition
Converts existing static partition support to a
dynamic partition.

Bug: 200082547
Test: TH
Signed-off-by: Ramji Jiyani <ramjiyani@google.com>
Change-Id: Ifd6d0c2a04e947b16f8b241e99cca594a1d315ae
2022-01-30 08:43:08 +00:00
Ramji Jiyani
bdef28f2ee releasetools: system_dlkm: add image to dist zip
Bug: 200082547
Test: Manual builds:
      1. Set BOARD_USES_SYSTEM_DLKM_PARTITION
      2. Set BOARD_PREBUILT_SYSTEM_DLKM_IMAGE to external image
      - Check that "m dist" put the system_dlkm image in
      out/dist/aosp_cf_x86_64_phone-img-eng.ramjiyani.zip
Signed-off-by: Ramji Jiyani <ramjiyani@google.com>
Change-Id: I8b7dd40e178db7fe542253bff19040095847fa57
2022-01-24 21:27:52 +00:00
Devin Moore
afdd7c74bc Update releasetools for new init_boot.img
This new init_boot.img contains the ramdisk that used to reside in the
boot.img file.

Test: set BOARD_PREBUILT_INIT_BOOT_IMAGE to an external init_boot.img
      - Check that "m" pulls in the init_boot.img to
      out/target/product/vsoc_x86_64/
      - Check that "m dist" adds the init_boot.img to
      aosp_cf_x86_64_phone-img-eng.devinmoore.zip
Test: atest --host releasetools_test
Bug: 203698939
Change-Id: If7ef2cf093e5e525529c7c44333c0f40f6ba0764
2021-12-20 20:54:23 +00:00
Bowgo Tsai
aba5c9eacd Fix aosp_x86_64-user build failure
Commit Ia982eb2ee3d1eb64db72c1836e433bcc53e71e3f removes boot-5.4.img
and leaves only a boot-5.10.img file, which makes 'boot_container'
become false. This leads to the failure in AddVbmetaDigest() as it will
search a 'boot.img' based on the 'boot' descriptor from the vbmeta.img.

Add a condition that if boot_images[0] is not 'boot.img' then the
build is also a boot container.

Bug: 199807830
Test: build aosp_x86_64-user
Change-Id: I4a9487b075186f0abf2ba74d3a1cf78072352a05
2021-09-27 14:11:14 +08:00
Daniel Norman
78554ea568 Build vendor images first when rebuilding with vendor_otatools.
Only supports rebuilding vendor.img and odm.img currently.

This allows "other" images (e.g. vbmeta, super_empty, ...) to
be rebuilt using the latest tooling.

Bug: 188491126
Bug: 192422274
Test: Sign a GRF R+S merged target_files package using:
      sign_target_files_apks \
        --vendor_otatools=<otatools.zip from R> \
	--vendor_partitions=vendor,odm \
	merged-target_files.zip \
	signed-target_files.zip
Change-Id: Ib93aea9f083ee9b722c31a42dcc780b5222053b8
2021-09-15 21:53:25 +00:00
Greg Kaiser
e086f7222e Revert "Rebuild vendor images in sign_target_files_apks with vendor otatools."
This reverts commit a1094e9f0c.

Reason for revert: Likely cause for b/199911935

Change-Id: Ib20400cc2f740a26fbbfb0da34fdf20ed7dc620e
2021-09-14 19:32:27 +00:00
Daniel Norman
a1094e9f0c Rebuild vendor images in sign_target_files_apks with vendor otatools.
This is needed until image compilation tooling is backwards compatible.

Bug: 188491126
Bug: 192253131
Bug: 192422274
Test: Sign a GRF R+S merged target_files package using:
      sign_target_files_apks \
        --vendor_otatools=<otatools.zip from R> \
	--vendor_partitions=vendor,odm,vbmeta \
	merged-target_files.zip \
	signed-target_files.zip
Change-Id: Iec208f544b56f8ed577344d8d6ca904773d8baab
2021-09-09 17:32:22 +00:00
Kelvin Zhang
25ab998d82 Fix a bug in computing streaming property of payload.bin
When computing the data offset of an entry in zip file, we used length
of extra field from central directory. That is correct most of the time
but wrong if the extra field in central directory has different length
than the one in local file directory. Since python's zipfile doesn't
provide an API to access local file header, we need to parse local file
header ourselves and extract length of extra field.

An incorrect offset will cause magic mismatch error from update_engine,
as update_engine expects to find uncompressed payload at the recorded
offset.

Test: th, partner verification
Bug: 191443484

Change-Id: Id670cd79b0bd65adffaaa5224ae4f8065d66b358
2021-07-28 11:40:18 -04:00
Tianjie
c3bf3d00a3 Write the vbmeta digest to the output zipfile also
During signing, we write the entry directly to the output zip, intead
of a temp directory. Add the logic to write vbmeta_digest.txt to output
zipfile too. So the digest file will show up in the signed target files.

Bug: 189926233
Test: add_img_to_target_files -a <target-files.zip>
Change-Id: Ibf28a8f97512bda8c8c695e06190e1fb6573c53e
2021-07-14 16:03:08 -07:00
Tianjie
a5fca03e0a Amend the apex info for ota package
We have already logged the compressed apexes in the target-files.
Because we want to support the apex metrics during OTA update, also
include the uncompressed apexes in the META/apex_info.pb.

For incremental OTA packages, include the source apex version for
each apex package as well.

Bug: 190244686
Test: unit test
Change-Id: I5cf2647c56c4feb5517f9a81aa1e9abc52515bf1
2021-06-07 20:33:46 -07:00
Tianjie
fdda51d2ae Calculate the runtime ro.build.id in ota scripts
If the build prop ro.build.id isn't set at build time, init will
set it at runtime. The logic is appending the vbmeta digest to
the ro.build.legacy.id.

Make the same change in ota scripts, so the correct build fingerprint
will be saved in the ota metadata.

Bug: 186786987
Test: generate an OTA, check the metadata
Change-Id: I278f59c41c1f98d4cbea749e5d9e4eaf7a6b9565
2021-05-10 11:35:48 -07:00
Tianjie
bbde59f9eb Calculate the vbmeta digest when building images
Calculate the vbmeta digest if the device builds vbmeta image. The
digest will used later to determine the build fingerprint in new
format.

One sample usage is the ota package generation, where we put the
build fingerprint in the ota metadata. But we don't have the runtime
vbmeta digest provided the bootloader.

Bug: 186786987
Test: unit tests
Change-Id: If572e2b973e295a6c95a9e23a65bb20b3afbf1b0
2021-05-05 18:04:51 +00:00
Daniel Norman
e9af70ad68 Performs basic APEX validation in the merged target files package.
Uses apex_utils.GetApexInfoFromTargetFiles to find and parse APEX files
in the target files partition dirs. Raises an error on failure to parse
or duplicate package names.

Bug: 177225446
Test: releasetools_test
Test: Create a merged build that provides the VNDK APEX on both vendor
      and system. Observe failure.
Change-Id: I1356e263b7b32d6063129e079f3ba7ab4ff132a7
2021-04-19 22:31:01 +00:00
Kelvin Zhang
2732413fa2 Moves care map generation logic to common.py, so it can be reused.
Test: th
Bug: 182296208
Change-Id: I045be7cb208412431d6ab1c26e31d38d2285738b
2021-04-13 11:42:26 -07:00
Daniel Norman
2d7989a635 Revert "Generate care map after merging target_files"
This reverts commit c184fa1887.

Reason for revert: b/184541365

Change-Id: Icbcbb6deb92863f4a468b9fa54f18d824a6f0939
2021-04-05 17:40:47 +00:00
Kelvin Zhang
c184fa1887 Generate care map after merging target_files
Test: th
Bug: 182296208
Change-Id: I12c0732961ec1c4d18f5cc4d136c98babba23d82
2021-04-01 17:04:12 -04:00
Yo Chiang
e86bab474b Revert^2 "Add option to allow product makefiles to skip building s..."
This reverts commit ccfea17fb7.

Reason for revert: Original bug was resolved by updating branch config

Change-Id: I2327092261a2147fa8f2be3d878db04228e65511
2021-03-25 10:12:28 +00:00
Tianjie Xu
ccfea17fb7 Revert "Add option to allow product makefiles to skip building s..."
Revert submission 1644988-nextgen-release-img-zip-cleanup

Reason for revert: b/183654818
Reverted Changes:
I2474e5fd6:Nextgen release: Don't build super_empty.img
I549439528:Add option to allow product makefiles to skip buil...

Change-Id: I711d1c288eed17686b048144d8d41c0fdd5cd8e1
2021-03-25 05:00:12 +00:00
Yi-Yo Chiang
ba959a5ff1 Add option to allow product makefiles to skip building super_empty.img
This change doesn't change the condition for building super_empty.img,
it just add a toggle PRODUCT_BUILD_SUPER_EMPTY_IMAGE that product
makefiles can use to skip building super_empty.img.
Products that don't use super_empty at all, for example GSI, can set
this option to ensure the super_empty.img is not built.

Bug: 183068624
Test: "m dist" on GSI and check the build artifacts under OUT and DIST
  directories, and check the contents of *-img-*.zip
Change-Id: I54943952873d2d297fd9d18cbe14742bc12ae9c6
2021-03-24 05:35:10 +00:00