Commit Graph

537 Commits

Author SHA1 Message Date
Kelvin Zhang
eb147e051e Fix ota build error when using boot variable files
When performing a deep copy of info dict, input_file is serialized as
filename. Make ExtractFromInputFile support reading from filename of a
ZipFile.

Test: build OTA with partner provided target_files
Bug: 253549364
Change-Id: I366a076c32d638d61b83a1df9bf864807cf5257c
2022-10-21 10:53:21 -07:00
Kelvin Zhang
5ef25191f7 Fix picle error on ota_from_target_files
When trying to make a deep copy of PartitionBuildProps, execution fails
with picle error because PartitionBuildProps stores a ZipFile object,
which stores a lock object, which doesn't support pickling.

To fix it, pickle ZipFile object as a str to its path.

Test: generate ota with boot variable files
Bug: 253549364
Change-Id: I92c5d78ce65045495203c85862922d33d886e9ea
2022-10-19 11:25:22 -07:00
Wei Li
8895f9ecea Change function ZipDelete to use Python module zipinfo instead of command 'zip'.
'zip' is a host command, and it can not be provided as a dependency when some release tools are used in Bazel rules.

Test: atest --host releasetools_test
Bug: 243748589

Change-Id: Ie5f42eadbd316ccd018b19194c466a908971af82
2022-10-14 18:31:56 -07:00
Kelvin Zhang
1caead09f9 Use AVB footer to determine caremap
Care maps need to store the original image size, which excludes bytes
used by hash tree or FEC code.

We used to propagate original image size using the global OPTIONS
dictionary. This is bad coding practice, and also fragile because we
have to make sure what's stored in dictionary and what's on disk are
consistent. Instead, let's read the content of images on disk, and parse
the AVB footer. The AVB footer contains the ground truth original image
size.

Test: build OTA, make sure the care maps have valid range
Bug: 246504616
Change-Id: I9250b478ab34dda60578a6b8c23ae6d7a9385788
2022-10-10 13:12:50 -07:00
Kelvin Zhang
f294c87cfe Move GetCareMap from common.py to add_img_to_target_files.py
common.py is too big, break it down if possible

Test: th
Bug: 246504616
Change-Id: Ic1c1f11b53673419558137dffc2e233eeafff2c1
2022-10-07 10:42:00 -07:00
Tim Zimmermann
a06f8330f6 releasetools: Support erofs on non-dynamic partitions
Change-Id: Ibeb228132d3c0cc1d7407c2a40498b72580c81d5
2022-10-01 13:03:36 +02:00
Sorin Basca
05085835ea Increase javac heap to 4096M
Bug: 246361502
Test: TH
Change-Id: Ic2d6f991ad65d13a83319af07944069d1f511783
2022-09-14 11:33:22 +01:00
Kelvin Zhang
197772f069 Add a tool to merge two partial OTAs
Some partners have large number of products that share common
images. Generating OTAs for all these products waste a lot of CPU
cycles, because we waste time diffing the same images multiple times.
To mitigate this, add a tool for merging partial OTAs. Partners can
generate a partial OTA for common images, and re-use the generated OTA
by merging with each product specific OTA.

Bug: 227848550
Test: Run merge_otas.py on 4 OTAs, install the generated OTA

Change-Id: Ie38d522cdc9b89b1123b3d8190516c3648f99788
2022-09-07 20:35:46 +00:00
Hung-Wei Chen
be469cf909 Merge changes Icc298256,I9268cb11,I446a0b16,I347447eb,Iaae39e4c, ...
* changes:
  Remove HashTreeInfo from verity_utils.py
  Remove verity keys from info_dict
  Remove replace verity key args
  Remove unused args from common.py
  Clean up unused code for VB in verity_utils.py
  Remove verity related props from build_image.py
2022-08-30 07:10:23 +00:00
hungweichen
cc9c05d5b8 Remove HashTreeInfo from verity_utils.py
We are removing VB related functionalities from release tools. This
change removes the hashtreeinfo and generator in verity_utils.py. It
also clears the function args using it.

Bug: 241044073
Test: atest under build/make
Change-Id: Icc298256be6917ffbd40cf4735f8e8092b541c9f
2022-08-25 03:30:06 +00:00
hungweichen
22e3b01d7b Remove verity keys from info_dict
We are removing VB support from release tools. This change aims to
remove verity keys in info_dict generated by common.LOADINFODICT

Bug: 242672222
Test: atest under build/make
Change-Id: I9268cb113c2c0bd11ce85b8485e5a4605bdbdca0
2022-08-24 06:37:12 +00:00
Melisa Carranza Zuniga
7ef1379927 Adding output_path arg to sign_sepolicy_path call
Bug: b/217570541
Test: mma and call sign_apex or sign_target_files_apks
Change-Id: I0fcab6e8a40739406a158a687acea448fd205f5d
2022-08-23 19:09:12 +02:00
hungweichen
49447913f6 Remove unused args from common.py
We are removing support of VB from release tools. This change aims to
remove args that no longer be used.

Bug: 242672222
Test: atest under build/make
Change-Id: I347447ebf1e589f8d3c4c8254f2940911c0be7f1
2022-08-19 06:04:06 +00:00
Yong Ma
253b106f95 set up the salt for partitions without build.prop
BUG:242954151

Test: make target-files-package, then check the two sha1 are same:
      1. sha1 of IMAGES/boot.img
      2. expected sha1 of boot in VENDOR/bin/install-recovery.sh

Test: make target-files-package; unzip target_files.zip IMAGES/\*`;
      `zip -d arget_files.zip IMAGES/\*`
      `add_img_to_target_files -a target_files.zip`.
      Verify that the rebuilt images are identical to the deleted ones.

Change-Id: I5906ac1696b6403674fd8330b1569aed53510945
2022-08-19 02:10:01 +00:00
Melisa Carranza Zúñiga
051c9a15af Merge "Add sign_sepolicy_path for a binary to sign sepolicy.apex." 2022-06-28 12:26:50 +00:00
Melisa Carranza Zuniga
e0a977affd Add sign_sepolicy_path for a binary to sign sepolicy.apex.
Bug: b/217570541
Test: sign_apex --container_key=testdata/testkey
--payload_key=testdata/testkey_RSA4096.key
--codename_to_api_level_map=S:31,T:32,UpsideDownCake:33
-e SEPolicy-33.zip=testdata/testkey com.android.sepolicy.apex
signed_sepolicy.apex

Change-Id: Ifdd1c6e7db07cbc4674450ab0e2226494c98605a
2022-06-27 11:03:26 +02:00
jiajia tang
e5ddfcd004 Fix potential issues if str has spaces
Uniform the split() function
str.split() will return a list split all spaces in str,
while str.split(' ') will return a list might contain ''
which might have potential issues.

Signed-off-by: jiajia tang <tangjiajia@xiaomi.com>
Change-Id: I0961659b140f800bdbe285f63bb4f02b8459ff8b
Signed-off-by: jiajia tang <tangjiajia@xiaomi.com>
2022-06-23 23:56:45 +00:00
Melisa Carranza Zúñiga
da308bf773 Revert "Merge "Adding sepolicy sign params to sign_target_files_apks." am: fb042449e0 am: bd8c313275 am: a2db49becd"
Test: sign_target_files_apks target_files.zip target_out.zip

This reverts commit 6e6c22c70e.

Reason for revert: Updateable SEPolicy is punted.

Change-Id: I99524d13811028a4036aa85c74cf10974fe08165
Merged-In: I99524d13811028a4036aa85c74cf10974fe08165
2022-05-31 09:55:22 +00:00
Kelvin Zhang
a9a87ec204 Disable verity for VABC updates on android R
Bug: 230277030
Test: th
Change-Id: I2872c490c852fb7d3784721e191ece80b4b1fd58
2022-05-04 16:45:14 -07:00
Treehugger Robot
98050a4e97 Merge "Expose GetRamdiskFormat and use in ota_utils.py" 2022-05-03 08:27:35 +00:00
TJ Rhoades
6f488e9e33 Expose GetRamdiskFormat and use in ota_utils.py
Some OTA files may have additional OTA props; when this is being computed within
ota_utils.py -> ComputeRuntimeBuildInfos, the ramdisk type needs to be passed into
PartitionBuildProps.FromInputFile as not all ramdisk may still use gzip. This check
is already defined in common.py, so this change also exposes the function GetRamdiskFormat.

Bug: 231075507

Signed-off-by: TJ Rhoades <tjr@microsoft.com>
Change-Id: If1b93b887990f0d90df2c6003122821c1e66a1ac
2022-05-02 13:19:49 -07:00
Kelvin Zhang
6a683ce02b Merge identical key/val pairs in dynamic partition info
We might add new values to dynamic_partition_info.txt, so some kind of
generic fallback mechanism is needed. If keys are different, we need to
decided on a case-by-case basis which side takes precedence. For
example, Virtual AB requires vendor support, so vendor side takes
precedence. VABC on T+ devices are implemented entirely in system, so
system/framework side takes precedence.

Bug: 230876542
Test: th
Change-Id: I67747368547d3ef3e29ad64f8f818ef4c5896246
2022-05-02 12:19:45 -07:00
Lucas Wei
2d5ecef1b3 Merge "vendor_kernel_boot: Update releasetools for vendor_kernel_boot" 2022-04-21 03:07:15 +00:00
Lucas Wei
03230250a3 vendor_kernel_boot: Update releasetools for vendor_kernel_boot
add_img_to_target_files.py will re-make image and build a new
signed vbmeta.img with new images in target-file.
We need to add vendor_kernel_boot into AVB signing list and make it
support that image.

Test: make dist -j110 and check vbmeta.img
Bug: 214409109
Signed-off-by: Lucas Wei <lucaswei@google.com>
Change-Id: Id07433f3dc33f95e2edd49de890f1e098cb9ef31
2022-04-21 00:27:27 +08:00
Bowgo Tsai
85578e0a9d Adding BOARD_BUILD_GKI_BOOT_IMAGE_WITHOUT_RAMDISK
GKI targets, e.g., `gki_arm64` or `gki_x86_64` have only
boot-*.img files without a ramdisk. It has no other images,
e.g., init_boot.img, system.img, etc.

The current build system assumes that the ramdisk is either
in a boot.img or in a init_boot.img, which is not true for
those GKI targets. Adding a new flag to support building
boot-*.img without a ramdisk while not building an init_boot.img.

Bug: 220834917
Test: build and `unpack_bootimg`
Change-Id: I789343c3e3d9ff0c36c0e19680a9792bd31a1c9f
2022-04-19 17:57:51 +08:00
Treehugger Robot
477b8e0561 Merge "Revert "Rename init_boot properties to ro.init_boot.*"" 2022-03-23 23:27:02 +00:00
Kelvin Zhang
8250d2cd9e Revert "Rename init_boot properties to ro.init_boot.*"
This reverts commit 411fac84d6.

Reason for revert: b/226318737

Change-Id: Ie1cf0e79b3ac11b91e031a479898a54856020500
2022-03-23 19:46:09 +00:00
Melisa Carranza Zúñiga
fb042449e0 Merge "Adding sepolicy sign params to sign_target_files_apks." 2022-03-22 18:00:43 +00:00
Yi-yo Chiang
e5ed34824f Merge "Rename init_boot properties to ro.init_boot.*" 2022-03-22 09:13:32 +00:00
Melisa Carranza Zuniga
e6d4fb5ae6 Adding sepolicy sign params to sign_target_files_apks.
Bug: b/217570541
Test: sign_target_files_apks --sepolicy_key=build/make/tools/releasetools/testdata/testkey_RSA4096.key
--sepolicy_cert=build/make/tools/releasetools/testdata/testkey.x509.pem target_files.zip target_out.zip
Change-Id: I4ae9f2a3297d10de68c6444dea1cd9996ad9dd11

Change-Id: I4a1ac7009ae0d6bb53c74bd34f8c74f12ea0a3b8
2022-03-21 22:56:33 +00:00
Yi-Yo Chiang
411fac84d6 Rename init_boot properties to ro.init_boot.*
Right now they're named ro.bootimage.* which is really confusing.
Change it so that if generic ramdisk is in boot.img, then name the
properties as ro.bootimage.*, otherwise if generic ramdisk is in
init_boot.img, then name the prop as ro.init_boot.*.

Bug: 211823573
Test: m dist
Change-Id: Icff3c7be04b9a3c6490fbf49ba4f05565b029890
2022-03-15 18:08:10 +08:00
Bowgo Tsai
341fe02903 validate_target_files: support gki_arm64
gki_arm64 is a special target to build multiple GKI boot-*.img
files. It has neither system.img nor vbmeta.img. Skips checking
system.img and vbmeta.img in this case.

Bug: 223078405
Test: validate_target_files signed-gki_arm64-target_files-8277156.zip
Change-Id: I9d3c102c11787cf70c1e1fc8e6a552f8fd19ff72
2022-03-09 16:35:19 +08:00
Yi-Yo Chiang
24da1a43bb releasetools: Update T GKI certification scheme
Companion change of I143680b1cab50a6915df56c8273f8741beaf1180.
Basically does the same thing as the other change.

Bug: 211741246
Test: m dist
Test: ./boot_signature_info.sh boot-5.10.img
Change-Id: I40c4d5866c74a9a2d525f9455969b8a71f22bdbb
2022-02-24 21:50:18 +08:00
Treehugger Robot
52f1acbd93 Merge "Emit warning when build.prop import can't be resolved" 2022-02-18 05:58:27 +00:00
Treehugger Robot
a88313f60b Merge "Improve error message for GetMinSdkVersion" 2022-02-18 05:58:23 +00:00
Kelvin Zhang
42ab828754 Emit warning when build.prop import can't be resolved
Some mountpoints are only available at device runtime, so ignore import
failures in release tools.

Bug: 214322896
Test: th
Change-Id: I1e8c4fb08c6b825fa19c0f0e019dfaf2b03f2927
2022-02-17 13:07:55 -08:00
Kelvin Zhang
21118bba67 Improve error message for GetMinSdkVersion
Bug: 219148978
Test: th
Change-Id: I72c653b6e7e159abf0b7aca394cf465869542483
2022-02-12 09:41:18 -08:00
Devin Moore
b5195ff5fe Get build.prop for init_boot.img from the ramdisk props
boot.img and init_boot.img get their build properties from the platform
ramdisk properties.

Test: "m dist" then drop the OTA zip file into
android.github.io/analyseOTA to verify init_boot is present.
Bug: 211823573

Change-Id: I411e62bf0fc3cb49d4b12dadb52d9820fca041fa
2022-02-11 18:44:26 +00:00
Kelvin Zhang
70876143b5 Fix signing failure when no entry to convert to store
zip -d doesn't work when no entry names specified. So if the list of
entry is empty, just skip calling zip -d.

Bug: 218438888

Test: th
Change-Id: Ie4419b9d6fdc2780255c6f12f9f4d35e5c0a7b26
2022-02-09 16:07:29 -08:00
Ramji Jiyani
13a4137929 core & tools: system_dlkm: add dynamic partition
Converts existing static partition support to a
dynamic partition.

Bug: 200082547
Test: TH
Signed-off-by: Ramji Jiyani <ramjiyani@google.com>
Change-Id: Ifd6d0c2a04e947b16f8b241e99cca594a1d315ae
2022-01-30 08:43:08 +00:00
Ramji Jiyani
bdef28f2ee releasetools: system_dlkm: add image to dist zip
Bug: 200082547
Test: Manual builds:
      1. Set BOARD_USES_SYSTEM_DLKM_PARTITION
      2. Set BOARD_PREBUILT_SYSTEM_DLKM_IMAGE to external image
      - Check that "m dist" put the system_dlkm image in
      out/dist/aosp_cf_x86_64_phone-img-eng.ramjiyani.zip
Signed-off-by: Ramji Jiyani <ramjiyani@google.com>
Change-Id: I8b7dd40e178db7fe542253bff19040095847fa57
2022-01-24 21:27:52 +00:00
Yi-Yo Chiang
36054e2daf releasetools: Android T GKI certification scheme
Companion change of Iaf48a6e3d4b97fa6bfb5e1635a288b045baa248f
To support new GKI certification scheme for boot.img and
init_boot.img on upgrading and launching device combinations.

Bug: 210367929
Bug: 211741246
Bug: 203698939
Test: atest --host releasetools_test:test_common
Test: unpack_bootimg --boot_img boot.img
Test: unpack_bootimg --boot_img init_boot.img
Test: avbtool info_image --image out/boot_signature
Change-Id: I3749297c09c3899046550e4be776acbeea37ef2e
2022-01-14 19:50:32 +08:00
Martin Stjernholm
76537be0a9 Merge "Add an option to specify the path to aapt2." 2022-01-09 19:11:49 +00:00
Martin Stjernholm
58472e8609 Add an option to specify the path to aapt2.
Thin manifests (e.g. master-art) may not have the aapt2 sources and
instead relies on the prebuilt in prebuilts/sdk/tools/linux/bin. In
that case there's no aapt2 in the search path, so it needs to be
specified.

Test: Heavy presubmit build on mainline_modules_bundles on git_master
Test: vendor/google/modules/ArtGoogle/build-art-module.sh
  with http://ag/16584845 on git_master-art, where only the prebuilt
  aapt2 is present
Bug: 212462945
Change-Id: I41a22e8146f5a88534c2721345b4d9d64f76698c
2022-01-07 23:08:53 +00:00
Kelvin Zhang
98ef7bb157 Fix GetCareMap for small images
Change-Id: Iac518e36dbb2fe9303e84d502cd7dfe74621c0af
2022-01-07 14:59:01 -08:00
Bowgo Tsai
88fc2bdd19 Avoid packing ramdisk into GKI boot-*.img
If init_boot.img is present, the GKI boot-*.img should
not include the ramdisk because the ramdisk will be packed
into the init_boot.img instead.

The "has_ramdisk" flag incorrectly checks the condition of:
prebuilt_name != "boot.img" to see if it is a boot.img.
Because "has_ramdisk" was added before we packed multiple
GKI boot-*.img, e.g., boot-5.10.img, boot-5.10-lz4.img, etc.,
into the target files.

Fix this by checking the partition_name is "boot" or not.
Also moving the logic into a new function with comments for each
condition.

Bug: 203698939
Bug: 213028932
Test: sign_target_files_apks \
        --gki_signing_key=external/avb/test/data/testkey_rsa4096.pem \
        --gki_signing_algorithm=SHA256_RSA4096 \
        ./out/dist/*-target_files-eng.*.zip signed.zip, then
        unpack_bootimg to checks the signed boot-*.img has no ramdisk
Change-Id: I5354669feb54d547dbe797e51b1b1baa187cb7cf
2022-01-06 17:50:04 +08:00
Devin Moore
afdd7c74bc Update releasetools for new init_boot.img
This new init_boot.img contains the ramdisk that used to reside in the
boot.img file.

Test: set BOARD_PREBUILT_INIT_BOOT_IMAGE to an external init_boot.img
      - Check that "m" pulls in the init_boot.img to
      out/target/product/vsoc_x86_64/
      - Check that "m dist" adds the init_boot.img to
      aosp_cf_x86_64_phone-img-eng.devinmoore.zip
Test: atest --host releasetools_test
Bug: 203698939
Change-Id: If7ef2cf093e5e525529c7c44333c0f40f6ba0764
2021-12-20 20:54:23 +00:00
Kelvin Zhang
4fc3aa0bc3 Fix signapk not found error when running ota_from_target_files locally
location of ota_from_target_files changed from
out/host/linux-x86/bin to
out/soong/host/linux-x86/bin . This changes relative position of
signapj.jar. To fix, use ANDROID_HOST_OUT as search path

Change-Id: I5397171566e9d7598b5ef16ae26641f0c183d748
2021-11-19 11:34:54 -08:00
Cole Faust
b820bcd829 Prepare release tools for python 3
- Sort dictionaries before looping over them
- Don't call sorted() on lists with Nones
- Open file in binary format when serializing protobufs

Change-Id: If5dbc908f7125f6184014b3c1c7891f833d1d8bf
Bug: 203436762
Test: Presubmits
2021-11-04 17:58:44 -07:00
Kelvin Zhang
2639048c21 Fix a failure in signing test
Pixel moved away from sparse images, so validate_target_files is failing
because it expects sparse images.

Test: th
Change-Id: I322ff10c2afbacfb4d78991be60c11aac92a6d4c
2021-11-03 21:43:22 -07:00