Commit Graph

6780 Commits

Author SHA1 Message Date
Hongguang Chen
d7c160ffbf Skip _oem_props if oem_dicts is None.
The oem_dicts in BuildInfo is only available to ota_from_target_files
when it's called with "--oem_settings" input. However, aosp/1135332
starts to use BuildInfo in add_img_to_target_files w/o oem_dicts. An
assert is triggered due to it when oem_fingerprint_properties is in the
info_dict.
This change skip _oem_props reading if oem_dicts is None to allow to
this case.

BUG: 155360923
BUG: 154171021
Test: Sign images with oem_fingerprint_properties.
Change-Id: I6a73feecd9567fd4d85f4eab3d4e11c5df28fe39
2020-05-04 16:54:10 +00:00
P.Adarsh Reddy
d8e24eee70 releasetools: Fallback to "mkbootimg_args" if "recovery_mkbootimg_args" not found.
For older target_files.zip, since "recovery_mkbootimg_args" may not be
set fallback to the earlier config -"mkbootimg_args" for recovery image.
This will make releasetools backward compatible, and usable for
older/already generated target-files package.

Change-Id: If40359130b8e6da48cb8c345d8cb079f457fe36f
2020-05-04 14:30:09 +00:00
Yifan Hong
975e2bef86 Fix check target files vintf test.
check_vintf now checks deprecation as well, which
requires matrix fragments to exist. Update tests to
reflect this.

The test also uses legacy vendor manifest path, which makes
it not combined with the ODM manifests, causing sku_compat
test to fail. Fix it by using the new vendor manifest path
(which libvintf correctly combines it with the ODM manifests.)

Fixes: 155131894
Test: python -m unittest test_check_target_files_vintf
Change-Id: Ib660f8796efc3465d9513688695dac29c63a7514
2020-05-01 16:34:22 -07:00
Baligh Uddin
d9fcafd612 Allow passing in codename_to_api_level_map to sign_apex.
BUG: 154517483
Change-Id: I93a90ae066b0e9eeb1b2ff26d9cb9b07a11fe68c
Merged-In: I93a90ae066b0e9eeb1b2ff26d9cb9b07a11fe68c
2020-05-01 13:56:04 +00:00
Steven Laver
8e2086e6e3 releasetools: correct allowed property sources for incremental OTAs
When loading build info from a previous version of Android, the set of
allowed property sources should match those available in that version.
In this particular case, the product_services partition was a valid
property source in Android 10.

Bug: 155053195
Test: ran unit tests from test_common.py
Test: generated an incremental OTA which previously failed
Change-Id: Ic0b0a112656533eca78dee31517deff7e3c8d7cc
2020-04-29 22:30:42 +00:00
Treehugger Robot
7cdc91f5ea Merge "Add elsk to the owner of scripts" 2020-04-29 00:32:31 +00:00
Tianjie
ded2ae16a5 Add elsk to the owner of scripts
Test: build
Change-Id: Ia722605596c354d4e295a6d18a2696fdf90c582c
2020-04-28 14:56:01 -07:00
Elliott Hughes
b5e650a119 Merge "releasetools: Use du -b" 2020-04-28 16:55:49 +00:00
Tianjie Xu
f53f313c36 Merge "Update the argument when signing aftl" 2020-04-21 22:39:31 +00:00
Tianjie
20dd8f20b1 Update the argument when signing aftl
The arguments transparency_log_servers and transparency_log_pub_keys
have been merged. Update the caller in the OTA script accordingly.

Also disable the test to contact aftl server until we have
a public server.

Bug: 153940575
Test: check the argument
Change-Id: If6a7e7d644884d395c75c2fcdfd6aa7c2380d851
2020-04-21 22:38:55 +00:00
Steve Muckle
9793cf6c8a support multiple boot.img files in release tools
Support for generating multiple boot.img files is required in the
release tools to enable GKI distribution/signing.

Bug: 151094943
Change-Id: I536a286d3123f35918106a52c49b1148d746370f
2020-04-16 17:54:25 -07:00
Steve Muckle
f84668e1da Enable non-v3 recovery image with v3 boot image
Add support for building a non-v3 recovery image along with a
v3 boot.img and vendor_boot image.

Bug: 150825361
Change-Id: I80d4e71ff09b83dd1c7f9accb4de99a30f95f4bf
2020-04-14 13:03:58 -07:00
Jaewoong Jung
e57fb1a3aa Merge "Add signing certificate lineage file support." 2020-04-07 22:33:25 +00:00
Jaewoong Jung
95445e6913 Add signing certificate lineage file support.
Also add multi-cert support to prebuilt apps so that they can benefit
from the new lineage feature.

(This is a cherry-pick change.)

Test: m GoogleServicesFramework w/ modified build rules
Test: m PrebuiltGmsCore w/ modified build rules
Test: apksigner lineage -v --print-certs -in <built_module_path>
Fixes: 152897457
Change-Id: If7d5d4bd308629c8340231520214c76c8a568a65
Merged-In: If7d5d4bd308629c8340231520214c76c8a568a65
2020-04-06 13:42:23 -07:00
Bill Peckham
f6baf822ac Merge "Ensure that merge_target_files.py prints INFO level logs." 2020-04-06 19:00:56 +00:00
Tianjie Xu
0889aaeeae Merge "Change the parameter type of offset in read" 2020-04-04 20:00:20 +00:00
Bill Peckham
cb84817fdf Ensure that merge_target_files.py prints INFO level logs.
Bug: 153166596
Test: Run a merge build, inspect logs.
Change-Id: Ic2795f26922db32030c7ab7eb4ec6f153c1e9654
2020-04-04 04:20:45 +00:00
Bill Peckham
5c7b034a5c Make the partition= tag optional.
Since we might use ToT release tools to sign a package
generated by an older build, we make the new
`partition=` tag optional. This also means we need to be
careful to use non-greedy regex matching.

Bug: 153133823
Test: python3 -m unittest
Test: input with and without the new `partition=` tag
Test: new test_ReadApkCerts_WithWithoutOptionalFields
Change-Id: Ic57efd34e745ad302ae17150c6f2318f0b4524cb
2020-04-03 17:09:37 -07:00
Tom Cherry
8c7a4f6fc1 Merge "Include private/fs_config.h directly when needed" 2020-04-03 14:36:58 +00:00
Tianjie Xu
2995096483 Merge "Add aftltool is a signing parameter" 2020-04-02 18:09:59 +00:00
Tianjie
2751d2b8f3 Change the parameter type of offset in read
As we support zip64 in libziparchive, we change the parameter of
the ReadAtOffset() in zip reader from uin32_t to off64_t. So the
derived class needs to be updated as well.

Bug: 150900468
Test: build
Change-Id: Icbfd2dd54b01ff62da988ba8598c1057f9bb6368
2020-04-02 06:16:43 +00:00
Kiyoung Kim
39197f846b Merge "Build required modules from default" 2020-04-02 06:14:43 +00:00
Tianjie
0f3074566c Add aftltool is a signing parameter
Right now we assert that the aftltool should exist since the one
in aosp won't work due to grpc dependencies.

Bug: 147870995
Test: build
Change-Id: Iabb2c375167572a965493a7648fdc1abe287af67
2020-04-01 12:20:21 -07:00
Kiyoung Kim
0a8d26210e Build required modules from default
Some required modules from host tool (such as imgdiff) are bound to
required from releasetools_common, but unfortunately required modules
from python_library_host does not work (b/140144201). To make this work
we should mark dependency from each host tool.

Bug: 152258157
Test: Build succeeded without system image
Change-Id: Iedfda9b48d78687d8f622b818eeb93896d4f119d
2020-04-01 14:21:01 +00:00
Tianjie Xu
a3541df291 Merge "Add aftl inclusion proof" 2020-03-31 22:12:46 +00:00
Bill Peckham
19c3feb2d3 Build merged apexkeys.txt/apkcerts.txt by partition.
Propagate partition tag data to apexkeys.txt and
apkcerts.txt so that merge_target_files.py can build
merged versions of these files by filtering the
framework files for framework partitions and filtering
the vendor files for vendor partitions.

Bug: 138942268
Change-Id: Ic3226728e97dae84d38ec230ccc86d1b124bea94
2020-03-30 22:33:27 +00:00
Baligh Uddin
639b3b7a65 Continue to pass in --signing_helper for avbtool.
BUG: 152424931
Test: Tested Locally
Change-Id: I4e93a899b9055c937980a496f20b35c01ccbb4c1
2020-03-29 22:05:27 -07:00
Treehugger Robot
8b3c9a1b40 Merge "releasetools: Add missing log format specifier" 2020-03-27 22:26:42 +00:00
Luca Stefani
88e1a14523 releasetools: Add missing log format specifier
Test: m otapackage
Change-Id: I273cf315400dd2b624d2853ece0c30c6fb3d529d
2020-03-27 13:07:56 +00:00
Luca Stefani
3e4b44868b releasetools: Add signapk as required modules
* common.py requires the signapk jar

Test: m otapackage ( before it was broken )
Change-Id: Ic0cff0a21258cb1e7d1ced5553a940dcf5cd6d04
2020-03-27 13:05:21 +00:00
Chirayu Desai
96a913e7f4 releasetools: Use du -b
* When $OUT is on a compressed disk,
  `du -k -s` might not return the correct sizes.
* Adding `-b` gets us close to the actual size.

toybox commit: https://github.com/landley/toybox/pull/177

Test: m, builds and boots.
Devices: crosshatch blueline bonito sargo (anything with dynamic partitions)
2020-03-27 04:17:45 +05:30
Baligh Uddin
b1ad7c6dc7 Merge "Build the payload image only during apk-in-apex signing" 2020-03-24 04:54:29 +00:00
Tianjie
c180a5d222 Build the payload image only during apk-in-apex signing
When doing apk-in-apex signing, the package name is not reserved
during the apex repacking. As a result, the name accidentally
reverts to 'com.android.wifi' from 'com.google.android.wifi'.

This cl changes the behavior to call 'apexer' by passing the
'--payload_only' argument. So we don't build the apex file from
scratch and the old AndroidManifest.xml will be reused.

Test: 152084536
Bug: unit tests pass
Change-Id: I8332b2ee84832fb196f2e1c4309abac5ab92e153
2020-03-23 18:15:34 -07:00
Tianjie Xu
eaed60c1a1 Add aftl inclusion proof
The otatools should talk to the aftl server and append the inclusion
proofs when building the vbmeta image. We should only do this during the
signing process when the network is always available.

Also the inclusion proof doesn't impact the final vbmeta image size on
coral, where the final size is 8192 after padding. This is below the
physical image of 65536.

Bug: 147870995
Test: unit tests pass, run sign_target_file_apks
Change-Id: If84c6bf5198c9b05f5e0c16ae6335971915f47e3
2020-03-18 13:11:41 -07:00
Yifan Hong
69430e65fe Also check empty SKU if SKUS are not defined
In the case when both ODM_MANIFEST_SKUS and ODM_MANIFEST_FILE are
both empty, also checks VINTF compatibility for empty SKU case.

Same for device manifest. Although right now, DEVICE_MANIFEST_FILE and
DEVICE_MANIFEST_SKUS cannot be both empty (because libvintf requires
vendor manifest exists barring manifest fragments), let libvintf throws
an error for this case. In the build system, simply handle vendor manifest
the same way as ODM manifest.

Bug: 148601741
Test: manual
Change-Id: Ia0655fd4d37f6286b164a3221ec4b2716f29147d
2020-03-17 20:27:32 -07:00
Yifan Hong
28ffd73591 Proper checks for vendor and ODM SKUs
If DEVICE_MANIFEST_FILE / ODM_MANIFEST_FILES is set, also include check
for empty vendor / odm SKU, respectively. Then, do cross product of
vendor SKUs and ODM skus, and check each possible combination.

Test: build with and without vendor skus
Test: m check-vintf-all
Bug: 148601741

Change-Id: I23cc81998e7afc36c43540bf6d615b4d4b288447
2020-03-13 18:32:07 -07:00
Xindong Xu
2a7aaa69ae ota_tool: fix incremental ota error
Problem:
Logical partitions do not update right since commit
f67dd8059d

    Generate block based OTA for product partitions

due to the wrong order of parameters. This is logged in
/tmp/recovery.log:

    skipping 0 blocks already patched to 202

Bug: 149794632
Change-Id: I92f7a8a226a8a96b892254362d362c515bfb6dc6
2020-03-13 14:25:35 +01:00
Tom Cherry
7cff1fa1fc Include private/fs_config.h directly when needed
This is no longer a transitive include of android_filesystem_config.h

Bug: 149785767
Test: build
Change-Id: Ib546dbc3b6f546ba48009cc9d6d0f4564acf0872
2020-03-05 10:55:45 -08:00
Daniel Norman
d5fe862628 Uses a per-partition fingerprint for building images and avb_salt.
This causes the output image files of a merged build to be identical
to the image files of the input partial builds, for each images in
PARTITIONS_WITH_CARE_MAP.

Test: python -m unittest test_common
Test: `m dist`; `unzip out/dist/target_files.zip IMAGES/\*`;
      `zip -d out/dist/target_files.zip IMAGES/\*`
      `add_img_to_target_files -a out/dist/target_files.zip`.
      Verify that the rebuilt images are identical to the deleted ones.
Test: Build a merged target (using merge_target_files.py). Verify that
      the partial target-files.zip IMAGES are identical to the merged
      target-files.zip IMAGES for PARTITIONS_WITH_CARE_MAP images.
Bug: 150405807
Change-Id: I5fdf5783c1aff9c14cf5408090389b1f65b69ca6
2020-03-02 19:05:20 +00:00
Chih-Hung Hsieh
3cce2bcc60 Separate html/csv output functions into html_writer.py
* also suppress some pylint warnings

Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: I497dbb7496ca21470ba33db03eedb27f5e8f1e96
2020-02-27 15:39:18 -08:00
Tianjie Xu
223d055768 Merge "Preserve the monotonic property when subtracting 0 block" 2020-02-27 21:22:23 +00:00
Tianjie Xu
f02ececc7f Preserve the monotonic property when subtracting 0 block
The monotonic property of the range is lost after the recent change to
subtract 0th block from the file range. As a result, we read the block
in the wrong order when pull files from the raw image. This yields a
incorrect hash during validation.

Bug: 150334561
Test: run validate_target_file. unittests
Change-Id: Id4fedaaf273c42c8d21a4549fda4c9a45e22066a
2020-02-26 21:50:47 -08:00
Chih-Hung Hsieh
445ad810c8 Recognize more warning patterns.
* add more Java, Kotlin, and Rust warning patterns
* recognize more clang-tidy cert-* and bugprone-* warnings;
  still keep the catch-all 'cert' and 'bugprone' groups.

Test: warn.py build.log > warnings.html
Change-Id: I909ec09822f42f60493ebe943c055bf0d4b69938
2020-02-26 14:34:21 -08:00
Chih-Hung Hsieh
5ae5519661 Use same warn_common.py and warn.py for Android and Chrome
* add chrome_project_list.py
* use newer Chrome's warn_common.py and severity.py
* modify warn_common.py to work with both python2 and python3,
  to work with and without google3

Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: I9aeee7031c11988d8ead1733ff4846ae6cb1ee73
2020-02-25 19:47:35 -08:00
Bowgo Tsai
2fe786a528 Support removing AVB public keys
AVB public keys might be installed into the first-stage ramdisk.
This CL supports removing some AVB keys during the signing process,
which can be useful when a key needs to be revoked.

Also correcting the path of force_debuggable detection.

Bug: 150109393
Test: make dist
Test: ./build/tools/releasetools/sign_target_files_apks \
      --remove_avb_public_keys key1.avbpubkey,key2.avbpubkey \
      out/dist/*-target_files-*.zip signed-target_files.zip
Change-Id: I1af95adbbec655f40121f3392965dd1950aca4e5
2020-02-24 17:53:10 +08:00
David Anderson
ab9c7e6d04 Exclude holes from the block map.
e2fsdroid can export holes as a "0" block, in order to make sure logical
and physical block assignments are maintained properly. Exclude these
hole blocks from range sets in LoadFileBlockMap.

Bug: 139201772
Test: m dist on non-A/B
Change-Id: Ic04e869572229678fdeaaa6fbf6f464d6c672051
2020-02-21 17:04:00 +00:00
P.Adarsh Reddy
8b2e7c9c51 Make oemaids_headers available to vendor.
This change makes the oemaids_headers header library
available to vendor side.

Bug: 149665231
Test: Tested compilation, oemaids_headers is available to
a sample vendor binary.

Change-Id: I87369f1db883e40e83d6076defdd4c0d84aac46f
2020-02-18 09:44:23 +00:00
Tamas Petz
c0a8c63935 build_image: check value of "ext4_share_dup_blocks"
Regardless of the value of "ext4_share_dup_blocks" "-c" flag is passed
unconditionally. The correct behavior is to check if
"ext4_share_dup_blocks" is set to "true".

Test: 1. build bonito with "BOARD_EXT4_SHARE_DUP_BLOCKS := false"
      2. check for "shared_blocks" feature of system.img
         ./out/host/linux-x86/bin/tune2fs -l \
         ./out/target/product/bonito/system.img | \
         grep -o shared_blocks

Change-Id: I8c0e5606658dd0f47b71cfc6c453a77a57da4f42
2020-02-10 11:45:06 +01:00
Tianjie Xu
9bd832a012 Remove the key parameter when verifying avb images in validate_target_files
If a key is specified, the avbtool always use the input key to verify
all the chained images. And this will cause failures when the vbmeta &
system use different keys (e.g. RSA 4096 vs RSA2048). Because the public
key to vbmeta will always fail to verify the system image. Remove the
'--key' parameter in the verification command, so the avbtool will use
the embedded public in the image.

Test: validate target-file from sdk_gphone_x86_64
Bug: 148916990
Change-Id: I9d31be0f8c32af605af94fa73d07818f40f51ec4
2020-02-06 13:31:26 -08:00
Chih-Hung Hsieh
ed74896600 More Java/Kotlin warning patterns
* Simplify java warning pattern functions.

Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: I837fa9290acded3b936182877d9e0c00de4cdf4f
2020-02-04 15:34:12 -08:00