Commit Graph

7218 Commits

Author SHA1 Message Date
Yifan Hong
b284e367bf Merge changes from topic "vendor_sku" am: d6077aa7ca am: b8ecdfdd05
Change-Id: I7cf8a4e94ca9ea2da8300ca5193eb37307da38e2
2020-03-19 01:30:41 +00:00
Tianjie Xu
eaed60c1a1 Add aftl inclusion proof
The otatools should talk to the aftl server and append the inclusion
proofs when building the vbmeta image. We should only do this during the
signing process when the network is always available.

Also the inclusion proof doesn't impact the final vbmeta image size on
coral, where the final size is 8192 after padding. This is below the
physical image of 65536.

Bug: 147870995
Test: unit tests pass, run sign_target_file_apks
Change-Id: If84c6bf5198c9b05f5e0c16ae6335971915f47e3
2020-03-18 13:11:41 -07:00
Yifan Hong
69430e65fe Also check empty SKU if SKUS are not defined
In the case when both ODM_MANIFEST_SKUS and ODM_MANIFEST_FILE are
both empty, also checks VINTF compatibility for empty SKU case.

Same for device manifest. Although right now, DEVICE_MANIFEST_FILE and
DEVICE_MANIFEST_SKUS cannot be both empty (because libvintf requires
vendor manifest exists barring manifest fragments), let libvintf throws
an error for this case. In the build system, simply handle vendor manifest
the same way as ODM manifest.

Bug: 148601741
Test: manual
Change-Id: Ia0655fd4d37f6286b164a3221ec4b2716f29147d
2020-03-17 20:27:32 -07:00
Automerger Merge Worker
0de248e7eb Merge changes from topic "vendor_sku" am: 352e6a4d18 am: 01f66b11a1
Change-Id: I761041be4b353135edccbb537d28c6e668da1adc
2020-03-16 22:40:50 +00:00
Yifan Hong
922c38fb6e Proper checks for vendor and ODM SKUs
If DEVICE_MANIFEST_FILE / ODM_MANIFEST_FILES is set, also include check
for empty vendor / odm SKU, respectively. Then, do cross product of
vendor SKUs and ODM skus, and check each possible combination.

Test: build with and without vendor skus
Test: m check-vintf-all
Bug: 148601741

Change-Id: I23cc81998e7afc36c43540bf6d615b4d4b288447
Merged-In: I23cc81998e7afc36c43540bf6d615b4d4b288447
2020-03-16 15:06:11 -07:00
Yifan Hong
28ffd73591 Proper checks for vendor and ODM SKUs
If DEVICE_MANIFEST_FILE / ODM_MANIFEST_FILES is set, also include check
for empty vendor / odm SKU, respectively. Then, do cross product of
vendor SKUs and ODM skus, and check each possible combination.

Test: build with and without vendor skus
Test: m check-vintf-all
Bug: 148601741

Change-Id: I23cc81998e7afc36c43540bf6d615b4d4b288447
2020-03-13 18:32:07 -07:00
Automerger Merge Worker
0cc8390c39 Merge "ota_tool: fix incremental ota error" am: 059d22231c am: d8c3b91a90
Change-Id: I8631cceab19964296da4721c49fa797604fcbc6a
2020-03-13 20:31:31 +00:00
Xindong Xu
3e75aec327 ota_tool: fix incremental ota error
Problem:
Logical partitions do not update right since commit
f67dd8059d

    Generate block based OTA for product partitions

due to the wrong order of parameters. This is logged in
/tmp/recovery.log:

    skipping 0 blocks already patched to 202

Test: build
Bug: 149794632
Change-Id: I92f7a8a226a8a96b892254362d362c515bfb6dc6
(cherry picked from commit 2a7aaa69ae)
2020-03-13 13:03:10 -07:00
Xindong Xu
2a7aaa69ae ota_tool: fix incremental ota error
Problem:
Logical partitions do not update right since commit
f67dd8059d

    Generate block based OTA for product partitions

due to the wrong order of parameters. This is logged in
/tmp/recovery.log:

    skipping 0 blocks already patched to 202

Bug: 149794632
Change-Id: I92f7a8a226a8a96b892254362d362c515bfb6dc6
2020-03-13 14:25:35 +01:00
Daniel Norman
ab5acefb6d RESTRICT AUTOMERGE Uses a per-partition fingerprint for building images and avb_salt.
This causes the output image files of a merged build to be identical
to the image files of the input partial builds, for each images in
PARTITIONS_WITH_CARE_MAP.

Test: python -m unittest test_common
Test: `m dist`; `unzip out/dist/target_files.zip IMAGES/\*`;
      `zip -d out/dist/target_files.zip IMAGES/\*`
      `add_img_to_target_files -a out/dist/target_files.zip`.
      Verify that the rebuilt images are identical to the deleted ones.
Test: Build a merged target (using merge_target_files.py). Verify that
      the partial target-files.zip IMAGES are identical to the merged
      target-files.zip IMAGES for PARTITIONS_WITH_CARE_MAP images.
Bug: 150405807
Change-Id: I5fdf5783c1aff9c14cf5408090389b1f65b69ca6
(cherry picked from commit d5fe862628)
2020-03-10 11:04:31 -07:00
Tom Cherry
7cff1fa1fc Include private/fs_config.h directly when needed
This is no longer a transitive include of android_filesystem_config.h

Bug: 149785767
Test: build
Change-Id: Ib546dbc3b6f546ba48009cc9d6d0f4564acf0872
2020-03-05 10:55:45 -08:00
Bowgo Tsai
a7276405c9 Support removing AVB public keys
AVB public keys might be installed into the first-stage ramdisk.
This CL supports removing some AVB keys during the signing process,
which can be useful when a key needs to be revoked.

Also correcting the path of force_debuggable detection.

Bug: 150109393
Test: make dist
Test: ./build/tools/releasetools/sign_target_files_apks \
      --remove_avb_public_keys key1.avbpubkey,key2.avbpubkey \
      out/dist/*-target_files-*.zip signed-target_files.zip
Change-Id: I1af95adbbec655f40121f3392965dd1950aca4e5
Merged-In: I1af95adbbec655f40121f3392965dd1950aca4e5
(cherry picked from commit 2fe786a528)
2020-03-03 17:39:20 +08:00
Automerger Merge Worker
8c6fa8f710 Merge "Uses a per-partition fingerprint for building images and avb_salt." am: b6145d320c am: 14b4bbc4a2
Change-Id: I8dd484d03402db3541ea432b723f1bfac53f256d
2020-03-02 23:38:30 +00:00
Daniel Norman
d5fe862628 Uses a per-partition fingerprint for building images and avb_salt.
This causes the output image files of a merged build to be identical
to the image files of the input partial builds, for each images in
PARTITIONS_WITH_CARE_MAP.

Test: python -m unittest test_common
Test: `m dist`; `unzip out/dist/target_files.zip IMAGES/\*`;
      `zip -d out/dist/target_files.zip IMAGES/\*`
      `add_img_to_target_files -a out/dist/target_files.zip`.
      Verify that the rebuilt images are identical to the deleted ones.
Test: Build a merged target (using merge_target_files.py). Verify that
      the partial target-files.zip IMAGES are identical to the merged
      target-files.zip IMAGES for PARTITIONS_WITH_CARE_MAP images.
Bug: 150405807
Change-Id: I5fdf5783c1aff9c14cf5408090389b1f65b69ca6
2020-03-02 19:05:20 +00:00
Automerger Merge Worker
36ac6bf35e Merge "Separate html/csv output functions into html_writer.py" am: cabdab0a40 am: b68bc994e9
Change-Id: I57e44b5c23dbf66c9a088e2081d68faee76e272c
2020-02-28 02:01:37 +00:00
Chih-Hung Hsieh
3cce2bcc60 Separate html/csv output functions into html_writer.py
* also suppress some pylint warnings

Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: I497dbb7496ca21470ba33db03eedb27f5e8f1e96
2020-02-27 15:39:18 -08:00
Automerger Merge Worker
aaea42dab0 Merge "Preserve the monotonic property when subtracting 0 block" am: 223d055768 am: 7ca340fb91
Change-Id: I52d12bdf9797119eb9ea9f904f567a679be57430
2020-02-27 21:46:50 +00:00
Tianjie Xu
e86d1bccc0 Preserve the monotonic property when subtracting 0 block
The monotonic property of the range is lost after the recent change to
subtract 0th block from the file range. As a result, we read the block
in the wrong order when pull files from the raw image. This yields a
incorrect hash during validation.

Bug: 150334561
Test: run validate_target_file. unittests
Change-Id: Id4fedaaf273c42c8d21a4549fda4c9a45e22066a
(cherry picked from commit f02ececc7f)
2020-02-27 13:24:11 -08:00
Tianjie Xu
223d055768 Merge "Preserve the monotonic property when subtracting 0 block" 2020-02-27 21:22:23 +00:00
Automerger Merge Worker
79f72d667b Merge "Recognize more warning patterns." am: ef7f5467e9 am: 74272124df
Change-Id: Id7d2c1d12d49615256bdd63fa8762eae2bd7435d
2020-02-27 08:37:19 +00:00
Tianjie Xu
f02ececc7f Preserve the monotonic property when subtracting 0 block
The monotonic property of the range is lost after the recent change to
subtract 0th block from the file range. As a result, we read the block
in the wrong order when pull files from the raw image. This yields a
incorrect hash during validation.

Bug: 150334561
Test: run validate_target_file. unittests
Change-Id: Id4fedaaf273c42c8d21a4549fda4c9a45e22066a
2020-02-26 21:50:47 -08:00
Chih-Hung Hsieh
445ad810c8 Recognize more warning patterns.
* add more Java, Kotlin, and Rust warning patterns
* recognize more clang-tidy cert-* and bugprone-* warnings;
  still keep the catch-all 'cert' and 'bugprone' groups.

Test: warn.py build.log > warnings.html
Change-Id: I909ec09822f42f60493ebe943c055bf0d4b69938
2020-02-26 14:34:21 -08:00
Automerger Merge Worker
392dde3398 Merge "Use same warn_common.py and warn.py for Android and Chrome" am: 3728312005 am: 3bcf176437
Change-Id: I1183d6c22936aab1e002c93f7bcfc2543a8912f6
2020-02-26 18:54:27 +00:00
Chih-Hung Hsieh
5ae5519661 Use same warn_common.py and warn.py for Android and Chrome
* add chrome_project_list.py
* use newer Chrome's warn_common.py and severity.py
* modify warn_common.py to work with both python2 and python3,
  to work with and without google3

Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: I9aeee7031c11988d8ead1733ff4846ae6cb1ee73
2020-02-25 19:47:35 -08:00
Automerger Merge Worker
a89cdfcb1c Merge "Support removing AVB public keys" am: b5aad1b937 am: 33c9bd38da
Change-Id: I8d711a9c51aa8e35d27c88ecf0dc85c869059037
2020-02-26 00:29:20 +00:00
Bowgo Tsai
2fe786a528 Support removing AVB public keys
AVB public keys might be installed into the first-stage ramdisk.
This CL supports removing some AVB keys during the signing process,
which can be useful when a key needs to be revoked.

Also correcting the path of force_debuggable detection.

Bug: 150109393
Test: make dist
Test: ./build/tools/releasetools/sign_target_files_apks \
      --remove_avb_public_keys key1.avbpubkey,key2.avbpubkey \
      out/dist/*-target_files-*.zip signed-target_files.zip
Change-Id: I1af95adbbec655f40121f3392965dd1950aca4e5
2020-02-24 17:53:10 +08:00
Automerger Merge Worker
87ca28cbfa Merge "Exclude holes from the block map." am: 9581181338 am: 4969b45d50 am: de72054e03
Change-Id: I46db3c1d99ab77d04171b7067e6303d91e626400
2020-02-21 18:05:55 +00:00
David Anderson
ab9c7e6d04 Exclude holes from the block map.
e2fsdroid can export holes as a "0" block, in order to make sure logical
and physical block assignments are maintained properly. Exclude these
hole blocks from range sets in LoadFileBlockMap.

Bug: 139201772
Test: m dist on non-A/B
Change-Id: Ic04e869572229678fdeaaa6fbf6f464d6c672051
2020-02-21 17:04:00 +00:00
Automerger Merge Worker
16b3ff3727 Merge "Make oemaids_headers available to vendor." am: 8e28db7acc am: c9d21efbbf am: b46724c634
Change-Id: I3a07016189e1ba5537a0a44b78e1fcb8fb53a8fa
2020-02-18 21:23:57 +00:00
P.Adarsh Reddy
8b2e7c9c51 Make oemaids_headers available to vendor.
This change makes the oemaids_headers header library
available to vendor side.

Bug: 149665231
Test: Tested compilation, oemaids_headers is available to
a sample vendor binary.

Change-Id: I87369f1db883e40e83d6076defdd4c0d84aac46f
2020-02-18 09:44:23 +00:00
Automerger Merge Worker
4e3bceed4d Merge "build_image: check value of "ext4_share_dup_blocks"" am: 9021d5c4b4 am: d6437ec539 am: 74f2a0ec45
Change-Id: I18aaea958e048d07fc9daf91ca721169243a8c88
2020-02-13 16:46:22 +00:00
Dianne Hackborn
25648e60b2 Reland: Rework platform version to hide codenames.
The public platform version no longer can be a codename, it is
always the most recently released platform.  A new build property
and API provides either the offical version or the current codename
as appropriate.  This will avoid breaking apps that look at the
platform version while development is under a codename.

Changes from the original version:
- Logic for emitting system properties has also been added to
  buildinfo_common.sh to make sure that ro.${partition}.build.*
  are consistent with ro.build.*.
- We also use PLATFORM_VERSION_LAST_STABLE as the value set in the
  boot image os_version header / hash tree additional headers to
  ensure they match with the ro.build.version.release system property.
  This is hopefully a temporary workaround while we stop using the
  system property for this purpose.

Bug:  143175463
Test: manual
Change-Id: Iaeefb35137de01c6ea57115d8830dd9bc30eccba
2020-02-11 13:20:20 +00:00
Tamas Petz
c0a8c63935 build_image: check value of "ext4_share_dup_blocks"
Regardless of the value of "ext4_share_dup_blocks" "-c" flag is passed
unconditionally. The correct behavior is to check if
"ext4_share_dup_blocks" is set to "true".

Test: 1. build bonito with "BOARD_EXT4_SHARE_DUP_BLOCKS := false"
      2. check for "shared_blocks" feature of system.img
         ./out/host/linux-x86/bin/tune2fs -l \
         ./out/target/product/bonito/system.img | \
         grep -o shared_blocks

Change-Id: I8c0e5606658dd0f47b71cfc6c453a77a57da4f42
2020-02-10 11:45:06 +01:00
Automerger Merge Worker
6534a8c6f7 Merge "Remove the key parameter when verifying avb images in validate_target_files" am: 1fc958b4fe am: c543f548a5 am: c2e0b4f5a3
Change-Id: Ie9a0de8283aad85e707d7daabde0918aa7b54be4
2020-02-07 01:39:26 +00:00
Tianjie Xu
9bd832a012 Remove the key parameter when verifying avb images in validate_target_files
If a key is specified, the avbtool always use the input key to verify
all the chained images. And this will cause failures when the vbmeta &
system use different keys (e.g. RSA 4096 vs RSA2048). Because the public
key to vbmeta will always fail to verify the system image. Remove the
'--key' parameter in the verification command, so the avbtool will use
the embedded public in the image.

Test: validate target-file from sdk_gphone_x86_64
Bug: 148916990
Change-Id: I9d31be0f8c32af605af94fa73d07818f40f51ec4
2020-02-06 13:31:26 -08:00
Narayan Kamath
825c78bf0f Merge "Add basic support ro.build.version.release_or_codename" 2020-02-06 10:51:15 +00:00
Dianne Hackborn
0023fa857c Add basic support ro.build.version.release_or_codename
Currently no different from "release". Will be updated to differ
in a follow up change, once all related issues have been resolved.

Bug:  143175463
Test: manual

Change-Id: I2e7de023d49cb1dcb83637e2c9af5a3e48e46f7d
2020-02-05 12:22:29 +00:00
Chih-Hung Hsieh
ed74896600 More Java/Kotlin warning patterns
* Simplify java warning pattern functions.

Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: I837fa9290acded3b936182877d9e0c00de4cdf4f
2020-02-04 15:34:12 -08:00
Chih-hung Hsieh
9f7a42025d Merge "Add new Java/Kotlin/Make warning patterns" 2020-02-04 18:41:57 +00:00
Chih-Hung Hsieh
a7f5f3fcbb Add new Java/Kotlin/Make warning patterns
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: I3195999d60f268f105ead022c193319de4af4e64
2020-01-31 16:14:04 -08:00
Tianjie Xu
0e38d64e49 Add quote to the signing arguments when passing to apexer
We should add quote when passing the signing arguments to apexer. So
the final argument whould look like
apexer ... --signing_args "--signing_helper_with_files=%path"

And the argument parser in avbtool will eventually parse the
"--signing_helper_with_files" correctly.

Bug: 148627666
Test: unittest pass, run a smoke sign_target_files_apk with signing arguments
Change-Id: Ie203d26a508cb98ca94e6c364a2640b681ad79f2
2020-01-31 20:19:18 +00:00
Tianjie Xu
cea6ad1fb0 Pass signing_args to apexer to repack apex
In the build signing environment, we don't necessary have access to the
physical private key. And we work around it by passing an extra
signing_helper to the avbtool instead of a private key. Since we need
to perform signing to add the hashtree footer when rebuilding the apex
image. We should propogate the signing_helper to apexer and eventually
to avbtool.

Bug: 148627666
Test: unit tests pass. Smoke test by pass a public key to apexer and
hardcode the private key in signing helper script.
Change-Id: I56d57485493a5ba2cbe4c49e493ee9604f68088b
2020-01-31 01:37:36 +00:00
Tianjie Xu
ba5de57120 Merge "Assets should be optional in apex repacking" 2020-01-29 22:59:30 +00:00
Tianjie Xu
83bd55c1db Assets should be optional in apex repacking
We should only provide the --assets_dir option when the assets directory
is available in the original apex.

Bug: 148452817
Test: unit tests pass
Change-Id: I44308b80a43ff3f441223f0ecc248991d1d83f31
2020-01-29 11:40:42 -08:00
Chih-Hung Hsieh
b09530bd65 Add new C++ and Java warning patterns
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: I11457fdab7abd475e835367582c0b9aab11cfe6f
2020-01-29 11:01:36 -08:00
Tianjie Xu
8fff584906 Merge "Make the error more instructive on missing android_jar_path" 2020-01-29 03:19:23 +00:00
Daniel Rosenberg
dac25a96df Merge "Add build properties to enable casefold and prj_quota" 2020-01-28 23:53:25 +00:00
Tianjie Xu
61a792f323 Make the error more instructive on missing android_jar_path
So when $ANDROID_BUILD_TOP is missing, the error message print out the
path of the missing file; instead of a type error.

Bug: 148452817
Test: check result with missing ANDROID_BUILD_TOP
Change-Id: I5f841f6626eb3843364f20e78b278d3244505396
2020-01-28 10:58:14 -08:00
Daniel Rosenberg
6cc2c81f4d Add build properties to enable casefold and prj_quota
This adds the ability to format f2fs and ext4 in a way that supports
casefolding and project quotas. Project quotas require wider inodes, and
thus cannot be enable apart from at build time. Casefolding can be
configured with f2fs.fsck/tune2fs later.

Both features are enabled via inheriting emulated_storage.mk in the
device.mk

Bug: 138322712
Bug: 138321217
Test: Enable on device with related patches, view fs configuration on /data
Change-Id: Ib1ab0589006ff1b3fb7478b7e9b0a8aad71e4b09
2020-01-27 12:45:47 -08:00
Tianjie Xu
88a759d651 Resign apks contained in apex
Some apex payload images contain apk files. And these apks need to be
signed during the signing processed when sign_target_files_apks is
called. To support the signing, we can extract the payload and repack
the apex file with the (de)apexer tool. Add the signing support in the
apex_util.

Bug: 146508800
Test: unit tests pass, run sign_apex, sign_target_files_apks
Change-Id: If6d58975248709a144b07dbabf47c27916e5695e
2020-01-27 19:48:39 +00:00