Commit Graph

7449 Commits

Author SHA1 Message Date
Treehugger Robot
82ad6d35f4 Merge "Write signature size even when it's empty" am: 555c3a7dc7 am: 77f7f00e3d am: 5c8ad6415b am: b7015798b9
Original change: https://android-review.googlesource.com/c/platform/build/+/1938523

Change-Id: I824c50ffea8cd8e122d473b068520d5faf3df014
2022-01-08 03:17:21 +00:00
Treehugger Robot
8d77820a48 Merge "Fix GetCareMap for small images" am: 8ef44309b2 am: 2dbcae4a5a am: 04d6278863 am: bcf6567964
Original change: https://android-review.googlesource.com/c/platform/build/+/1940646

Change-Id: Ibf3103142e5bb05c4e0e8d5df3b6916e2341d39e
2022-01-08 03:17:07 +00:00
Treehugger Robot
555c3a7dc7 Merge "Write signature size even when it's empty" 2022-01-08 02:20:11 +00:00
Kelvin Zhang
98ef7bb157 Fix GetCareMap for small images
Change-Id: Iac518e36dbb2fe9303e84d502cd7dfe74621c0af
2022-01-07 14:59:01 -08:00
Treehugger Robot
cfbdb4bd4a Merge "Move fsverity metadata generation to Makefile" am: 7c88c9c1e6 am: 2d6411ed06 am: eea503cf81 am: 9c4a78ceae
Original change: https://android-review.googlesource.com/c/platform/build/+/1937129

Change-Id: I8a3b44b5cd20af1089dccd61c041e8cd97638fda
2022-01-07 04:39:34 +00:00
Treehugger Robot
7c88c9c1e6 Merge "Move fsverity metadata generation to Makefile" 2022-01-07 03:37:16 +00:00
Treehugger Robot
1f087d2f37 Merge "Avoid packing ramdisk into GKI boot-*.img" am: bd5fd25265 am: a551c5d3d1 am: 2ba4030ac0 am: 447d5ec8ea
Original change: https://android-review.googlesource.com/c/platform/build/+/1937120

Change-Id: Ia1a40496d3bae1c969a403f63d54291718a0a25c
2022-01-07 03:35:54 +00:00
Treehugger Robot
bd5fd25265 Merge "Avoid packing ramdisk into GKI boot-*.img" 2022-01-07 02:35:06 +00:00
Inseob Kim
3d5ae80f50 Write signature size even when it's empty
Normally this has no effect, but when we generate metadata for small
files (<=4KB), merkle tree isn't generated. In such case, writing zero
will make the metadata format simpler and unconditional.

Test: manual
Change-Id: Ibe18175b580af3409c896a8bb97323792ad9c459
2022-01-07 10:56:17 +09:00
Inseob Kim
135c1f144f Move fsverity metadata generation to Makefile
build_image.py has been handling fsverity metadata generation in the
packing step, but it can cause issues because the metadata files are
missing in the $OUT directory, and they only exist in result system.img.
This change moves the generation logic into Makefile, and makes the
metadata tracked by ninja graph.

Bug: 206326351
Test: PRODUCT_SYSTEM_FSVERITY_GENERATE_METADATA := true and build
Change-Id: I1f910d8ac6e2cc3c54f35916871733c632f18e44
2022-01-07 09:16:42 +09:00
Bowgo Tsai
88fc2bdd19 Avoid packing ramdisk into GKI boot-*.img
If init_boot.img is present, the GKI boot-*.img should
not include the ramdisk because the ramdisk will be packed
into the init_boot.img instead.

The "has_ramdisk" flag incorrectly checks the condition of:
prebuilt_name != "boot.img" to see if it is a boot.img.
Because "has_ramdisk" was added before we packed multiple
GKI boot-*.img, e.g., boot-5.10.img, boot-5.10-lz4.img, etc.,
into the target files.

Fix this by checking the partition_name is "boot" or not.
Also moving the logic into a new function with comments for each
condition.

Bug: 203698939
Bug: 213028932
Test: sign_target_files_apks \
        --gki_signing_key=external/avb/test/data/testkey_rsa4096.pem \
        --gki_signing_algorithm=SHA256_RSA4096 \
        ./out/dist/*-target_files-eng.*.zip signed.zip, then
        unpack_bootimg to checks the signed boot-*.img has no ramdisk
Change-Id: I5354669feb54d547dbe797e51b1b1baa187cb7cf
2022-01-06 17:50:04 +08:00
Jacky Liu
0e3352a241 Merge "Add PRODUCT_EXTRA_OTA_KEYS to add extra ota keys in otacerts.zip" am: bff997fd1f am: 4f9eb6b9e4 am: 429e1f339b am: 5f1bdb32ae
Original change: https://android-review.googlesource.com/c/platform/build/+/1934214

Change-Id: Ic9e4bdd75e95d6df1595e1371977a532ae830db3
2022-01-06 07:37:19 +00:00
Treehugger Robot
8a3ac7b505 Merge "Fix the signing error on no-system-image targets" am: 790214a4c9 am: ba922579b1 am: 2a00929dec am: 50bf9c8a11
Original change: https://android-review.googlesource.com/c/platform/build/+/1936199

Change-Id: Ie9d4557587a1df9267d1a00ee97d4f46ec8d3ac2
2022-01-06 07:37:07 +00:00
Jacky Liu
bff997fd1f Merge "Add PRODUCT_EXTRA_OTA_KEYS to add extra ota keys in otacerts.zip" 2022-01-06 06:31:37 +00:00
Treehugger Robot
790214a4c9 Merge "Fix the signing error on no-system-image targets" 2022-01-06 06:27:17 +00:00
Treehugger Robot
c81279b860 Merge "Show directories and files with most warnings." am: b2610d7ecd am: 756adcf38b am: c7142be738 am: 152176c056
Original change: https://android-review.googlesource.com/c/platform/build/+/1927679

Change-Id: I63eccb78ac4c81b63371e017316a8cef8847737f
2022-01-06 04:13:51 +00:00
Chih-Hung Hsieh
d4691ecfb6 Show directories and files with most warnings.
* Now with 4 sections that can be shown/hidden independently.
* After the 'selected_projects' section, add two more sections:
  * 'top_directory_section' contains
    * a table of directories with at least 1% warnings.
    * a 'selected_directory_warnings' subsection to show all warnings
      in the clicked directory from the directory table
  * 'top_file_section' contains
    * a table of files with at least 100 or 1% warnings.
    * a 'selected_file_warnings' subsection to show all warnings
      in the clicked file from the file table
* Adjust button/section spaces and reduce button font size to 100%.
* Rename drawTable to genTables.

Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: I765b09a46adc111cfe781719ba7aa0f917aa6ffc
2022-01-05 15:43:52 -08:00
Wei Su
cf2d0e0ef3 Merge "Write files in a temporary directory" am: 22ecce0658 am: 5d03df4516 am: 8623e17527 am: e6551e2973
Original change: https://android-review.googlesource.com/c/platform/build/+/1920697

Change-Id: I6b698f48bf6187b1873d46a707ff9e12f4efb24a
2022-01-05 21:04:45 +00:00
Wei Su
22ecce0658 Merge "Write files in a temporary directory" 2022-01-05 20:03:07 +00:00
Treehugger Robot
a95a2a82a7 Merge "Convert generate-self-extracting-archive.py to python 3" am: 7ca77ea994 am: ecf39825f0 am: 053bce1c76 am: bf0e5be139
Original change: https://android-review.googlesource.com/c/platform/build/+/1935067

Change-Id: I9e294ea8b3d5a59ef53076f7c053762f1796671b
2022-01-05 20:02:24 +00:00
Treehugger Robot
7ca77ea994 Merge "Convert generate-self-extracting-archive.py to python 3" 2022-01-05 18:58:58 +00:00
Jacky Liu
beb0b6927c Add PRODUCT_EXTRA_OTA_KEYS to add extra ota keys in otacerts.zip
Equivalent to PRODUCT_EXTRA_RECOVERY_KEYS but for A/B OTA.

Bug: 211848136
Test: set PRODUCT_EXTRA_OTA_KEYS and check otacerts.zip
Change-Id: I81e27d12a22b405f6227b09c01ed684dfcede19e
2022-01-05 17:54:40 +08:00
Bowgo Tsai
8d4b72405e Fix the signing error on no-system-image targets
Currently when running sign_target_files_apks on a no-system-image
target, it will raise the following error:

  ValueError: max() arg is an empty sequence

This is because there is no APK files in the target_files.zip.
Fixing this by setting maxsize to zero in this case.

Bug: 213028932
Test: lunch gki_arm64-userdebug; make dist
Test: sign_target_files_apks \
        --gki_signing_key=external/avb/test/data/testkey_rsa4096.pem \
        --gki_signing_algorithm=SHA256_RSA4096 \
        --gki_signing_extra_args="--prop gki:prop1 --prop gki:prop2" \
        ./out/dist/*-target_files-eng.*.zip signed.zip
Change-Id: I40daecbc2ff3f89d3e635d1a4a1c1dea31ba9a27
2022-01-04 17:55:19 +08:00
Cole Faust
547ca205ae Convert generate-self-extracting-archive.py to python 3
Test: Add a VENDOR_BLOBS_LICENSE entry to an existing product, then `m with-license`
Bug: 203436762
Change-Id: I4bbe77bda7789b6c44bea141518b1cd2e699d326
2021-12-29 13:58:28 -08:00
Treehugger Robot
d7d4a2d466 Merge "--sign_tool is an optional flag applicable to virtApex." am: ff3a1c73cf am: aabf327615 am: 676de41c33 am: 886bf936c3
Original change: https://android-review.googlesource.com/c/platform/build/+/1933021

Change-Id: I6f9e75a436710222941540be7cfb327698c38f10
2021-12-24 12:15:00 +00:00
Baligh Uddin
06cc018629 --sign_tool is an optional flag applicable to virtApex.
For not virtApex, this results in a KeyError.
line 151, in <module>\n', '  File
"/usr/local/google/home/baligh/clients/goog/master/out/host/linux-x86/bin/sign_apex/sign_apex.py",
line 144, in main\n', "KeyError: 'sign_tool'\n"]

BUG: 193504286
Test: TH
Change-Id: Id982e5c57086ada78168163d2293813df121847d
2021-12-24 06:17:47 +00:00
Devin Moore
5847b680c8 Merge changes from topic "init_boot_img" am: 6363a04e79 am: 1368d81efa am: 2bc48d1025 am: fa9ffed08d
Original change: https://android-review.googlesource.com/c/platform/build/+/1920818

Change-Id: I0f56b8f39335731483ee004bc18a5153518f4423
2021-12-23 00:30:02 +00:00
Devin Moore
6363a04e79 Merge changes from topic "init_boot_img"
* changes:
  Update releasetools for new init_boot.img
  Build a new init_boot.img that contains the boot ramdisk
2021-12-22 23:21:17 +00:00
Treehugger Robot
8b631bf35e Merge "Fix some python2/3 migration issues" am: ceb81e2bef am: fa3f8bdd90 am: 949f4fc6b6 am: 90c9ad6f8b
Original change: https://android-review.googlesource.com/c/platform/build/+/1930407

Change-Id: Ieb08fc938455e0a6f97e01bea9bae99ef7c34e8c
2021-12-22 07:18:25 +00:00
Treehugger Robot
42af7d7584 Merge changes I8d60efc5,Ie291ea87 am: 01ca0d0c5e am: 6a4a3d0ed7 am: 2cbdab25ed am: 17a941e0d3
Original change: https://android-review.googlesource.com/c/platform/build/+/1900893

Change-Id: I7ce3c5657b150e048fa1f497a0349cae3ecdab94
2021-12-22 07:17:59 +00:00
Treehugger Robot
ceb81e2bef Merge "Fix some python2/3 migration issues" 2021-12-22 06:33:32 +00:00
Treehugger Robot
01ca0d0c5e Merge changes I8d60efc5,Ie291ea87
* changes:
  SignApk - change signature of readPassword to use char[] instead
  SignApk - use existing password mechanism when using keystore
2021-12-22 06:23:19 +00:00
Treehugger Robot
7a8d144644 Merge "Update comment up to date" am: cf7300e9c2 am: a93b9ff575 am: ba37412ac7 am: fcf7cd2d05
Original change: https://android-review.googlesource.com/c/platform/build/+/1928383

Change-Id: I78ca8faa5c512216da6977d406933292b24275f2
2021-12-22 01:53:00 +00:00
weisu
eae45315cd Write files in a temporary directory
Avoid writing test files in the build output directory which fails when
run with Bazel. This happens because Bazel's sandboxing environment
ensures that the test's working directory is unwritable.

See https://docs.bazel.build/versions/main/sandboxing.html for more
information.

Bug: 209687942
Test: atest --bazel-mode zipalign_tests
Test: atest zipalign_tests

Change-Id: Ie22f464830c1ffe4d38a94a16dbd39dafa7fe317
2021-12-22 00:06:14 +00:00
Kelvin Zhang
0d0ca5d6a0 Fix some python2/3 migration issues
Exception objects don't necessarily have a .message member, use str()
instead.

Test: th
Change-Id: I57b68dff1ace4937d6ef3b7c564f031e219491b1
2021-12-21 12:31:55 -08:00
Inseob Kim
e7b222aa9e Update comment up to date
nonplat* is renamed to vendor*.

Bug: 210536608
Test: N/A
Change-Id: Ief699bf11395a5b0d9bee22220c09acac6e450f2
2021-12-21 15:57:03 +09:00
Devin Moore
afdd7c74bc Update releasetools for new init_boot.img
This new init_boot.img contains the ramdisk that used to reside in the
boot.img file.

Test: set BOARD_PREBUILT_INIT_BOOT_IMAGE to an external init_boot.img
      - Check that "m" pulls in the init_boot.img to
      out/target/product/vsoc_x86_64/
      - Check that "m dist" adds the init_boot.img to
      aosp_cf_x86_64_phone-img-eng.devinmoore.zip
Test: atest --host releasetools_test
Bug: 203698939
Change-Id: If7ef2cf093e5e525529c7c44333c0f40f6ba0764
2021-12-20 20:54:23 +00:00
Inseob Kim
ea3444a05d Merge changes I6f61a908,Iafd22881 am: 32961d0203 am: 7cc500615e am: eef47a47e4 am: 1c4355d7b2
Original change: https://android-review.googlesource.com/c/platform/build/+/1922320

Change-Id: I9058e470e2c8e1adfae4550204c1a67cc3600e21
2021-12-15 08:32:44 +00:00
Inseob Kim
067492988a fsverity_metadata: Support PEM key
When we have a PEM key, we don't need the process converting a DER key
to PEM format, but we just need to use the PEM key as-is.

Bug: 205987437
Test: build and manual test
Change-Id: I6f61a9088efc0f7193737d3c33b8cfde399b2b6f
2021-12-15 09:41:39 +09:00
Inseob Kim
2d03493341 Add fsverity_metadata_generator helper binary
Making this a host tool will help users generate their own fsverity
metadata easily.

Bug: 205987437
Test: m fsverity_metadata_generator and run it
Change-Id: Iafd228815a74d298d87ca1466c6909c0d24c5874
2021-12-15 09:41:29 +09:00
Ben Fennema
6082d0aeb5 Add --avb_recovery_* options to replace recovery avb key
Test: run sign_target_files_apks with --avb_recovery_key to specify a different key and check with avbtool the key was replaced
Bug: 210126985
Signed-off-by: Ben Fennema <fennema@google.com>
Change-Id: Ic2bb3f6855a49ec065a4c778c429ff076902b95c
2021-12-11 14:05:06 -08:00
Treehugger Robot
ccdfbfc3de Merge "license metadata remove path on top-down walk" am: 159ae38aaa am: ccfd736eda am: d009ec93e6 am: 4fda5ba158
Original change: https://android-review.googlesource.com/c/platform/build/+/1915582

Change-Id: Iff1ee33abc19b38a67f751f07f1e51f83158ac60
2021-12-09 01:10:40 +00:00
Treehugger Robot
44f30de96e Merge "license metadata performance" am: d497acc1c9 am: 1fb8dbd1c8 am: 8b18caeeca am: dee8c12a0a
Original change: https://android-review.googlesource.com/c/platform/build/+/1914147

Change-Id: Iee703eca9967a972cfed1152780ec85aa05f828e
2021-12-09 00:09:07 +00:00
Bob Badour
b285515ca1 license metadata remove path on top-down walk
Performance optimization means not every path will be traversed.

Instead of updating parents via the path, perform a 2nd bottom-up walk
after the top-down walk to propagate the new resolutions to parents.

Note: the 2nd walk method will add resolutions to statically linked
libraries etc. at deeper levels, but those do not affect what gets
reported. In particular, note that test data for dumpresolutions
changes, but none of the test data for listshare, checkshare etc.
changes.

Test: m all systemlicense listshare checkshare dumpgraph dumpresolutions

Bug: 68860345
Bug: 151177513
Bug: 151953481
Change-Id: I76361c4e33bbadbbea38cbec260430e8f9407628
2021-12-08 12:52:59 -08:00
Bob Badour
3a820dd5a0 license metadata performance
Tune the top-down walk to avoid needlessly walking the same subtree
over and over again with the same condition(s).

Takes walking system image down from 3m to 1.5s.

Test: m all systemlicense listshare checkshare dumpgraph dumpresolutions

Bug: 68860345
Bug: 151177513
Bug: 151953481

Change-Id: I4354800cd8dfc42efd4df274d2ce45eaa3e0a99f
2021-12-07 15:36:10 -08:00
Treehugger Robot
acb47f6abf Merge changes I5d48eaba,I4ff3f988 am: 203bbf3254 am: 195ace0fa6 am: 414d918656 am: c4f1a012c9
Original change: https://android-review.googlesource.com/c/platform/build/+/1870079

Change-Id: Iec28602c69315095034578552ab0b026f89178f6
2021-12-07 04:28:44 +00:00
Treehugger Robot
7bd1126ea6 Merge changes I98694001,I5fe57d36 am: ea1a012972 am: a1a1606184 am: 63f1664541 am: 55bec85e70
Original change: https://android-review.googlesource.com/c/platform/build/+/1911224

Change-Id: I7d29101fdf5be5f701414ab02c3e0247e9dc2a66
2021-12-07 04:04:08 +00:00
Bob Badour
afaeb6ae5f compliance package documentation
Bug: 68860345
Bug: 151177513
Bug: 151953481

Test: m all
Test: m systemlicense
Test: m listshare; out/soong/host/linux-x86/bin/listshare ...
Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ...
Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ...
Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ...

where ... is the path to the .meta_lic file for the system image. In my
case if

$ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD)

... can be expressed as:

${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic

Change-Id: I5d48eababce7bba39795d3668eee86b332cbe43d
2021-12-06 16:39:35 -08:00
Bob Badour
6dd0035e04 compliance package: listshare and checkshare
package to read, consume, and analyze license metadata and dependency
graph.

Includes the below command-line tools:

listshare outputs csv of projects to share to meet restricted and
reciprocal license requirements with one project per line. The first
field is the path to the project, and subsequent fields identify the
license resolutions as colon-separated target:annotations tuples.

checkshare outputs error messages to stderr for any targets where
policy dictates both sharing and not sharing the source-code, and PASS
or FAIL to stdout. exit status indicates success 0 or conflict found 1

Bug: 68860345
Bug: 151177513
Bug: 151953481

Test: m all
Test: m systemlicense
Test: m listshare; out/soong/host/linux-x86/bin/listshare ...
Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ...
Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ...
Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ...

where ... is the path to the .meta_lic file for the system image. In my
case if

$ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD)

... can be expressed as:

${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic

Change-Id: I4ff3f98848f7e6d03a35734300d763ef5f245d53
2021-12-06 16:39:35 -08:00
Bob Badour
1ded0a1b18 compliance package: dumpresolutions
package to read, consume, and analyze license metadata and dependency
graph.

Includes the below command-line tool:

dumpresolutions outputs the resulting set of resolutions after the
bottom-up and top-down resolves, or after joining 1 or more condition
walks.

Bug: 68860345
Bug: 151177513
Bug: 151953481

Test: m all
Test: m systemlicense
Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ...
Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ...

where ... is the path to the .meta_lic file for the system image. In my
case if

$ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD)

... can be expressed as:

${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic

Change-Id: I9869400126cd7ad4b7376b0bab31b46aad732f5d
2021-12-06 16:38:08 -08:00