Commit Graph

7484 Commits

Author SHA1 Message Date
Kelvin Zhang
988f92d7c5 Merge "Fix check_tf failure when duplicate package names are present" am: 4a376986ea am: 6a7d589ed2 am: 68b0980e5b am: 367d28b2c6
Original change: https://android-review.googlesource.com/c/platform/build/+/1949256

Change-Id: I232a775698ab70b0bf0cddeee9af55cb8af2652e
2022-01-18 19:31:51 +00:00
Kelvin Zhang
4a376986ea Merge "Fix check_tf failure when duplicate package names are present" 2022-01-18 18:14:02 +00:00
Treehugger Robot
e2b5582b7f Merge "Migrate java-event-logs-tags.py to python3." am: 6c38edc0b9 am: eb1391adb9 am: e330539009 am: d4df224693
Original change: https://android-review.googlesource.com/c/platform/build/+/1948776

Change-Id: I039c3c56de57f6ec040499e0d7089bcd8b4638d8
2022-01-18 11:52:25 +00:00
Jingwen Chen
2a31f3df87 Migrate java-event-logs-tags.py to python3.
These are used in the Bazel Android app builds.

Test: m droid
Test: CI
Test: b run //build/make/tools:java-event-log-tags

Change-Id: Iaffe6f974008d1a0532a849353d25df02197afd2
2022-01-17 13:02:53 +00:00
Yi-yo Chiang
14d0e73497 Merge changes from topic "gki-avb-vts" am: 8b4e2fd6c0 am: b45caeb134 am: f2c62d7ddb am: e3874400a1
Original change: https://android-review.googlesource.com/c/platform/build/+/1941496

Change-Id: I252c986a2e0a15df168a14ac4d31f19f9f0dc19b
2022-01-17 05:51:06 +00:00
Yi-yo Chiang
8b4e2fd6c0 Merge changes from topic "gki-avb-vts"
* changes:
  releasetools: Android T GKI certification scheme
  Android T GKI certification scheme
2022-01-17 04:52:28 +00:00
Yi-Yo Chiang
36054e2daf releasetools: Android T GKI certification scheme
Companion change of Iaf48a6e3d4b97fa6bfb5e1635a288b045baa248f
To support new GKI certification scheme for boot.img and
init_boot.img on upgrading and launching device combinations.

Bug: 210367929
Bug: 211741246
Bug: 203698939
Test: atest --host releasetools_test:test_common
Test: unpack_bootimg --boot_img boot.img
Test: unpack_bootimg --boot_img init_boot.img
Test: avbtool info_image --image out/boot_signature
Change-Id: I3749297c09c3899046550e4be776acbeea37ef2e
2022-01-14 19:50:32 +08:00
Treehugger Robot
bd1b5694fd Merge "Add option to enable lz4diff" am: 2f1f985480 am: 3187ce1680 am: b89cd6aad3 am: 8a1662d71c
Original change: https://android-review.googlesource.com/c/platform/build/+/1944184

Change-Id: I2adfe5357fec0cedf58393b6922d3aa116f472ec
2022-01-13 21:19:31 +00:00
Treehugger Robot
2f1f985480 Merge "Add option to enable lz4diff" 2022-01-13 20:33:31 +00:00
Kelvin Zhang
67553739e3 Fix check_tf failure when duplicate package names are present
Test: check_target_files_signatures sdk_gphone64_x86_64-target_files-8069754.zip
Bug: 214388458

Change-Id: I0a8a0d0596c04a9da4283d4d4caeb3bb62762ccc
2022-01-13 12:30:22 -08:00
Kelvin Zhang
f2728d615e Add option to enable lz4diff
When lz4diff is enabled, we inject the host copy of liblz4.so in
target_file to delta_generator. This is to ensure that host tooling can
produce same compressed output as the device.

Test: th
Bug: 206729162

Change-Id: I2d8206f7ec54fadedea16bf3d811b8353bc9414d
2022-01-12 17:21:00 -08:00
Treehugger Robot
6095899cc6 Merge "Regression testcase for shipped vs nonshipped." am: 3a0656307c am: 192875fdfc am: 6300667501 am: b928cd6011
Original change: https://android-review.googlesource.com/c/platform/build/+/1946016

Change-Id: I54d0761adb5c3a34f028624ed2d0c3bae1d18910
2022-01-13 00:45:27 +00:00
Treehugger Robot
3a0656307c Merge "Regression testcase for shipped vs nonshipped." 2022-01-12 23:35:17 +00:00
Treehugger Robot
e74a41a8b9 Merge "Fix typo in doc: code segment for container dot" am: 710813e871 am: 1dd148f737 am: f79ab4b3a3 am: 00195a11bf
Original change: https://android-review.googlesource.com/c/platform/build/+/1946012

Change-Id: I7a4dc7026406c3e9adb8a17d79c369556a04349e
2022-01-12 23:27:18 +00:00
Treehugger Robot
710813e871 Merge "Fix typo in doc: code segment for container dot" 2022-01-12 22:31:40 +00:00
Treehugger Robot
adf5feeda2 Merge changes Ia2ec1b81,Ibc831ae8 am: 49380ebecf am: 5955f8a521 am: 4817ded07b am: 2e04c83b6b
Original change: https://android-review.googlesource.com/c/platform/build/+/1941454

Change-Id: I54efe335ac9d3ceea817c176872249be701900f4
2022-01-12 01:41:09 +00:00
Treehugger Robot
022e8e293f Merge "Fold annotations." am: b3f3e7b3f7 am: e27ecb85d4 am: b878b8ff56 am: 04b2d70747
Original change: https://android-review.googlesource.com/c/platform/build/+/1944765

Change-Id: Ie95025ef51be307a97b699d8659e91036fd0525d
2022-01-12 01:40:48 +00:00
Bob Badour
00bf3d9b29 Regression testcase for shipped vs nonshipped.
Bug: 68860345
Bug: 151177513
Bug: 151953481

Test: m all
Test: m systemlicense
Test: m listshare; out/soong/host/linux-x86/bin/listshare ...
Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ...
Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ...
Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ...

where ... is the path to the .meta_lic file for the system image. In my
case if

$ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD)

... can be expressed as:

${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic

Change-Id: I17221ec8fe429b5f745c0733f8bb1ab0ea2f4567
2022-01-11 16:38:55 -08:00
Treehugger Robot
49380ebecf Merge changes Ia2ec1b81,Ibc831ae8
* changes:
  Performance and scale.
  Use struct{}
2022-01-12 00:27:41 +00:00
Treehugger Robot
b3f3e7b3f7 Merge "Fold annotations." 2022-01-12 00:27:35 +00:00
Bob Badour
99dba55b4b Fix typo in doc: code segment for container dot
Test: not applicable
Change-Id: Ibaadc6e8fd4a267b50a46feb2dd6278ff1f5ace1
2022-01-11 14:46:22 -08:00
Bob Badour
103eb0f9bc Performance and scale.
Defer edge creation.

Don't create edges until the count is known to avoid repeated allocate+
copy operatios.

Limit resolutions.

Allow only a single resolution condition set per target, and overwrite
intermediate results. Reduces memory and obviates allocations.

Propagate fewer conditions.

Instead of propagating notice conditions to parents in graph during
initial resolve, leave them on leaf node, and attach to ancestors in
the final walk. Reduces copies.

Parallelize resolutions.

Use goroutines, mutexes, and waitgroups to resolve branches of the
graph in parallel. Makes better use of available cores.

Don't accumulate resolutions inside non-containers.

During the final resolution walk, only attach actions to ancestors from
the root down until the 1st non-aggregate. Prevents an explosion of
copies in the lower levels of the graph.

Drop origin for scale.

Tracking the origin of every potential origin for every restricted
condition does not scale. By dropping origin, propagating from top
to bottom can prune many redundant paths avoiding an exponential
explosion.

Conditions as bitmask.

Use bit masks for license conditions and condition sets. Reduces maps
and allocations.

Bug: 68860345
Bug: 151177513
Bug: 151953481

Test: m all
Test: m systemlicense
Test: m listshare; out/soong/host/linux-x86/bin/listshare ...
Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ...
Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ...
Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ...

where ... is the path to the .meta_lic file for the system image. In my
case if

$ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD)

... can be expressed as:

${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic

Change-Id: Ia2ec1b818de6122c239fbd0824754f1d65daffd3
2022-01-11 10:40:50 -08:00
Bob Badour
5446a6f8e1 Use struct{}
Using struct{}{} as the payload for set maps reduces memory use for
large sets.

Bug: 68860345
Bug: 151177513
Bug: 151953481

Test: m all
Test: m systemlicense
Test: m listshare; out/soong/host/linux-x86/bin/listshare ...
Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ...
Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ...
Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ...

where ... is the path to the .meta_lic file for the system image. In my
case if

$ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD)

... can be expressed as:

${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic

Change-Id: Ibc831ae80fc50f35e1000348fb28fc0167d0ebed
2022-01-11 10:40:50 -08:00
Bob Badour
67d8ae390a Fold annotations.
Use annotation slices looked up from map instead of read from disk to
reduce copies.

Bug: 68860345
Bug: 151177513
Bug: 151953481

Test: m all
Test: m systemlicense
Test: m listshare; out/soong/host/linux-x86/bin/listshare ...
Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ...
Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ...
Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ...

where ... is the path to the .meta_lic file for the system image. In my
case if

$ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD)

... can be expressed as:

${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic

Change-Id: Ibf6f7d319092e0d54ab451ffbbd35dcd7d0080e0
2022-01-11 10:40:39 -08:00
Treehugger Robot
28e245f711 Merge "Do not append .meta_lic to meta_lic" am: 9d83209f12 am: 0f12de8954 am: 9212d765d1 am: 4250af6e75
Original change: https://android-review.googlesource.com/c/platform/build/+/1944764

Change-Id: I9f7a7d1824bd2ac198b34a26a61537e4c6242000
2022-01-11 07:17:58 +00:00
Treehugger Robot
9d83209f12 Merge "Do not append .meta_lic to meta_lic" 2022-01-11 06:11:14 +00:00
Bob Badour
63a281cc29 Do not append .meta_lic to meta_lic
Test: manual
Change-Id: I8d7162dc55f057d542faaa7feb546050d3a872e7
2022-01-10 18:00:03 -08:00
Victor Hsieh
9e12870560 Merge "Add min_sdk_version to BuildManifest.apk" am: 70ca7c6add am: 30afe79e30 am: 829fe6d623 am: 271ca249c3
Original change: https://android-review.googlesource.com/c/platform/build/+/1941946

Change-Id: I2c5281988a1bc9c14340d207c71275565eff866a
2022-01-10 21:17:34 +00:00
Jiyong Park
efbb6ffa00 Add min_sdk_version to BuildManifest.apk
sign_target_files_apks demands it.

Bug: 213823227
Test: run com.google.android.tradefed.ota.SigningToolTest#Testcase1_Signingtest
Change-Id: I1bba2b25a71449a57d284024e77fb76d1b79a0ee
2022-01-10 20:14:05 +09:00
Treehugger Robot
e7edcdc66d Merge "Add option to enable zucchini" am: dcd7e283d2 am: 3468085607 am: cd9f83eeb8 am: 2f5176647a
Original change: https://android-review.googlesource.com/c/platform/build/+/1939053

Change-Id: I0e155c9433a51c78a4fd63de68581ab0135f08db
2022-01-09 21:46:07 +00:00
Treehugger Robot
dcd7e283d2 Merge "Add option to enable zucchini" 2022-01-09 20:38:18 +00:00
Martin Stjernholm
6c0015b95e Merge "Add an option to specify the path to aapt2." am: 76537be0a9 am: 96cea0f44b am: a6e2ff4804 am: 948e8c78ab
Original change: https://android-review.googlesource.com/c/platform/build/+/1938648

Change-Id: Ie2c28d23ce90ac36cc2258192b6ea51cb7366511
2022-01-09 20:30:51 +00:00
Martin Stjernholm
76537be0a9 Merge "Add an option to specify the path to aapt2." 2022-01-09 19:11:49 +00:00
Treehugger Robot
82ad6d35f4 Merge "Write signature size even when it's empty" am: 555c3a7dc7 am: 77f7f00e3d am: 5c8ad6415b am: b7015798b9
Original change: https://android-review.googlesource.com/c/platform/build/+/1938523

Change-Id: I824c50ffea8cd8e122d473b068520d5faf3df014
2022-01-08 03:17:21 +00:00
Treehugger Robot
8d77820a48 Merge "Fix GetCareMap for small images" am: 8ef44309b2 am: 2dbcae4a5a am: 04d6278863 am: bcf6567964
Original change: https://android-review.googlesource.com/c/platform/build/+/1940646

Change-Id: Ibf3103142e5bb05c4e0e8d5df3b6916e2341d39e
2022-01-08 03:17:07 +00:00
Treehugger Robot
555c3a7dc7 Merge "Write signature size even when it's empty" 2022-01-08 02:20:11 +00:00
Martin Stjernholm
58472e8609 Add an option to specify the path to aapt2.
Thin manifests (e.g. master-art) may not have the aapt2 sources and
instead relies on the prebuilt in prebuilts/sdk/tools/linux/bin. In
that case there's no aapt2 in the search path, so it needs to be
specified.

Test: Heavy presubmit build on mainline_modules_bundles on git_master
Test: vendor/google/modules/ArtGoogle/build-art-module.sh
  with http://ag/16584845 on git_master-art, where only the prebuilt
  aapt2 is present
Bug: 212462945
Change-Id: I41a22e8146f5a88534c2721345b4d9d64f76698c
2022-01-07 23:08:53 +00:00
Kelvin Zhang
410bb389d8 Add option to enable zucchini
Change-Id: Id952826c9c5af912fba679af61f2a2e1645641dd
2022-01-07 14:59:01 -08:00
Kelvin Zhang
98ef7bb157 Fix GetCareMap for small images
Change-Id: Iac518e36dbb2fe9303e84d502cd7dfe74621c0af
2022-01-07 14:59:01 -08:00
Treehugger Robot
cfbdb4bd4a Merge "Move fsverity metadata generation to Makefile" am: 7c88c9c1e6 am: 2d6411ed06 am: eea503cf81 am: 9c4a78ceae
Original change: https://android-review.googlesource.com/c/platform/build/+/1937129

Change-Id: I8a3b44b5cd20af1089dccd61c041e8cd97638fda
2022-01-07 04:39:34 +00:00
Treehugger Robot
7c88c9c1e6 Merge "Move fsverity metadata generation to Makefile" 2022-01-07 03:37:16 +00:00
Treehugger Robot
1f087d2f37 Merge "Avoid packing ramdisk into GKI boot-*.img" am: bd5fd25265 am: a551c5d3d1 am: 2ba4030ac0 am: 447d5ec8ea
Original change: https://android-review.googlesource.com/c/platform/build/+/1937120

Change-Id: Ia1a40496d3bae1c969a403f63d54291718a0a25c
2022-01-07 03:35:54 +00:00
Treehugger Robot
bd5fd25265 Merge "Avoid packing ramdisk into GKI boot-*.img" 2022-01-07 02:35:06 +00:00
Inseob Kim
3d5ae80f50 Write signature size even when it's empty
Normally this has no effect, but when we generate metadata for small
files (<=4KB), merkle tree isn't generated. In such case, writing zero
will make the metadata format simpler and unconditional.

Test: manual
Change-Id: Ibe18175b580af3409c896a8bb97323792ad9c459
2022-01-07 10:56:17 +09:00
Inseob Kim
135c1f144f Move fsverity metadata generation to Makefile
build_image.py has been handling fsverity metadata generation in the
packing step, but it can cause issues because the metadata files are
missing in the $OUT directory, and they only exist in result system.img.
This change moves the generation logic into Makefile, and makes the
metadata tracked by ninja graph.

Bug: 206326351
Test: PRODUCT_SYSTEM_FSVERITY_GENERATE_METADATA := true and build
Change-Id: I1f910d8ac6e2cc3c54f35916871733c632f18e44
2022-01-07 09:16:42 +09:00
Bowgo Tsai
88fc2bdd19 Avoid packing ramdisk into GKI boot-*.img
If init_boot.img is present, the GKI boot-*.img should
not include the ramdisk because the ramdisk will be packed
into the init_boot.img instead.

The "has_ramdisk" flag incorrectly checks the condition of:
prebuilt_name != "boot.img" to see if it is a boot.img.
Because "has_ramdisk" was added before we packed multiple
GKI boot-*.img, e.g., boot-5.10.img, boot-5.10-lz4.img, etc.,
into the target files.

Fix this by checking the partition_name is "boot" or not.
Also moving the logic into a new function with comments for each
condition.

Bug: 203698939
Bug: 213028932
Test: sign_target_files_apks \
        --gki_signing_key=external/avb/test/data/testkey_rsa4096.pem \
        --gki_signing_algorithm=SHA256_RSA4096 \
        ./out/dist/*-target_files-eng.*.zip signed.zip, then
        unpack_bootimg to checks the signed boot-*.img has no ramdisk
Change-Id: I5354669feb54d547dbe797e51b1b1baa187cb7cf
2022-01-06 17:50:04 +08:00
Jacky Liu
0e3352a241 Merge "Add PRODUCT_EXTRA_OTA_KEYS to add extra ota keys in otacerts.zip" am: bff997fd1f am: 4f9eb6b9e4 am: 429e1f339b am: 5f1bdb32ae
Original change: https://android-review.googlesource.com/c/platform/build/+/1934214

Change-Id: Ic9e4bdd75e95d6df1595e1371977a532ae830db3
2022-01-06 07:37:19 +00:00
Treehugger Robot
8a3ac7b505 Merge "Fix the signing error on no-system-image targets" am: 790214a4c9 am: ba922579b1 am: 2a00929dec am: 50bf9c8a11
Original change: https://android-review.googlesource.com/c/platform/build/+/1936199

Change-Id: Ie9d4557587a1df9267d1a00ee97d4f46ec8d3ac2
2022-01-06 07:37:07 +00:00
Jacky Liu
bff997fd1f Merge "Add PRODUCT_EXTRA_OTA_KEYS to add extra ota keys in otacerts.zip" 2022-01-06 06:31:37 +00:00
Treehugger Robot
790214a4c9 Merge "Fix the signing error on no-system-image targets" 2022-01-06 06:27:17 +00:00