Commit Graph

7484 Commits

Author SHA1 Message Date
Treehugger Robot
1fb1088e88 Merge "Fix signapk not found error when running ota_from_target_files locally" 2021-11-19 23:25:05 +00:00
Kelvin Zhang
4fc3aa0bc3 Fix signapk not found error when running ota_from_target_files locally
location of ota_from_target_files changed from
out/host/linux-x86/bin to
out/soong/host/linux-x86/bin . This changes relative position of
signapj.jar. To fix, use ANDROID_HOST_OUT as search path

Change-Id: I5397171566e9d7598b5ef16ae26641f0c183d748
2021-11-19 11:34:54 -08:00
Treehugger Robot
10f69082b6 Merge changes from topic "fsverity_digest" am: 866d824328 am: f90fb54430 am: 173e8a3f90 am: 53a0a19fef
Original change: https://android-review.googlesource.com/c/platform/build/+/1855993

Change-Id: I82d7ce6062a8b253644d8c93bc598da9b745bbef
2021-11-12 07:37:38 +00:00
Inseob Kim
f69346e0a8 Add fsverity digest manifest
fsverity digest manifest stores a map from files to fsverity digests.
The manifest is installed as a serialized protobuf file, to a signed apk
system/etc/security/fsverity/BuildManifest.apk.

Bug: 193113311
Test: build with PRODUCT_SYSTEM_FSVERITY_GENERATE_METADATA := true
Change-Id: I55fc10400206b8ce0d5f198faea08fe3930b362c
2021-11-11 11:19:44 +00:00
Inseob Kim
9cda397948 Implement fsverity metadata generator
Using fsverity tool, fsverity metadata for specific artifacts in system
mage can be generated. Users can do that by setting a makefile variable
PRODUCT_SYSTEM_FSVERITY_GENERATE_METADATA to true.

If set to true, the following artifacts will be signed.

- system/framework/*.jar
- system/framework/oat/<arch>/*.{oat,vdex,art}
- system/etc/boot-image.prof
- system/etc/dirty-image-objects

One fsverity metadata container file per one input file will be
generated in system.img, with a suffix ".fsv_meta". e.g. a container
file for "system/framework/foo.jar" will be
"system/framework/foo.jar.fsv_meta".

Bug: 193113311
Test: build with PRODUCT_SYSTEM_FSVERITY_GENERATE_METADATA := true
Change-Id: Ib70d591a72d23286b5debcb05fbad799dfd79b94
2021-11-11 17:30:19 +09:00
Tianjie Xu
befecc0ee2 Merge "Fix python3 incompatiable code" am: 7a048996b7 am: 4d6bc649e8 am: bff0316e3c am: 1fbeb8b694 am: 29ef79795f
Original change: https://android-review.googlesource.com/c/platform/build/+/1886461

Change-Id: I51bdc9e85b4230fc0bfe0f882032ba666e12cf19
2021-11-10 18:11:32 +00:00
Tianjie
b049191e78 Fix python3 incompatiable code
http://aosp/1883069 switch the releasetool to use python3.
But target_files_diff still have py2 code that cause failures.
Fix that to unblock OTA generation.

Bug: 205790608
Test: generate an incremental OTA
Change-Id: Ib4d86dc1842afeae8b35681c7d809da140fac600
2021-11-09 23:37:11 -08:00
Treehugger Robot
212fac995e Merge "Skip signing key check in some cases" am: 984ad44276 am: 165b1be894 am: 92327efd67 am: d9d006e03b am: b00d947ed6
Original change: https://android-review.googlesource.com/c/platform/build/+/1885988

Change-Id: I51888d4a3a9b77726016371331e758494c0c69c3
2021-11-10 02:18:41 +00:00
Treehugger Robot
984ad44276 Merge "Skip signing key check in some cases" 2021-11-10 00:48:27 +00:00
Tianjie Xu
c67a611103 Merge "Temporarily remove the care_map in the ota package" am: 3fcc52519e am: c8e5f1de93 am: 4cd4b0c572 am: 16b4beb644 am: edac9f8897
Original change: https://android-review.googlesource.com/c/platform/build/+/1883497

Change-Id: Ief5ef5829293048b82ebfdc3425fe91e9c7d74d9
2021-11-09 21:39:46 +00:00
Cole Faust
391cbc69b2 Merge "Use python 3 for release tools" am: ece722ca92 am: 9220d1a841 am: 454757ca83 am: 6df2faf751 am: f05bbe47e1
Original change: https://android-review.googlesource.com/c/platform/build/+/1883069

Change-Id: I1db1436cfef34ce4d0c4bff7e91556eb3cf564a1
2021-11-09 21:39:16 +00:00
Tianjie Xu
3fcc52519e Merge "Temporarily remove the care_map in the ota package" 2021-11-09 19:38:14 +00:00
Kelvin Zhang
4b588cf75a Skip signing key check in some cases
Test: th
Bug: 205657204

Change-Id: I3a2fc886cfb3fe3d53619bfb8fe84aa799d80de3
2021-11-09 08:43:01 -08:00
Tianjie
5856cfbf82 Temporarily remove the care_map in the ota package
The care_map has incorrect size for non-sparse images. Temporarily
removes it until the root cause is fixed

Bug: 205541521
Test: build
Change-Id: I76bcd2c0c778566b4a6c69b6d45428952225d406
2021-11-08 22:55:53 -08:00
Cole Faust
c76931fe8b Use python 3 for release tools
The release tools have already been updated in prior cls to
support python 3. To test this, I added code to print the script
name to a temp file at the beginning of every script, and then
ran various builds, ensuring that the scripts were run afterwards.

The builds run:
m brillo_update_payload checkvintf minigzip lz4 toybox unpack_bootimg deapexer (needed for releasetools_test)
m (as both sdk_phone_x86_64 and beagle_x15)
m target-files-package (as both sdk_phone_x86_64 and beagle_x15)
m releasetools_test && out/host/linux-x86/nativetest64/releasetools_test/releasetools_test (25 errors with and without python3)
m check_target_files_signatures apksigner target-files-package && out/host/linux-x86/bin/check_target_files_signatures out/target/product/emulator_x86_64/obj/PACKAGING/target_files_intermediates/sdk_phone_x86_64-target_files-eng.colefaust.zip
m dist
As aosp_cf_x86_64_phone: m dist && sign_target_files_apks  out/dist/aosp_cf_x86_64_phone-target_files-eng.$USER.zip /tmp/signed_target_files.zip && validate_target_files /tmp/signed_target_files.zip

This hit all the scripts except for make_recovery_patch,
merge_builds, and ota_package_parser.

I couldn't find anything that uses merge_builds, so it must
only be run manually.

make_recovery_patch and ota_package_parser are only run if
TARGET_OTA_ALLOW_NON_AB is true (among other things), which
is not the case for any product in aosp.

Test: Described in commit message + presubmits
Change-Id: I1a29eafa7ff1a69973b27055e311de77f7ee628b
2021-11-05 13:47:28 -07:00
Cole Faust
6f396911fc Merge "Prepare release tools for python 3" am: fdb8f17a83 am: ad943ea6df am: fecb7bb3a4 am: 48aac85b06 am: 67fdf2968b
Original change: https://android-review.googlesource.com/c/platform/build/+/1874167

Change-Id: I7a5aefb927ae70a86cb9d3f4b69593c37db536f6
2021-11-05 18:36:07 +00:00
Cole Faust
b820bcd829 Prepare release tools for python 3
- Sort dictionaries before looping over them
- Don't call sorted() on lists with Nones
- Open file in binary format when serializing protobufs

Change-Id: If5dbc908f7125f6184014b3c1c7891f833d1d8bf
Bug: 203436762
Test: Presubmits
2021-11-04 17:58:44 -07:00
Treehugger Robot
6ebf09d040 Merge "Fix a failure in signing test" am: 936e704999 am: 5c8fc606d4 am: aeba7359a0 am: 86250621a3 am: ce19509a15
Original change: https://android-review.googlesource.com/c/platform/build/+/1879016

Change-Id: Ib3106fa7f14873ad1b0d5d11b06a2e570bc7a3f4
2021-11-04 10:37:42 +00:00
Treehugger Robot
c62a01004a Merge "Add a compressor types option" am: 1e392720c2 am: a0c34f2c21 am: 44c33933c0 am: 99a9435e6c am: cd3ca41f20
Original change: https://android-review.googlesource.com/c/platform/build/+/1873795

Change-Id: I490cf27520ac8e85e20ca8e790c35ad512683610
2021-11-04 05:43:24 +00:00
Kelvin Zhang
2639048c21 Fix a failure in signing test
Pixel moved away from sparse images, so validate_target_files is failing
because it expects sparse images.

Test: th
Change-Id: I322ff10c2afbacfb4d78991be60c11aac92a6d4c
2021-11-03 21:43:22 -07:00
Treehugger Robot
1e392720c2 Merge "Add a compressor types option" 2021-11-04 04:13:13 +00:00
Treehugger Robot
ce15a031ce Merge changes from topic "tproto" am: cd128891ed am: 0f0e4d23e5 am: 40bd79e8b7 am: aa3348dd1b am: d1a921ec8b
Original change: https://android-review.googlesource.com/c/platform/build/+/1865920

Change-Id: I16f08374a3c9aa1d561fb9653cd2cbad515a1388
2021-11-03 20:15:35 +00:00
Bob Badour
380a1afb1a Add macros for non-module license metadata and deps am: 879cfa867d am: 1efd13a2a1 am: c1d98c6310 am: e6d1b1d160 am: d57a36fadc
Original change: https://android-review.googlesource.com/c/platform/build/+/1741493

Change-Id: I0c8fc45c6b8c7a76ed6411437513b6ea8997127d
2021-11-03 20:15:33 +00:00
Kelvin Zhang
9b6d4ec01b Add a compressor types option
Test: th
Change-Id: I1ff1bccf179bfd87ce9e08def0e5e38921d7d158
2021-11-03 11:17:04 -07:00
Colin Cross
1ee26cae73 Convert .meta_lic files to textproto
Make it easier to write tools against .meta_lic files and store complex
data by writing them in textproto.

Test: builds
Change-Id: Ibbb6cfbb1bdddd3d938a86d563673a049d826d66
2021-11-03 15:44:00 +00:00
Bob Badour
879cfa867d Add macros for non-module license metadata and deps
Gets rid of .meta_module files and instead defers emitting rules until
after all the non-module targets have been processed. Allows direct
dependency on .meta_lic files, which in turn depend on license text
files.

Bug: 68860345
Bug: 151177513
Bug: 151953481

Test: m all
Test: m systemlicense
Test: m reportmissinglicenses

Change-Id: I2c467feac6e13a9366ff66f924889f1dbd48c3f1
2021-11-03 00:52:25 +00:00
Treehugger Robot
4865a3c4a7 Merge "Support custom APEX signing tool" am: 76a4c62f56 am: c24e277d03 am: 4cb46d6fc8 am: 690f6364e4 am: d6dfa2a9b4
Original change: https://android-review.googlesource.com/c/platform/build/+/1870245

Change-Id: Id1ad821e06c0bdeefd81a406bb2f7f29f0fcb41a
2021-10-30 01:06:39 +00:00
Jooyung Han
bac375bf57 Merge changes from topic "sign_apex-sign_tool" am: f531336c6f am: 01bb43478f am: f7c262f502 am: 0e7616f29b am: c7ccad5845
Original change: https://android-review.googlesource.com/c/platform/build/+/1870244

Change-Id: I6e5338dd181ae8c4a073b9d64a9de64d766d9c5a
2021-10-28 06:05:31 +00:00
Jooyung Han
ca275ba622 fix releasetools_test am: c4b7b34b4b am: 4197f3a4c6 am: c404ba2e2e am: 5f4eaf17e8 am: 24eba94ba0
Original change: https://android-review.googlesource.com/c/platform/build/+/1870249

Change-Id: Iac5e595781da028fa609cdff8f97b71e70c1bc9a
2021-10-28 06:05:30 +00:00
Rupert Shuttleworth
072daa5f98 Merge "Add ability to set --rotation-min-sdk-version for signapk." 2021-10-27 12:56:18 +00:00
Rupert Shuttleworth
a95ab436f6 Add ability to set --rotation-min-sdk-version for signapk.
Bug: 197787352

Test: m signapk

Change-Id: I3b5e9bb3985c2ae83381499a44b339a7b1deb05c
2021-10-27 12:53:16 +00:00
Jooyung Han
8caba5e14e Support custom APEX signing tool
When an APEX specifies its custom signing tool (custom_sign_tool:),
apexkeys.txt contains the info and sign_target_files_apks pass the value
to apex_util.

For now the Virt APEX has its own custom signing tool (sign_virt_apex),
which is added to OTATOOLS.

Bug: 193504286
Test: sign_target_files_apks invokes sign_virt_apex
Change-Id: Iba845723fe3e18f542963324b9c58cd00914c5ba
2021-10-27 16:56:22 +09:00
Jooyung Han
0f5a41da56 sign_apex --sign_tool
A new argument is a custom signing tool for APEX contents. When
specified, apex_util invokes the tool with payload's key and payload
directory.

For now, the Virt APEX has its own custom signing tool (sign_virt_apex)
to re-sign filesystem images in it.

Bug: 193504286
Test: atest releasetools_test
Test: m sign_apex sign_virt_apex
Test: sign_apex --sign_tool sign_virt_apex --payload_key ..
  --container_key .. resigned.apex
      adb install resigned.apex
      reboot & vm run-app
Change-Id: Ic4d369c2ba42a8295044a0a75e054dc8def93208
2021-10-27 16:56:22 +09:00
Jooyung Han
c4b7b34b4b fix releasetools_test
Some test cases are failing because tests are not updated accordinly.

Bug: n/a
Test: atest releasetools_test
Change-Id: I3df071f72f01dedd6df4fa462ca52b8a0b1ffd4e
2021-10-27 16:06:10 +09:00
Treehugger Robot
2992dccd43 Merge "Allow data wipe for full OTAs" 2021-10-27 04:53:16 +00:00
Kelvin Zhang
81641afae4 Allow data wipe for full OTAs
Test: th
Change-Id: I7a751b461a5442d22ebca07c98624f35c02dca15
2021-10-26 16:03:25 -07:00
Lingfeng Guan
8e7cdf65d1 SignApk - support loading private keys from pkcs#11 keystore
Summary:
Add two flags to load the keys from pkcs#11 keystore. When the option
-loadPrivateKeysFromKeyStore is specified, will load private keys from
the keystore with specified keyStoreName instead of load from file.

Test: make dist for arm_sunfish-user, which includes apk
and ota (wholefile) signing
Test:
- manually call signapk in Java11 (java9 may need additional
change to support), with statically registered pkcs#11 keystore, signed
both apk and ota-package.
- verified using apksigner and extracting otacert from ota-package, both
correct

Change-Id: I3efb8017f73d3d992c07ed4562acfef016a109fe
2021-10-26 12:13:40 -07:00
Sasha Smundak
2107a731b4 Rename cmd/main.go to cmd/canoninja.go
So that it can be built and installed with
````
(cd build/make/tools/canoninja && go install cmd/canoninja.go)
```

Bug: 201713929
Test: internal
Change-Id: I38133bf26ccfae5ebf8bc3c68bc595b7274576b9
2021-10-20 15:12:07 -07:00
Treehugger Robot
3bd9c7d36a Merge "A tool to facilitate large ninja files comparison." 2021-10-19 19:11:35 +00:00
Sasha Smundak
26c705f764 A tool to facilitate large ninja files comparison.
Bug: 201713929
Test: internal
Change-Id: Ifd976eed1e58b7409e3deacf99917206f0149ade
2021-10-19 09:48:04 -07:00
Treehugger Robot
d8f17b1fc4 Merge "Add pcluster and deduplication support for erofs images." 2021-10-16 01:56:43 +00:00
Treehugger Robot
777ef8384c Merge "sign_target_files_apks: adding --allow_gsi_debug_sepolicy" 2021-10-15 05:41:55 +00:00
David Anderson
fce20c249e Merge "Add blocklist support for EROFS." 2021-10-15 05:41:06 +00:00
David Anderson
333d256b39 Merge "Clean up code duplication in build_image.py." 2021-10-15 01:31:48 +00:00
Treehugger Robot
0a467339ca Merge "Fix formatting of rbcrun errors" 2021-10-14 22:22:10 +00:00
Bowgo Tsai
2a781692ae sign_target_files_apks: adding --allow_gsi_debug_sepolicy
https://android-review.googlesource.com/q/topic:gsi_debug_policy
adds userdebug_plat_sepolicy.cil into the GSI system.img to
reduce the steps of repacking a debug ramdisk.

This CL checks that the file userdebug_plat_sepolicy.cil shouldn't
exist before signing, unless the caller explicitly specifies
--allow_gsi_debug_sepolicy to allow it.

Note: also fixes the indentation around the block.

Bug: 201482141
Test: sign_target_files_apks *-target_files-*.zip signed.zip
Change-Id: I56ed328a9ae70cf49dbd3c6efb5a4a8c54e1b7a7
2021-10-14 16:54:37 +08:00
Cole Faust
e95122ea14 Fix formatting of rbcrun errors
Was using println instead of printf, and not expanding
the variable arguments.

Test: Manually
Change-Id: I70cd44c11a9b887e396dfa48ba445e2453d3da9b
2021-10-13 12:15:21 -07:00
David Anderson
64b351b882 Add pcluster and deduplication support for erofs images.
This adds BOARD_xIMAGE_EROFS_PCLUSTER_SIZE and
BOARD_EROFS_PCLUSTER_SIZE, which set the "pcluster size" of erofs images
for individual images or all erofs images respectively. The pcluster
size affects the maximum size of a physical compressed block.

This also adds BOARD_EROFS_SHARE_DUP_BLOCKS, which turns on chunk
support in EROFS.

Bug: 201685920
Test: manual test
Change-Id: I27ec0899f89890562796dd9fa567fc74182fbefb
2021-10-13 01:03:10 -07:00
David Anderson
d29e5371e1 Add blocklist support for EROFS.
This also fixes argument ordering for mkerofsimage.sh.

Bug: 201685920
Test: build OTA with erofs
Change-Id: Ic2ba794e37a0feb73a1028eb7522d35bffbf3af6
2021-10-13 01:03:08 -07:00
David Anderson
271dab6ceb Clean up code duplication in build_image.py.
This is another code block where lots of stuff is duplicated, making it
hard to add new partitions or partition features.

Bug: N/A
Test: m otapackage, treehugger
Change-Id: I4c71275303a9246b37c03b24f531925b90d26fc8
2021-10-11 21:20:35 -07:00