Commit Graph

7546 Commits

Author SHA1 Message Date
Bob Badour
9ee7d03e1c compliance package policy and resolves
package to read, consume, and analyze license metadata and dependency
graph.

Bug: 68860345
Bug: 151177513
Bug: 151953481

Change-Id: Ic08406fa2250a08ad26f2167d934f841c95d9148
2021-12-03 15:52:48 -08:00
Bob Badour
a99ac620dd compliance package structures for license metadata
package to read, consume, and analyze license metadata and dependency
graph.

Bug: 68860345
Bug: 151177513
Bug: 151953481
Change-Id: I3ebf44e4d5195b9851fd076161049bf82ed76dd2
2021-12-03 15:52:48 -08:00
Tianjie Xu
c736bb90be Merge "Revert "Temporarily remove the care_map in the ota package"" am: fab626b7d0 am: 1cdcb088bb am: a8c667a4b7 am: b6765ff3e5
Original change: https://android-review.googlesource.com/c/platform/build/+/1907490

Change-Id: Ie3757b54a44a507a884c369cce760e9c82ac1bab
2021-12-01 19:19:05 +00:00
Tianjie Xu
fab626b7d0 Merge "Revert "Temporarily remove the care_map in the ota package"" 2021-12-01 18:23:41 +00:00
Tianjie Xu
2f76f90616 Revert "Temporarily remove the care_map in the ota package"
This reverts commit 5856cfbf82.

Reason for revert: re-enable care_map b/205541521

Change-Id: I64b4637991a80c9d08a28bdd324c47b4067b1762
2021-12-01 05:54:27 +00:00
David Anderson
2218b091dd Merge "build_image: Query the actual image size after invoking the final mkfs." am: 0096a750c2 am: 965ca0d965 am: f301d211d8 am: 161afa9b6a
Original change: https://android-review.googlesource.com/c/platform/build/+/1889816

Change-Id: I0c3d47e4d590c7d52e5a8ec431c6ec6bdb2764b2
2021-12-01 04:30:08 +00:00
David Anderson
009d6f8c73 build_image: Query the actual image size after invoking the final mkfs.
It's not guaranteed that the requested image size to mkfs is precisely
respected, due to metadata alignment and such. For accurate care maps
use the real image size rather than requested.

Bug: 205541521
Test: smartsync to 7892270, check that care_map.pb has the right block
      count
Change-Id: I60fe64f720db13d3c3c4f1d8968341d7293217c9
2021-11-30 15:13:56 -08:00
Lingfeng Guan
c549841617 SignApk - change signature of readPassword to use char[] instead
Summary:
Use char[] is more conventional for password handling. See this question
for reference.
https://stackoverflow.com/questions/8881291

This is to address a concern raised in
https://android-review.googlesource.com/c/platform/build/+/1890395/2

Test: mma
Change-Id: I8d60efc557d7641c057e49a2aa4613fea67cd1e6
2021-11-23 23:53:08 +00:00
Michael Groover
4d12cbfb68 Merge "SignApk - support loading private keys from pkcs#11 keystore" am: 495fafef6f am: 5d6aa9a5b8 am: a8fcebeb02 am: 77614a40bf
Original change: https://android-review.googlesource.com/c/platform/build/+/1849654

Change-Id: If6da3cfa599cb05f9bdf06b064a0e79c9d58944a
2021-11-23 21:16:45 +00:00
Lingfeng Guan
65672df61f SignApk - use existing password mechanism when using keystore
Summary:
In my last diff, I've added mechanism to load private key from keystore.
However, that mechanism will reveal password as part of the java param.
This diff tries to use existing ANDROID_PW_FILE mechanism to support
password for keystore private keys (through stdin)

This diff also fix a null pointer bug in the existing password handling

Test: This diff has been tested locally, and could sign correctly with our
  keystore with or without password

Tags:
Change-Id: Ie291ea8702a3b4d270b0f8689b023c3f290980a7
2021-11-23 12:02:16 -08:00
Michael Groover
495fafef6f Merge "SignApk - support loading private keys from pkcs#11 keystore" 2021-11-23 19:53:38 +00:00
Treehugger Robot
c8a2df3f56 Merge "Fix signapk not found error when running ota_from_target_files locally" am: 1fb1088e88 am: 6f017f5ecd am: 203b192030 am: e7050a1e29
Original change: https://android-review.googlesource.com/c/platform/build/+/1894855

Change-Id: Ia5a9c22f62953a4d7fb414a5b0a3ba91d3857674
2021-11-20 00:19:38 +00:00
Treehugger Robot
1fb1088e88 Merge "Fix signapk not found error when running ota_from_target_files locally" 2021-11-19 23:25:05 +00:00
Kelvin Zhang
4fc3aa0bc3 Fix signapk not found error when running ota_from_target_files locally
location of ota_from_target_files changed from
out/host/linux-x86/bin to
out/soong/host/linux-x86/bin . This changes relative position of
signapj.jar. To fix, use ANDROID_HOST_OUT as search path

Change-Id: I5397171566e9d7598b5ef16ae26641f0c183d748
2021-11-19 11:34:54 -08:00
Treehugger Robot
10f69082b6 Merge changes from topic "fsverity_digest" am: 866d824328 am: f90fb54430 am: 173e8a3f90 am: 53a0a19fef
Original change: https://android-review.googlesource.com/c/platform/build/+/1855993

Change-Id: I82d7ce6062a8b253644d8c93bc598da9b745bbef
2021-11-12 07:37:38 +00:00
Inseob Kim
f69346e0a8 Add fsverity digest manifest
fsverity digest manifest stores a map from files to fsverity digests.
The manifest is installed as a serialized protobuf file, to a signed apk
system/etc/security/fsverity/BuildManifest.apk.

Bug: 193113311
Test: build with PRODUCT_SYSTEM_FSVERITY_GENERATE_METADATA := true
Change-Id: I55fc10400206b8ce0d5f198faea08fe3930b362c
2021-11-11 11:19:44 +00:00
Inseob Kim
9cda397948 Implement fsverity metadata generator
Using fsverity tool, fsverity metadata for specific artifacts in system
mage can be generated. Users can do that by setting a makefile variable
PRODUCT_SYSTEM_FSVERITY_GENERATE_METADATA to true.

If set to true, the following artifacts will be signed.

- system/framework/*.jar
- system/framework/oat/<arch>/*.{oat,vdex,art}
- system/etc/boot-image.prof
- system/etc/dirty-image-objects

One fsverity metadata container file per one input file will be
generated in system.img, with a suffix ".fsv_meta". e.g. a container
file for "system/framework/foo.jar" will be
"system/framework/foo.jar.fsv_meta".

Bug: 193113311
Test: build with PRODUCT_SYSTEM_FSVERITY_GENERATE_METADATA := true
Change-Id: Ib70d591a72d23286b5debcb05fbad799dfd79b94
2021-11-11 17:30:19 +09:00
Tianjie Xu
befecc0ee2 Merge "Fix python3 incompatiable code" am: 7a048996b7 am: 4d6bc649e8 am: bff0316e3c am: 1fbeb8b694 am: 29ef79795f
Original change: https://android-review.googlesource.com/c/platform/build/+/1886461

Change-Id: I51bdc9e85b4230fc0bfe0f882032ba666e12cf19
2021-11-10 18:11:32 +00:00
Tianjie
b049191e78 Fix python3 incompatiable code
http://aosp/1883069 switch the releasetool to use python3.
But target_files_diff still have py2 code that cause failures.
Fix that to unblock OTA generation.

Bug: 205790608
Test: generate an incremental OTA
Change-Id: Ib4d86dc1842afeae8b35681c7d809da140fac600
2021-11-09 23:37:11 -08:00
Treehugger Robot
212fac995e Merge "Skip signing key check in some cases" am: 984ad44276 am: 165b1be894 am: 92327efd67 am: d9d006e03b am: b00d947ed6
Original change: https://android-review.googlesource.com/c/platform/build/+/1885988

Change-Id: I51888d4a3a9b77726016371331e758494c0c69c3
2021-11-10 02:18:41 +00:00
Treehugger Robot
984ad44276 Merge "Skip signing key check in some cases" 2021-11-10 00:48:27 +00:00
Tianjie Xu
c67a611103 Merge "Temporarily remove the care_map in the ota package" am: 3fcc52519e am: c8e5f1de93 am: 4cd4b0c572 am: 16b4beb644 am: edac9f8897
Original change: https://android-review.googlesource.com/c/platform/build/+/1883497

Change-Id: Ief5ef5829293048b82ebfdc3425fe91e9c7d74d9
2021-11-09 21:39:46 +00:00
Cole Faust
391cbc69b2 Merge "Use python 3 for release tools" am: ece722ca92 am: 9220d1a841 am: 454757ca83 am: 6df2faf751 am: f05bbe47e1
Original change: https://android-review.googlesource.com/c/platform/build/+/1883069

Change-Id: I1db1436cfef34ce4d0c4bff7e91556eb3cf564a1
2021-11-09 21:39:16 +00:00
Tianjie Xu
3fcc52519e Merge "Temporarily remove the care_map in the ota package" 2021-11-09 19:38:14 +00:00
Kelvin Zhang
4b588cf75a Skip signing key check in some cases
Test: th
Bug: 205657204

Change-Id: I3a2fc886cfb3fe3d53619bfb8fe84aa799d80de3
2021-11-09 08:43:01 -08:00
Tianjie
5856cfbf82 Temporarily remove the care_map in the ota package
The care_map has incorrect size for non-sparse images. Temporarily
removes it until the root cause is fixed

Bug: 205541521
Test: build
Change-Id: I76bcd2c0c778566b4a6c69b6d45428952225d406
2021-11-08 22:55:53 -08:00
Cole Faust
c76931fe8b Use python 3 for release tools
The release tools have already been updated in prior cls to
support python 3. To test this, I added code to print the script
name to a temp file at the beginning of every script, and then
ran various builds, ensuring that the scripts were run afterwards.

The builds run:
m brillo_update_payload checkvintf minigzip lz4 toybox unpack_bootimg deapexer (needed for releasetools_test)
m (as both sdk_phone_x86_64 and beagle_x15)
m target-files-package (as both sdk_phone_x86_64 and beagle_x15)
m releasetools_test && out/host/linux-x86/nativetest64/releasetools_test/releasetools_test (25 errors with and without python3)
m check_target_files_signatures apksigner target-files-package && out/host/linux-x86/bin/check_target_files_signatures out/target/product/emulator_x86_64/obj/PACKAGING/target_files_intermediates/sdk_phone_x86_64-target_files-eng.colefaust.zip
m dist
As aosp_cf_x86_64_phone: m dist && sign_target_files_apks  out/dist/aosp_cf_x86_64_phone-target_files-eng.$USER.zip /tmp/signed_target_files.zip && validate_target_files /tmp/signed_target_files.zip

This hit all the scripts except for make_recovery_patch,
merge_builds, and ota_package_parser.

I couldn't find anything that uses merge_builds, so it must
only be run manually.

make_recovery_patch and ota_package_parser are only run if
TARGET_OTA_ALLOW_NON_AB is true (among other things), which
is not the case for any product in aosp.

Test: Described in commit message + presubmits
Change-Id: I1a29eafa7ff1a69973b27055e311de77f7ee628b
2021-11-05 13:47:28 -07:00
Cole Faust
6f396911fc Merge "Prepare release tools for python 3" am: fdb8f17a83 am: ad943ea6df am: fecb7bb3a4 am: 48aac85b06 am: 67fdf2968b
Original change: https://android-review.googlesource.com/c/platform/build/+/1874167

Change-Id: I7a5aefb927ae70a86cb9d3f4b69593c37db536f6
2021-11-05 18:36:07 +00:00
Cole Faust
b820bcd829 Prepare release tools for python 3
- Sort dictionaries before looping over them
- Don't call sorted() on lists with Nones
- Open file in binary format when serializing protobufs

Change-Id: If5dbc908f7125f6184014b3c1c7891f833d1d8bf
Bug: 203436762
Test: Presubmits
2021-11-04 17:58:44 -07:00
Treehugger Robot
6ebf09d040 Merge "Fix a failure in signing test" am: 936e704999 am: 5c8fc606d4 am: aeba7359a0 am: 86250621a3 am: ce19509a15
Original change: https://android-review.googlesource.com/c/platform/build/+/1879016

Change-Id: Ib3106fa7f14873ad1b0d5d11b06a2e570bc7a3f4
2021-11-04 10:37:42 +00:00
Treehugger Robot
c62a01004a Merge "Add a compressor types option" am: 1e392720c2 am: a0c34f2c21 am: 44c33933c0 am: 99a9435e6c am: cd3ca41f20
Original change: https://android-review.googlesource.com/c/platform/build/+/1873795

Change-Id: I490cf27520ac8e85e20ca8e790c35ad512683610
2021-11-04 05:43:24 +00:00
Kelvin Zhang
2639048c21 Fix a failure in signing test
Pixel moved away from sparse images, so validate_target_files is failing
because it expects sparse images.

Test: th
Change-Id: I322ff10c2afbacfb4d78991be60c11aac92a6d4c
2021-11-03 21:43:22 -07:00
Treehugger Robot
1e392720c2 Merge "Add a compressor types option" 2021-11-04 04:13:13 +00:00
Treehugger Robot
ce15a031ce Merge changes from topic "tproto" am: cd128891ed am: 0f0e4d23e5 am: 40bd79e8b7 am: aa3348dd1b am: d1a921ec8b
Original change: https://android-review.googlesource.com/c/platform/build/+/1865920

Change-Id: I16f08374a3c9aa1d561fb9653cd2cbad515a1388
2021-11-03 20:15:35 +00:00
Bob Badour
380a1afb1a Add macros for non-module license metadata and deps am: 879cfa867d am: 1efd13a2a1 am: c1d98c6310 am: e6d1b1d160 am: d57a36fadc
Original change: https://android-review.googlesource.com/c/platform/build/+/1741493

Change-Id: I0c8fc45c6b8c7a76ed6411437513b6ea8997127d
2021-11-03 20:15:33 +00:00
Kelvin Zhang
9b6d4ec01b Add a compressor types option
Test: th
Change-Id: I1ff1bccf179bfd87ce9e08def0e5e38921d7d158
2021-11-03 11:17:04 -07:00
Colin Cross
1ee26cae73 Convert .meta_lic files to textproto
Make it easier to write tools against .meta_lic files and store complex
data by writing them in textproto.

Test: builds
Change-Id: Ibbb6cfbb1bdddd3d938a86d563673a049d826d66
2021-11-03 15:44:00 +00:00
Bob Badour
879cfa867d Add macros for non-module license metadata and deps
Gets rid of .meta_module files and instead defers emitting rules until
after all the non-module targets have been processed. Allows direct
dependency on .meta_lic files, which in turn depend on license text
files.

Bug: 68860345
Bug: 151177513
Bug: 151953481

Test: m all
Test: m systemlicense
Test: m reportmissinglicenses

Change-Id: I2c467feac6e13a9366ff66f924889f1dbd48c3f1
2021-11-03 00:52:25 +00:00
Treehugger Robot
4865a3c4a7 Merge "Support custom APEX signing tool" am: 76a4c62f56 am: c24e277d03 am: 4cb46d6fc8 am: 690f6364e4 am: d6dfa2a9b4
Original change: https://android-review.googlesource.com/c/platform/build/+/1870245

Change-Id: Id1ad821e06c0bdeefd81a406bb2f7f29f0fcb41a
2021-10-30 01:06:39 +00:00
Jooyung Han
bac375bf57 Merge changes from topic "sign_apex-sign_tool" am: f531336c6f am: 01bb43478f am: f7c262f502 am: 0e7616f29b am: c7ccad5845
Original change: https://android-review.googlesource.com/c/platform/build/+/1870244

Change-Id: I6e5338dd181ae8c4a073b9d64a9de64d766d9c5a
2021-10-28 06:05:31 +00:00
Jooyung Han
ca275ba622 fix releasetools_test am: c4b7b34b4b am: 4197f3a4c6 am: c404ba2e2e am: 5f4eaf17e8 am: 24eba94ba0
Original change: https://android-review.googlesource.com/c/platform/build/+/1870249

Change-Id: Iac5e595781da028fa609cdff8f97b71e70c1bc9a
2021-10-28 06:05:30 +00:00
Rupert Shuttleworth
072daa5f98 Merge "Add ability to set --rotation-min-sdk-version for signapk." 2021-10-27 12:56:18 +00:00
Rupert Shuttleworth
a95ab436f6 Add ability to set --rotation-min-sdk-version for signapk.
Bug: 197787352

Test: m signapk

Change-Id: I3b5e9bb3985c2ae83381499a44b339a7b1deb05c
2021-10-27 12:53:16 +00:00
Jooyung Han
8caba5e14e Support custom APEX signing tool
When an APEX specifies its custom signing tool (custom_sign_tool:),
apexkeys.txt contains the info and sign_target_files_apks pass the value
to apex_util.

For now the Virt APEX has its own custom signing tool (sign_virt_apex),
which is added to OTATOOLS.

Bug: 193504286
Test: sign_target_files_apks invokes sign_virt_apex
Change-Id: Iba845723fe3e18f542963324b9c58cd00914c5ba
2021-10-27 16:56:22 +09:00
Jooyung Han
0f5a41da56 sign_apex --sign_tool
A new argument is a custom signing tool for APEX contents. When
specified, apex_util invokes the tool with payload's key and payload
directory.

For now, the Virt APEX has its own custom signing tool (sign_virt_apex)
to re-sign filesystem images in it.

Bug: 193504286
Test: atest releasetools_test
Test: m sign_apex sign_virt_apex
Test: sign_apex --sign_tool sign_virt_apex --payload_key ..
  --container_key .. resigned.apex
      adb install resigned.apex
      reboot & vm run-app
Change-Id: Ic4d369c2ba42a8295044a0a75e054dc8def93208
2021-10-27 16:56:22 +09:00
Jooyung Han
c4b7b34b4b fix releasetools_test
Some test cases are failing because tests are not updated accordinly.

Bug: n/a
Test: atest releasetools_test
Change-Id: I3df071f72f01dedd6df4fa462ca52b8a0b1ffd4e
2021-10-27 16:06:10 +09:00
Treehugger Robot
2992dccd43 Merge "Allow data wipe for full OTAs" 2021-10-27 04:53:16 +00:00
Kelvin Zhang
81641afae4 Allow data wipe for full OTAs
Test: th
Change-Id: I7a751b461a5442d22ebca07c98624f35c02dca15
2021-10-26 16:03:25 -07:00
Lingfeng Guan
8e7cdf65d1 SignApk - support loading private keys from pkcs#11 keystore
Summary:
Add two flags to load the keys from pkcs#11 keystore. When the option
-loadPrivateKeysFromKeyStore is specified, will load private keys from
the keystore with specified keyStoreName instead of load from file.

Test: make dist for arm_sunfish-user, which includes apk
and ota (wholefile) signing
Test:
- manually call signapk in Java11 (java9 may need additional
change to support), with statically registered pkcs#11 keystore, signed
both apk and ota-package.
- verified using apksigner and extracting otacert from ota-package, both
correct

Change-Id: I3efb8017f73d3d992c07ed4562acfef016a109fe
2021-10-26 12:13:40 -07:00
Sasha Smundak
2107a731b4 Rename cmd/main.go to cmd/canoninja.go
So that it can be built and installed with
````
(cd build/make/tools/canoninja && go install cmd/canoninja.go)
```

Bug: 201713929
Test: internal
Change-Id: I38133bf26ccfae5ebf8bc3c68bc595b7274576b9
2021-10-20 15:12:07 -07:00