Commit Graph

7575 Commits

Author SHA1 Message Date
Treehugger Robot
d7d4a2d466 Merge "--sign_tool is an optional flag applicable to virtApex." am: ff3a1c73cf am: aabf327615 am: 676de41c33 am: 886bf936c3
Original change: https://android-review.googlesource.com/c/platform/build/+/1933021

Change-Id: I6f9e75a436710222941540be7cfb327698c38f10
2021-12-24 12:15:00 +00:00
Baligh Uddin
06cc018629 --sign_tool is an optional flag applicable to virtApex.
For not virtApex, this results in a KeyError.
line 151, in <module>\n', '  File
"/usr/local/google/home/baligh/clients/goog/master/out/host/linux-x86/bin/sign_apex/sign_apex.py",
line 144, in main\n', "KeyError: 'sign_tool'\n"]

BUG: 193504286
Test: TH
Change-Id: Id982e5c57086ada78168163d2293813df121847d
2021-12-24 06:17:47 +00:00
Devin Moore
5847b680c8 Merge changes from topic "init_boot_img" am: 6363a04e79 am: 1368d81efa am: 2bc48d1025 am: fa9ffed08d
Original change: https://android-review.googlesource.com/c/platform/build/+/1920818

Change-Id: I0f56b8f39335731483ee004bc18a5153518f4423
2021-12-23 00:30:02 +00:00
Devin Moore
6363a04e79 Merge changes from topic "init_boot_img"
* changes:
  Update releasetools for new init_boot.img
  Build a new init_boot.img that contains the boot ramdisk
2021-12-22 23:21:17 +00:00
Treehugger Robot
8b631bf35e Merge "Fix some python2/3 migration issues" am: ceb81e2bef am: fa3f8bdd90 am: 949f4fc6b6 am: 90c9ad6f8b
Original change: https://android-review.googlesource.com/c/platform/build/+/1930407

Change-Id: Ieb08fc938455e0a6f97e01bea9bae99ef7c34e8c
2021-12-22 07:18:25 +00:00
Treehugger Robot
42af7d7584 Merge changes I8d60efc5,Ie291ea87 am: 01ca0d0c5e am: 6a4a3d0ed7 am: 2cbdab25ed am: 17a941e0d3
Original change: https://android-review.googlesource.com/c/platform/build/+/1900893

Change-Id: I7ce3c5657b150e048fa1f497a0349cae3ecdab94
2021-12-22 07:17:59 +00:00
Treehugger Robot
ceb81e2bef Merge "Fix some python2/3 migration issues" 2021-12-22 06:33:32 +00:00
Treehugger Robot
01ca0d0c5e Merge changes I8d60efc5,Ie291ea87
* changes:
  SignApk - change signature of readPassword to use char[] instead
  SignApk - use existing password mechanism when using keystore
2021-12-22 06:23:19 +00:00
Treehugger Robot
7a8d144644 Merge "Update comment up to date" am: cf7300e9c2 am: a93b9ff575 am: ba37412ac7 am: fcf7cd2d05
Original change: https://android-review.googlesource.com/c/platform/build/+/1928383

Change-Id: I78ca8faa5c512216da6977d406933292b24275f2
2021-12-22 01:53:00 +00:00
weisu
eae45315cd Write files in a temporary directory
Avoid writing test files in the build output directory which fails when
run with Bazel. This happens because Bazel's sandboxing environment
ensures that the test's working directory is unwritable.

See https://docs.bazel.build/versions/main/sandboxing.html for more
information.

Bug: 209687942
Test: atest --bazel-mode zipalign_tests
Test: atest zipalign_tests

Change-Id: Ie22f464830c1ffe4d38a94a16dbd39dafa7fe317
2021-12-22 00:06:14 +00:00
Kelvin Zhang
0d0ca5d6a0 Fix some python2/3 migration issues
Exception objects don't necessarily have a .message member, use str()
instead.

Test: th
Change-Id: I57b68dff1ace4937d6ef3b7c564f031e219491b1
2021-12-21 12:31:55 -08:00
Inseob Kim
e7b222aa9e Update comment up to date
nonplat* is renamed to vendor*.

Bug: 210536608
Test: N/A
Change-Id: Ief699bf11395a5b0d9bee22220c09acac6e450f2
2021-12-21 15:57:03 +09:00
Devin Moore
afdd7c74bc Update releasetools for new init_boot.img
This new init_boot.img contains the ramdisk that used to reside in the
boot.img file.

Test: set BOARD_PREBUILT_INIT_BOOT_IMAGE to an external init_boot.img
      - Check that "m" pulls in the init_boot.img to
      out/target/product/vsoc_x86_64/
      - Check that "m dist" adds the init_boot.img to
      aosp_cf_x86_64_phone-img-eng.devinmoore.zip
Test: atest --host releasetools_test
Bug: 203698939
Change-Id: If7ef2cf093e5e525529c7c44333c0f40f6ba0764
2021-12-20 20:54:23 +00:00
Inseob Kim
ea3444a05d Merge changes I6f61a908,Iafd22881 am: 32961d0203 am: 7cc500615e am: eef47a47e4 am: 1c4355d7b2
Original change: https://android-review.googlesource.com/c/platform/build/+/1922320

Change-Id: I9058e470e2c8e1adfae4550204c1a67cc3600e21
2021-12-15 08:32:44 +00:00
Inseob Kim
067492988a fsverity_metadata: Support PEM key
When we have a PEM key, we don't need the process converting a DER key
to PEM format, but we just need to use the PEM key as-is.

Bug: 205987437
Test: build and manual test
Change-Id: I6f61a9088efc0f7193737d3c33b8cfde399b2b6f
2021-12-15 09:41:39 +09:00
Inseob Kim
2d03493341 Add fsverity_metadata_generator helper binary
Making this a host tool will help users generate their own fsverity
metadata easily.

Bug: 205987437
Test: m fsverity_metadata_generator and run it
Change-Id: Iafd228815a74d298d87ca1466c6909c0d24c5874
2021-12-15 09:41:29 +09:00
Ben Fennema
6082d0aeb5 Add --avb_recovery_* options to replace recovery avb key
Test: run sign_target_files_apks with --avb_recovery_key to specify a different key and check with avbtool the key was replaced
Bug: 210126985
Signed-off-by: Ben Fennema <fennema@google.com>
Change-Id: Ic2bb3f6855a49ec065a4c778c429ff076902b95c
2021-12-11 14:05:06 -08:00
Treehugger Robot
ccdfbfc3de Merge "license metadata remove path on top-down walk" am: 159ae38aaa am: ccfd736eda am: d009ec93e6 am: 4fda5ba158
Original change: https://android-review.googlesource.com/c/platform/build/+/1915582

Change-Id: Iff1ee33abc19b38a67f751f07f1e51f83158ac60
2021-12-09 01:10:40 +00:00
Treehugger Robot
44f30de96e Merge "license metadata performance" am: d497acc1c9 am: 1fb8dbd1c8 am: 8b18caeeca am: dee8c12a0a
Original change: https://android-review.googlesource.com/c/platform/build/+/1914147

Change-Id: Iee703eca9967a972cfed1152780ec85aa05f828e
2021-12-09 00:09:07 +00:00
Bob Badour
b285515ca1 license metadata remove path on top-down walk
Performance optimization means not every path will be traversed.

Instead of updating parents via the path, perform a 2nd bottom-up walk
after the top-down walk to propagate the new resolutions to parents.

Note: the 2nd walk method will add resolutions to statically linked
libraries etc. at deeper levels, but those do not affect what gets
reported. In particular, note that test data for dumpresolutions
changes, but none of the test data for listshare, checkshare etc.
changes.

Test: m all systemlicense listshare checkshare dumpgraph dumpresolutions

Bug: 68860345
Bug: 151177513
Bug: 151953481
Change-Id: I76361c4e33bbadbbea38cbec260430e8f9407628
2021-12-08 12:52:59 -08:00
Bob Badour
3a820dd5a0 license metadata performance
Tune the top-down walk to avoid needlessly walking the same subtree
over and over again with the same condition(s).

Takes walking system image down from 3m to 1.5s.

Test: m all systemlicense listshare checkshare dumpgraph dumpresolutions

Bug: 68860345
Bug: 151177513
Bug: 151953481

Change-Id: I4354800cd8dfc42efd4df274d2ce45eaa3e0a99f
2021-12-07 15:36:10 -08:00
Treehugger Robot
acb47f6abf Merge changes I5d48eaba,I4ff3f988 am: 203bbf3254 am: 195ace0fa6 am: 414d918656 am: c4f1a012c9
Original change: https://android-review.googlesource.com/c/platform/build/+/1870079

Change-Id: Iec28602c69315095034578552ab0b026f89178f6
2021-12-07 04:28:44 +00:00
Treehugger Robot
7bd1126ea6 Merge changes I98694001,I5fe57d36 am: ea1a012972 am: a1a1606184 am: 63f1664541 am: 55bec85e70
Original change: https://android-review.googlesource.com/c/platform/build/+/1911224

Change-Id: I7d29101fdf5be5f701414ab02c3e0247e9dc2a66
2021-12-07 04:04:08 +00:00
Bob Badour
afaeb6ae5f compliance package documentation
Bug: 68860345
Bug: 151177513
Bug: 151953481

Test: m all
Test: m systemlicense
Test: m listshare; out/soong/host/linux-x86/bin/listshare ...
Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ...
Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ...
Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ...

where ... is the path to the .meta_lic file for the system image. In my
case if

$ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD)

... can be expressed as:

${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic

Change-Id: I5d48eababce7bba39795d3668eee86b332cbe43d
2021-12-06 16:39:35 -08:00
Bob Badour
6dd0035e04 compliance package: listshare and checkshare
package to read, consume, and analyze license metadata and dependency
graph.

Includes the below command-line tools:

listshare outputs csv of projects to share to meet restricted and
reciprocal license requirements with one project per line. The first
field is the path to the project, and subsequent fields identify the
license resolutions as colon-separated target:annotations tuples.

checkshare outputs error messages to stderr for any targets where
policy dictates both sharing and not sharing the source-code, and PASS
or FAIL to stdout. exit status indicates success 0 or conflict found 1

Bug: 68860345
Bug: 151177513
Bug: 151953481

Test: m all
Test: m systemlicense
Test: m listshare; out/soong/host/linux-x86/bin/listshare ...
Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ...
Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ...
Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ...

where ... is the path to the .meta_lic file for the system image. In my
case if

$ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD)

... can be expressed as:

${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic

Change-Id: I4ff3f98848f7e6d03a35734300d763ef5f245d53
2021-12-06 16:39:35 -08:00
Bob Badour
1ded0a1b18 compliance package: dumpresolutions
package to read, consume, and analyze license metadata and dependency
graph.

Includes the below command-line tool:

dumpresolutions outputs the resulting set of resolutions after the
bottom-up and top-down resolves, or after joining 1 or more condition
walks.

Bug: 68860345
Bug: 151177513
Bug: 151953481

Test: m all
Test: m systemlicense
Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ...
Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ...

where ... is the path to the .meta_lic file for the system image. In my
case if

$ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD)

... can be expressed as:

${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic

Change-Id: I9869400126cd7ad4b7376b0bab31b46aad732f5d
2021-12-06 16:38:08 -08:00
Bob Badour
fa739da020 compliance package for license metadata: dumpgraph
package to read, consume, and analyze license metadata and dependency
graph.

Includes testdata/ and the the below command-line tool:

dumpgraph outputs edges of the graph as "target dependency annotations"

Bug: 68860345
Bug: 151177513
Bug: 151953481

Test: m all
Test: m systemlicense
Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ...

where ... is the path to the .meta_lic file for the system image. In my
case if

$ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD)

... can be expressed as:

${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic

Change-Id: I5fe57d361da5155dbcb2c0d369626e9200c9d664
2021-12-06 16:37:17 -08:00
Treehugger Robot
9622a15023 Merge "compliance package policy and resolves" am: 45deca7ff7 am: e7d26816bb am: dfd9be96ce am: a24332dbde
Original change: https://android-review.googlesource.com/c/platform/build/+/1870078

Change-Id: I966e5796e3a51d80e59034529cdd3f544b7f7cbc
2021-12-04 04:30:49 +00:00
Treehugger Robot
44d6dfc930 Merge "compliance package structures for license metadata" am: f645c5041e am: 8b078073c0 am: ad7b35b7a8 am: 1cb7555db6
Original change: https://android-review.googlesource.com/c/platform/build/+/1906558

Change-Id: I9324f5eb0ac77ed1c280b361cc0a79df8e5eaba5
2021-12-04 03:59:02 +00:00
Bob Badour
9ee7d03e1c compliance package policy and resolves
package to read, consume, and analyze license metadata and dependency
graph.

Bug: 68860345
Bug: 151177513
Bug: 151953481

Change-Id: Ic08406fa2250a08ad26f2167d934f841c95d9148
2021-12-03 15:52:48 -08:00
Bob Badour
a99ac620dd compliance package structures for license metadata
package to read, consume, and analyze license metadata and dependency
graph.

Bug: 68860345
Bug: 151177513
Bug: 151953481
Change-Id: I3ebf44e4d5195b9851fd076161049bf82ed76dd2
2021-12-03 15:52:48 -08:00
Tianjie Xu
c736bb90be Merge "Revert "Temporarily remove the care_map in the ota package"" am: fab626b7d0 am: 1cdcb088bb am: a8c667a4b7 am: b6765ff3e5
Original change: https://android-review.googlesource.com/c/platform/build/+/1907490

Change-Id: Ie3757b54a44a507a884c369cce760e9c82ac1bab
2021-12-01 19:19:05 +00:00
Tianjie Xu
fab626b7d0 Merge "Revert "Temporarily remove the care_map in the ota package"" 2021-12-01 18:23:41 +00:00
Tianjie Xu
2f76f90616 Revert "Temporarily remove the care_map in the ota package"
This reverts commit 5856cfbf82.

Reason for revert: re-enable care_map b/205541521

Change-Id: I64b4637991a80c9d08a28bdd324c47b4067b1762
2021-12-01 05:54:27 +00:00
David Anderson
2218b091dd Merge "build_image: Query the actual image size after invoking the final mkfs." am: 0096a750c2 am: 965ca0d965 am: f301d211d8 am: 161afa9b6a
Original change: https://android-review.googlesource.com/c/platform/build/+/1889816

Change-Id: I0c3d47e4d590c7d52e5a8ec431c6ec6bdb2764b2
2021-12-01 04:30:08 +00:00
David Anderson
009d6f8c73 build_image: Query the actual image size after invoking the final mkfs.
It's not guaranteed that the requested image size to mkfs is precisely
respected, due to metadata alignment and such. For accurate care maps
use the real image size rather than requested.

Bug: 205541521
Test: smartsync to 7892270, check that care_map.pb has the right block
      count
Change-Id: I60fe64f720db13d3c3c4f1d8968341d7293217c9
2021-11-30 15:13:56 -08:00
Lingfeng Guan
c549841617 SignApk - change signature of readPassword to use char[] instead
Summary:
Use char[] is more conventional for password handling. See this question
for reference.
https://stackoverflow.com/questions/8881291

This is to address a concern raised in
https://android-review.googlesource.com/c/platform/build/+/1890395/2

Test: mma
Change-Id: I8d60efc557d7641c057e49a2aa4613fea67cd1e6
2021-11-23 23:53:08 +00:00
Michael Groover
4d12cbfb68 Merge "SignApk - support loading private keys from pkcs#11 keystore" am: 495fafef6f am: 5d6aa9a5b8 am: a8fcebeb02 am: 77614a40bf
Original change: https://android-review.googlesource.com/c/platform/build/+/1849654

Change-Id: If6da3cfa599cb05f9bdf06b064a0e79c9d58944a
2021-11-23 21:16:45 +00:00
Lingfeng Guan
65672df61f SignApk - use existing password mechanism when using keystore
Summary:
In my last diff, I've added mechanism to load private key from keystore.
However, that mechanism will reveal password as part of the java param.
This diff tries to use existing ANDROID_PW_FILE mechanism to support
password for keystore private keys (through stdin)

This diff also fix a null pointer bug in the existing password handling

Test: This diff has been tested locally, and could sign correctly with our
  keystore with or without password

Tags:
Change-Id: Ie291ea8702a3b4d270b0f8689b023c3f290980a7
2021-11-23 12:02:16 -08:00
Michael Groover
495fafef6f Merge "SignApk - support loading private keys from pkcs#11 keystore" 2021-11-23 19:53:38 +00:00
Treehugger Robot
c8a2df3f56 Merge "Fix signapk not found error when running ota_from_target_files locally" am: 1fb1088e88 am: 6f017f5ecd am: 203b192030 am: e7050a1e29
Original change: https://android-review.googlesource.com/c/platform/build/+/1894855

Change-Id: Ia5a9c22f62953a4d7fb414a5b0a3ba91d3857674
2021-11-20 00:19:38 +00:00
Treehugger Robot
1fb1088e88 Merge "Fix signapk not found error when running ota_from_target_files locally" 2021-11-19 23:25:05 +00:00
Kelvin Zhang
4fc3aa0bc3 Fix signapk not found error when running ota_from_target_files locally
location of ota_from_target_files changed from
out/host/linux-x86/bin to
out/soong/host/linux-x86/bin . This changes relative position of
signapj.jar. To fix, use ANDROID_HOST_OUT as search path

Change-Id: I5397171566e9d7598b5ef16ae26641f0c183d748
2021-11-19 11:34:54 -08:00
Treehugger Robot
10f69082b6 Merge changes from topic "fsverity_digest" am: 866d824328 am: f90fb54430 am: 173e8a3f90 am: 53a0a19fef
Original change: https://android-review.googlesource.com/c/platform/build/+/1855993

Change-Id: I82d7ce6062a8b253644d8c93bc598da9b745bbef
2021-11-12 07:37:38 +00:00
Inseob Kim
f69346e0a8 Add fsverity digest manifest
fsverity digest manifest stores a map from files to fsverity digests.
The manifest is installed as a serialized protobuf file, to a signed apk
system/etc/security/fsverity/BuildManifest.apk.

Bug: 193113311
Test: build with PRODUCT_SYSTEM_FSVERITY_GENERATE_METADATA := true
Change-Id: I55fc10400206b8ce0d5f198faea08fe3930b362c
2021-11-11 11:19:44 +00:00
Inseob Kim
9cda397948 Implement fsverity metadata generator
Using fsverity tool, fsverity metadata for specific artifacts in system
mage can be generated. Users can do that by setting a makefile variable
PRODUCT_SYSTEM_FSVERITY_GENERATE_METADATA to true.

If set to true, the following artifacts will be signed.

- system/framework/*.jar
- system/framework/oat/<arch>/*.{oat,vdex,art}
- system/etc/boot-image.prof
- system/etc/dirty-image-objects

One fsverity metadata container file per one input file will be
generated in system.img, with a suffix ".fsv_meta". e.g. a container
file for "system/framework/foo.jar" will be
"system/framework/foo.jar.fsv_meta".

Bug: 193113311
Test: build with PRODUCT_SYSTEM_FSVERITY_GENERATE_METADATA := true
Change-Id: Ib70d591a72d23286b5debcb05fbad799dfd79b94
2021-11-11 17:30:19 +09:00
Tianjie Xu
befecc0ee2 Merge "Fix python3 incompatiable code" am: 7a048996b7 am: 4d6bc649e8 am: bff0316e3c am: 1fbeb8b694 am: 29ef79795f
Original change: https://android-review.googlesource.com/c/platform/build/+/1886461

Change-Id: I51bdc9e85b4230fc0bfe0f882032ba666e12cf19
2021-11-10 18:11:32 +00:00
Tianjie
b049191e78 Fix python3 incompatiable code
http://aosp/1883069 switch the releasetool to use python3.
But target_files_diff still have py2 code that cause failures.
Fix that to unblock OTA generation.

Bug: 205790608
Test: generate an incremental OTA
Change-Id: Ib4d86dc1842afeae8b35681c7d809da140fac600
2021-11-09 23:37:11 -08:00
Treehugger Robot
212fac995e Merge "Skip signing key check in some cases" am: 984ad44276 am: 165b1be894 am: 92327efd67 am: d9d006e03b am: b00d947ed6
Original change: https://android-review.googlesource.com/c/platform/build/+/1885988

Change-Id: I51888d4a3a9b77726016371331e758494c0c69c3
2021-11-10 02:18:41 +00:00
Treehugger Robot
984ad44276 Merge "Skip signing key check in some cases" 2021-11-10 00:48:27 +00:00