Commit Graph

7846 Commits

Author SHA1 Message Date
Kelvin Zhang
e50bb51fa9 Add flags to override all apex/apk keys
Currently, if we want to override all apex/apk key, we must specify all apk/apex names exhaustively. This causes CLI arguments to be very long and sometimes exceeds OS's limit.

Test: run signing test locally on build 8862875 on git_master
Bug: 239991438
Change-Id: Idbd3b7eef9b2a27236f4ca7a27449fcdf0d2dd3f
2022-08-02 18:32:38 +00:00
Treehugger Robot
0b96e1a005 Merge "Fix go.mod for tools/compliance" am: f30267635e
Original change: https://android-review.googlesource.com/c/platform/build/+/2169821

Change-Id: Ibf4c760bc46a31512b2bd9ed5e6576b24fcd0479
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2022-08-02 03:44:13 +00:00
Sasha Smundak
94d473c5c7 Fix go.mod for tools/canoninja
Now it can be ld with `go build` and cross-referenced.

Test: treehugger
Change-Id: I299efc2d635176519345acf142114097be6bf814
2022-08-01 14:45:28 -07:00
Sasha Smundak
ded9acd86d Fix go.mod for tools/compliance
Now it can be built with `go build`, and can be cross-referenced.

Test: treehugger
Change-Id: Ic35631ff6a915c8cf58db14aa83c849549e64aba
2022-08-01 14:24:21 -07:00
Kelvin Zhang
89c59b8fa5 Merge "Fix typo in recently changed extra_apex_payload_key flag" am: 193de78dfd
Original change: https://android-review.googlesource.com/c/platform/build/+/2167622

Change-Id: Ia46cc5f2a651aefc4884d5d642a0c6c04d66a536
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2022-07-29 17:08:22 +00:00
Kelvin Zhang
87e452739e Fix typo in recently changed extra_apex_payload_key flag
The intention was to make extra_apex_payload_key take a comma separted
list of keys. Iterating over a string will only iterate over each
characters individually. Must split by "," to iterate over keys.

Test: Run signing test locally using tradefed.sh
Bug: 239991438
Change-Id: I51bfed53a43b1b0fc1e33e0dc3d418a727e6ff15
2022-07-27 11:15:58 -07:00
Treehugger Robot
488525a889 Merge "Allow extra_apex_payload_key to take multiple names at once" am: d024518e1e
Original change: https://android-review.googlesource.com/c/platform/build/+/2164515

Change-Id: I0d2545f758d7469cf3a68ccf505a6183306630b3
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2022-07-27 02:04:03 +00:00
Kelvin Zhang
085b6f3e84 Allow extra_apex_payload_key to take multiple names at once
When running tests, we might need to add many
extra_payload_keys. Currently we have to add --extra_apex_payload_key
for each key pair we pass, resulting in extremely long argument list.

Test: th
Bug: 239991438
Change-Id: I3e5f9d76c7f45822fb986b603dc089407c35b76b
2022-07-25 16:16:13 -07:00
Treehugger Robot
8493849334 Merge "make: fix memory leak" am: 66b010f2cf
Original change: https://android-review.googlesource.com/c/platform/build/+/2138495

Change-Id: Ica6a38075b8cc2320c9aa71a48c7c2b65af0e368
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2022-07-11 21:52:21 +00:00
Treehugger Robot
66b010f2cf Merge "make: fix memory leak" 2022-07-11 21:09:58 +00:00
Andrew Scull
040a786d18 Merge "Extract fsverity_manifest_generator logic" am: caf7014585
Original change: https://android-review.googlesource.com/c/platform/build/+/2144921

Change-Id: I01e0848590cdd3ab84e19bdb28fc331b1ea1a051
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2022-07-08 06:56:36 +00:00
Andrew Scull
caf7014585 Merge "Extract fsverity_manifest_generator logic" 2022-07-08 06:34:57 +00:00
Andrew Scull
0a6d2e54dc Extract fsverity_manifest_generator logic
Let the fsverity_manifest_generator logic be reused and just keep the
bits to generate the BuildManifest.apk. Since this can all be acheived
with a series of shell commands, remove the python script and just do it
directly in the Makefile.

Bug: 237384936
Test: TH
Change-Id: I168dc1cea0be72b5098f99e4183e080a687fb03e
2022-07-07 19:56:31 +00:00
Xin Li
05442a9213 Merge tm-dev-plus-aosp-without-vendor@8763363
Bug: 236760014
Merged-In: I1dffeb055a68250f574630d4ef5fedc51bad4a2b
Change-Id: I4e4b6a02fc140a1e38ce533031d7eb0a739396df
2022-06-28 21:23:05 +00:00
David Anderson
6531b32955 Merge "Make it easier to disable EROFS compression." 2022-06-28 18:28:01 +00:00
George Burgess IV
4645f98185 make: fix memory leak
`info->symlink_target` is a `std::string`. This `strdup` therefore leaks
memory. Just use `std::string::operator=(const char *)` instead.

Bug: 206470603
Test: TreeHugger
Change-Id: I71bcaec583c7d429b4d09522cc5eeb2ced009007
2022-06-28 11:23:21 -07:00
Melisa Carranza Zúñiga
051c9a15af Merge "Add sign_sepolicy_path for a binary to sign sepolicy.apex." 2022-06-28 12:26:50 +00:00
David Anderson
f3c81d7e10 Make it easier to disable EROFS compression.
Currently the only way to do this is to enable deduplication.

Bug: N/A
Test: BOARD_EROFS_COMPRESSOR := none
Change-Id: I7f8b99fbcefe7bbc15dfab7062b321aa3e2cf1ca
2022-06-27 23:20:34 +00:00
Melisa Carranza Zuniga
e0a977affd Add sign_sepolicy_path for a binary to sign sepolicy.apex.
Bug: b/217570541
Test: sign_apex --container_key=testdata/testkey
--payload_key=testdata/testkey_RSA4096.key
--codename_to_api_level_map=S:31,T:32,UpsideDownCake:33
-e SEPolicy-33.zip=testdata/testkey com.android.sepolicy.apex
signed_sepolicy.apex

Change-Id: Ifdd1c6e7db07cbc4674450ab0e2226494c98605a
2022-06-27 11:03:26 +02:00
Kelvin Zhang
91410f104d Merge "Fix potential issues if str has spaces" 2022-06-25 00:49:48 +00:00
Sasha Smundak
2bfc570d8f Copy symlink farm creation tool from Bazel.
SymlinkTree is one of the action types Blaze returns in response to
`aquery` (action query). build-runfiles is a tool copied from Bazel
to implement this action (see src/main/tools/build-runfiles.cc in
the Bazel Git repo). It creates a symlink farm (i.e., a directory
hiearchy consisting of symlinks) from a given manifest. Mixed builds
need this tool the mixed builds (in particular, every Python applicaion
has a runtime symlink farm).

Bug: 232085015
Test: m USE_BAZEL_ANALYSIS=1 com.android.adbd
Change-Id: I9cfcb33cb7d0f63bd36ffd2b4101f53cfc6a42fc
2022-06-24 09:57:45 -07:00
jiajia tang
e5ddfcd004 Fix potential issues if str has spaces
Uniform the split() function
str.split() will return a list split all spaces in str,
while str.split(' ') will return a list might contain ''
which might have potential issues.

Signed-off-by: jiajia tang <tangjiajia@xiaomi.com>
Change-Id: I0961659b140f800bdbe285f63bb4f02b8459ff8b
Signed-off-by: jiajia tang <tangjiajia@xiaomi.com>
2022-06-23 23:56:45 +00:00
Dan Willemsen
6b2bd9d652 Merge "Fix potential error for sys.platform" am: 59f36c1950 am: fa7bb8304a
Original change: https://android-review.googlesource.com/c/platform/build/+/2127932

Change-Id: I9ab6f15da248c7be9b73df84da0a67787b291167
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2022-06-15 21:12:26 +00:00
jiajia tang
7e0c12b1ba Fix potential error for sys.platform
This CL is to fix gap between python2.x and 3.x.
For python2.x: sys.platform returns 'linux2'
For python3.x: sys.platform returns 'linux'

python3.x(https://docs.python.org/3.6/library/sys.html#sys.platform)
python2.x(https://docs.python.org/2.7/library/sys.html#sys.platform)
Both 2.x and 3.x recommend to use 'startswith' to identify sys.platform.

Change-Id: Id5a0a007518b329d1320baf96ecf1164cd3ebec1
Signed-off-by: jiajia tang <tangjiajia@xiaomi.com>
2022-06-15 15:06:38 +00:00
Treehugger Robot
ae68a90b54 Merge "Remove danielnorman@ from OWNERs." am: 6417dca03f am: 123a85e7fb
Original change: https://android-review.googlesource.com/c/platform/build/+/2118053

Change-Id: Ibcc6f866114a7c267aa89cd9b98c13dfcbe90f39
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2022-06-07 21:55:55 +00:00
Daniel Norman
731767c99d Remove danielnorman@ from OWNERs.
Bug: 235251309
Change-Id: I164b4fcd2abbe0bd5a0a73c29887d7804f87544a
2022-06-07 19:26:54 +00:00
Treehugger Robot
1ac559c009 Merge "Improve flags for compliance tools." am: 10714789a9 am: 4033d64768
Original change: https://android-review.googlesource.com/c/platform/build/+/2116674

Change-Id: I3c1bd5554060aad67d604fbbb15de3db53a1bad0
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2022-06-07 01:31:24 +00:00
Treehugger Robot
10714789a9 Merge "Improve flags for compliance tools." 2022-06-07 00:51:23 +00:00
Treehugger Robot
71880d9de6 Merge "[Bugfix] Check for presence before ZipWrite" am: bb8d9ed70f am: c3a915294b
Original change: https://android-review.googlesource.com/c/platform/build/+/2115654

Change-Id: Ifb49b4e8f30ea1da194e914c68cd01f63d8ccd4a
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2022-06-04 05:36:55 +00:00
Bob Badour
986a839161 Improve flags for compliance tools.
Test: m droid dist reportmissinglicenses

Change-Id: I4090dae3d5d33d1908d67dff31aeee92d2b261da
2022-06-03 12:44:22 -07:00
jiangxu5
b67b0d5bc0 [Bugfix] Check for presence before ZipWrite
There are vendor products but no *.map situation

Signed-off-by: jiangxu5 <jiangxu5@xiaomi.com>
Change-Id: I93a710fd10870b7406b1d7beb5a6f51b6badc104
2022-06-03 14:46:56 +08:00
Melisa Carranza Zúñiga
8e3198ace3 Revert "Merge "Adding flags and logic to sign updateable SEPolicy in APEX" am: 77c1dfa6d9 am: 5a0d81a0be am: 54e08307dc"
Test: mma and run sign_apex

This reverts commit 8fe9e3a2c7.

Reason for revert: Updateable SEPolicy is punted.

Change-Id: I28ae2afbb1083e215f4e0a27a9164133cd73dd76
Merged-In: I28ae2afbb1083e215f4e0a27a9164133cd73dd76
2022-06-01 12:01:03 +02:00
Melisa Carranza Zúñiga
da308bf773 Revert "Merge "Adding sepolicy sign params to sign_target_files_apks." am: fb042449e0 am: bd8c313275 am: a2db49becd"
Test: sign_target_files_apks target_files.zip target_out.zip

This reverts commit 6e6c22c70e.

Reason for revert: Updateable SEPolicy is punted.

Change-Id: I99524d13811028a4036aa85c74cf10974fe08165
Merged-In: I99524d13811028a4036aa85c74cf10974fe08165
2022-05-31 09:55:22 +00:00
Chih-hung Hsieh
2df67498e9 Merge "Add date time, checks on android_root, etc." am: c15b738a7c am: 13faf1ec21
Original change: https://android-review.googlesource.com/c/platform/build/+/2101076

Change-Id: Ie748e934ce2252b196bfa9d57de8e47f5c1cd705
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2022-05-24 23:57:02 +00:00
Chih-Hung Hsieh
dd58ab0dc1 Add date time, checks on android_root, etc.
* Dump current date/time to the output .html file
  to distinguish files from different build dates.
* Guess android_root from the path of warn_common.py,
  but only if it contains build/make and build/soong.
* Do not depend on the TOP variable,
  which is no longer emitted by build/soong make.
* Do not depend on the USE_RBE variables,
  which is not emitted in local makes.

Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Bug: 198657613
Change-Id: Ib75c60216f9e06f9a768e63b536495fb9658843f
2022-05-19 15:28:41 -07:00
Dmitrii Merkurev
03e324006b Merge "Introduce EROFS compress hints to the Android build system" am: 494358dec8 am: a13fbaa01f
Original change: https://android-review.googlesource.com/c/platform/build/+/2098052

Change-Id: I3ed091156c4b39dad6eaf578144ac3d070025988
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2022-05-19 11:41:19 +00:00
Dmitrii Merkurev
494358dec8 Merge "Introduce EROFS compress hints to the Android build system" 2022-05-19 11:00:11 +00:00
Chih-hung Hsieh
992d08602c Merge "Handle more mixed warning lines from RBE" am: b28e4c2235 am: c7638c668c
Original change: https://android-review.googlesource.com/c/platform/build/+/2094756

Change-Id: I42c5719052895460eb52e9e28e57c82d9ce02225
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2022-05-18 22:04:06 +00:00
Chih-hung Hsieh
b28e4c2235 Merge "Handle more mixed warning lines from RBE" 2022-05-18 21:31:22 +00:00
Chih-hung Hsieh
9fe03ee318 Merge "Add warning type into TopDirs and TopFiles index" am: 89703b6374 am: 3fc466adad
Original change: https://android-review.googlesource.com/c/platform/build/+/2100523

Change-Id: Idc5de621534beb043a942a0d1756894bfce3be66
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2022-05-18 21:30:40 +00:00
Dmitrii Merkurev
8ab6603a43 Introduce EROFS compress hints to the Android build system
Make it possible to provide a prebuilt hints file to specify different compression strategies on a file basis for all partitions using:

BOARD_EROFS_COMPRESS_HINTS := <path>/erofs-hints.txt

or separately for every partition using:

BOARD_SYSTEMIMAGE_EROFS_COMPRESS_HINTS := <path>/system-partition-erofs-hints.txt
BOARD_VENDORIMAGE_EROFS_COMPRESS_HINTS := <path>/vendor-partition-erofs-hints.txt

Hints format: Each line is defined by tokens separated by spaces in the following form:

<pcluster-in-bytes> <match-pattern>

pcluster-in-bytes can be specified as 0 which mean file will not be compressed

Bug: 231934752
Test: used dump.erofs -S system.img to verify the system image isn't compressed after specifying BOARD_SYSTEMIMAGE_EROFS_COMPRESS_HINTS := no-compress.txt, where no-compress.txt contains 0 .*
Test: used dump.erofs --nid=<inode-for-all-apk-files> vendor.img to verify apk files are not compressed (the rest of the partition content is compressed)
Test: used dump.erofs -S <all-images> to verify all images are not compressed after specifying BOARD_EROFS_COMPRESS_HINTS := <path>/no-compress.txt
Change-Id: I64054e26af5ea5781c45d2183b7796a9ad2b7927
2022-05-17 23:10:37 +00:00
Chih-Hung Hsieh
a55837c7d1 Handle more mixed warning lines from RBE
* Use android_root or __file__ path to find
  the source tree root and its sub directories.
* Use the sub directory list to clean up RBE-returned
  warning lines, removing all mixed prefix before
  the top sub directories.
* Change some re.match rules and order to save
  unnecessary comparisons.

Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Bug: 198657613
Change-Id: I78cc17d04b8ab9e12935ef04797f3272298d5267
2022-05-17 11:46:58 -07:00
Chih-Hung Hsieh
4b020cd1d3 Add warning type into TopDirs and TopFiles index
* The index was a directory or file path.
  This change adds additional entries that have
  warning type prefix in the index.
* In the "Directories/Files with at least 1% warnings"
  section, the list will include number of warnings for
  (1) "all warnings in a directory/file"
  (2) "warnings of a type in a directory/file"
  (3) "warnings of a type in all directories/files"
  Examples:
    4273 (8.7%) frameworks/av/...
    4130 (8.4%) packages/...
    2427 (5.0%) [google-explicit-constructor] */...
    1628 (3.3%) [cert-err34-c] */...
    1099 (2.2%) [google-runtime-int] bionic/tests/math_data/...
     608 (1.2%) [cert-err34-c] external/...

Bug: 231245501
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: I13bb54c846ad514334f78c5a71e994a131a92963
2022-05-17 11:46:15 -07:00
Michael Groover
96bfead5db Merge "Add support for -providerArg in signapk" am: 9d1a0a47d5 am: 9401efe64f
Original change: https://android-review.googlesource.com/c/platform/build/+/1760832

Change-Id: Ie876bfee50a8e3dccc59bd8b5ba68ff1bb113fdd
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2022-05-12 19:39:26 +00:00
Michael Groover
2f8aad4f7f Add support for -providerArg in signapk
signapk currently accepts a Provider class that can be instantiated
and inserted before the signing. This commit adds support to specify
a -providerArg parameter that can be used to configure the Provider.
Prior to JDK 9 a Provider would accept a providerArg in a constructor
accepting a String; in JDK 9+ a Provider should first be instantiated
with the zero-arg constructor, then the configure method should be
called with the providerArg.

Bug: 142334653
Bug: 190974913
Fixes: 232134730
Test: Manually verified new Provider can be inserted with pre- and
      post-JDK 9 behavior.
Change-Id: I96f027640c59d3357e8dcf656626d1601bfef861
2022-05-11 21:55:24 +00:00
Kelvin Zhang
af7731528b Temporary hack for making T->S downgrade work am: 25d69c8bf5
Original change: https://googleplex-android-review.googlesource.com/c/platform/build/+/18246946

Change-Id: I1f8044750c7754965cba7ed0bea654fb30b0befc
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2022-05-10 22:53:37 +00:00
Kelvin Zhang
25d69c8bf5 Temporary hack for making T->S downgrade work
Downgrade OTA was broken because generic_ramdisk was removed from boot
partition in android T. Neither does O6 have init_boot. O6 also doesn't
include generic ramdisk in vendor boot, so no way for otatools to locate
contents of generic ramdisk. As a hack, omit boot partition timestamp if
we can't find ramdisk.

Test: generate T->S downgrade OTA for O6
Bug: 231656318
Merged-In: I9f5359462332aadadc324348873a3a4b2b126068
Change-Id: I9f5359462332aadadc324348873a3a4b2b126068
2022-05-10 05:06:47 +00:00
Kelvin Zhang
9a698e6a9f Temporary hack for making T->S downgrade work
Downgrade OTA was broken because generic_ramdisk was removed from boot
partition in android T. Neither does O6 have init_boot. O6 also doesn't
include generic ramdisk in vendor boot, so no way for otatools to locate
contents of generic ramdisk. As a hack, omit boot partition timestamp if
we can't find ramdisk.

Test: generate T->S downgrade OTA for O6
Bug: 231656318
Change-Id: I9f5359462332aadadc324348873a3a4b2b126068
2022-05-09 10:20:52 -07:00
Kelvin Zhang
f4406ca129 Merge identical key/val pairs in dynamic partition info
We might add new values to dynamic_partition_info.txt, so some kind of
generic fallback mechanism is needed. If keys are different, we need to
decided on a case-by-case basis which side takes precedence. For
example, Virtual AB requires vendor support, so vendor side takes
precedence. VABC on T+ devices are implemented entirely in system, so
system/framework side takes precedence.

Bug: 230876542
Test: th
Change-Id: I67747368547d3ef3e29ad64f8f818ef4c5896246
(cherry picked from commit 6a683ce02b)
Merged-In: I67747368547d3ef3e29ad64f8f818ef4c5896246
2022-05-06 21:40:14 +00:00
Treehugger Robot
1ca14f8415 Merge "Disable verity for VABC updates on android R" am: 46c4bb37d1 am: e654cbdfcd
Original change: https://android-review.googlesource.com/c/platform/build/+/2087483

Change-Id: I1e1c6583021ea106503a993fc844ce04f31a1317
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2022-05-06 00:37:11 +00:00