Commit Graph

7803 Commits

Author SHA1 Message Date
Kelvin Zhang
059bf6e50d Move some OTA signing functions to a separte file
This makes it easier for other otatools to re-use these logic without
having to pull in lots of dependencies.

Test: th
Bug: 227848550

Change-Id: I81ed01c5cea4b934a074650731b6f89752221de9
2022-08-22 18:20:14 +00:00
Yong Ma
253b106f95 set up the salt for partitions without build.prop
BUG:242954151

Test: make target-files-package, then check the two sha1 are same:
      1. sha1 of IMAGES/boot.img
      2. expected sha1 of boot in VENDOR/bin/install-recovery.sh

Test: make target-files-package; unzip target_files.zip IMAGES/\*`;
      `zip -d arget_files.zip IMAGES/\*`
      `add_img_to_target_files -a target_files.zip`.
      Verify that the rebuilt images are identical to the deleted ones.

Change-Id: I5906ac1696b6403674fd8330b1569aed53510945
2022-08-19 02:10:01 +00:00
Treehugger Robot
81994f0b27 Merge "Fix a bug in unsigned OTA generation" am: 2abfe5f3a3
Original change: https://android-review.googlesource.com/c/platform/build/+/2185088

Change-Id: Id6180edf4f57ccabe4ffb072a7eebaa4d6f07ccf
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2022-08-16 02:55:33 +00:00
Kelvin Zhang
b9fdf2dc08 Fix a bug in unsigned OTA generation
Caller expects output_file to contain the final OTA, so we must copy
data into that location. Modifying loca variables isn't enough.

Test: make ota with --no_signing
Change-Id: Id4aaefbc53b4250b7308701493d151f9408f47f1
2022-08-12 14:07:31 -07:00
Treehugger Robot
2e30eed994 Merge "Add flags to override all apex/apk keys" am: c7cd55bf17
Original change: https://android-review.googlesource.com/c/platform/build/+/2171705

Change-Id: I6c3556e51fa839c078e1eea6f6daa2dd8d6e4024
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2022-08-06 04:53:55 +00:00
Treehugger Robot
c7cd55bf17 Merge "Add flags to override all apex/apk keys" 2022-08-06 04:32:53 +00:00
Alexander Smundak
a77df0c808 Merge "Fix go.mod for tools/canoninja" am: 5cf8217978
Original change: https://android-review.googlesource.com/c/platform/build/+/2171782

Change-Id: I3cad683736ec403fbcb1fc0899ee6898480e372e
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2022-08-02 23:02:55 +00:00
Kelvin Zhang
e50bb51fa9 Add flags to override all apex/apk keys
Currently, if we want to override all apex/apk key, we must specify all apk/apex names exhaustively. This causes CLI arguments to be very long and sometimes exceeds OS's limit.

Test: run signing test locally on build 8862875 on git_master
Bug: 239991438
Change-Id: Idbd3b7eef9b2a27236f4ca7a27449fcdf0d2dd3f
2022-08-02 18:32:38 +00:00
Treehugger Robot
0b96e1a005 Merge "Fix go.mod for tools/compliance" am: f30267635e
Original change: https://android-review.googlesource.com/c/platform/build/+/2169821

Change-Id: Ibf4c760bc46a31512b2bd9ed5e6576b24fcd0479
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2022-08-02 03:44:13 +00:00
Sasha Smundak
94d473c5c7 Fix go.mod for tools/canoninja
Now it can be ld with `go build` and cross-referenced.

Test: treehugger
Change-Id: I299efc2d635176519345acf142114097be6bf814
2022-08-01 14:45:28 -07:00
Sasha Smundak
ded9acd86d Fix go.mod for tools/compliance
Now it can be built with `go build`, and can be cross-referenced.

Test: treehugger
Change-Id: Ic35631ff6a915c8cf58db14aa83c849549e64aba
2022-08-01 14:24:21 -07:00
Kelvin Zhang
89c59b8fa5 Merge "Fix typo in recently changed extra_apex_payload_key flag" am: 193de78dfd
Original change: https://android-review.googlesource.com/c/platform/build/+/2167622

Change-Id: Ia46cc5f2a651aefc4884d5d642a0c6c04d66a536
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2022-07-29 17:08:22 +00:00
Kelvin Zhang
87e452739e Fix typo in recently changed extra_apex_payload_key flag
The intention was to make extra_apex_payload_key take a comma separted
list of keys. Iterating over a string will only iterate over each
characters individually. Must split by "," to iterate over keys.

Test: Run signing test locally using tradefed.sh
Bug: 239991438
Change-Id: I51bfed53a43b1b0fc1e33e0dc3d418a727e6ff15
2022-07-27 11:15:58 -07:00
Treehugger Robot
488525a889 Merge "Allow extra_apex_payload_key to take multiple names at once" am: d024518e1e
Original change: https://android-review.googlesource.com/c/platform/build/+/2164515

Change-Id: I0d2545f758d7469cf3a68ccf505a6183306630b3
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2022-07-27 02:04:03 +00:00
Kelvin Zhang
085b6f3e84 Allow extra_apex_payload_key to take multiple names at once
When running tests, we might need to add many
extra_payload_keys. Currently we have to add --extra_apex_payload_key
for each key pair we pass, resulting in extremely long argument list.

Test: th
Bug: 239991438
Change-Id: I3e5f9d76c7f45822fb986b603dc089407c35b76b
2022-07-25 16:16:13 -07:00
Treehugger Robot
8493849334 Merge "make: fix memory leak" am: 66b010f2cf
Original change: https://android-review.googlesource.com/c/platform/build/+/2138495

Change-Id: Ica6a38075b8cc2320c9aa71a48c7c2b65af0e368
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2022-07-11 21:52:21 +00:00
Treehugger Robot
66b010f2cf Merge "make: fix memory leak" 2022-07-11 21:09:58 +00:00
Andrew Scull
040a786d18 Merge "Extract fsverity_manifest_generator logic" am: caf7014585
Original change: https://android-review.googlesource.com/c/platform/build/+/2144921

Change-Id: I01e0848590cdd3ab84e19bdb28fc331b1ea1a051
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2022-07-08 06:56:36 +00:00
Andrew Scull
caf7014585 Merge "Extract fsverity_manifest_generator logic" 2022-07-08 06:34:57 +00:00
Andrew Scull
0a6d2e54dc Extract fsverity_manifest_generator logic
Let the fsverity_manifest_generator logic be reused and just keep the
bits to generate the BuildManifest.apk. Since this can all be acheived
with a series of shell commands, remove the python script and just do it
directly in the Makefile.

Bug: 237384936
Test: TH
Change-Id: I168dc1cea0be72b5098f99e4183e080a687fb03e
2022-07-07 19:56:31 +00:00
Xin Li
05442a9213 Merge tm-dev-plus-aosp-without-vendor@8763363
Bug: 236760014
Merged-In: I1dffeb055a68250f574630d4ef5fedc51bad4a2b
Change-Id: I4e4b6a02fc140a1e38ce533031d7eb0a739396df
2022-06-28 21:23:05 +00:00
David Anderson
6531b32955 Merge "Make it easier to disable EROFS compression." 2022-06-28 18:28:01 +00:00
George Burgess IV
4645f98185 make: fix memory leak
`info->symlink_target` is a `std::string`. This `strdup` therefore leaks
memory. Just use `std::string::operator=(const char *)` instead.

Bug: 206470603
Test: TreeHugger
Change-Id: I71bcaec583c7d429b4d09522cc5eeb2ced009007
2022-06-28 11:23:21 -07:00
Melisa Carranza Zúñiga
051c9a15af Merge "Add sign_sepolicy_path for a binary to sign sepolicy.apex." 2022-06-28 12:26:50 +00:00
David Anderson
f3c81d7e10 Make it easier to disable EROFS compression.
Currently the only way to do this is to enable deduplication.

Bug: N/A
Test: BOARD_EROFS_COMPRESSOR := none
Change-Id: I7f8b99fbcefe7bbc15dfab7062b321aa3e2cf1ca
2022-06-27 23:20:34 +00:00
Melisa Carranza Zuniga
e0a977affd Add sign_sepolicy_path for a binary to sign sepolicy.apex.
Bug: b/217570541
Test: sign_apex --container_key=testdata/testkey
--payload_key=testdata/testkey_RSA4096.key
--codename_to_api_level_map=S:31,T:32,UpsideDownCake:33
-e SEPolicy-33.zip=testdata/testkey com.android.sepolicy.apex
signed_sepolicy.apex

Change-Id: Ifdd1c6e7db07cbc4674450ab0e2226494c98605a
2022-06-27 11:03:26 +02:00
Kelvin Zhang
91410f104d Merge "Fix potential issues if str has spaces" 2022-06-25 00:49:48 +00:00
Sasha Smundak
2bfc570d8f Copy symlink farm creation tool from Bazel.
SymlinkTree is one of the action types Blaze returns in response to
`aquery` (action query). build-runfiles is a tool copied from Bazel
to implement this action (see src/main/tools/build-runfiles.cc in
the Bazel Git repo). It creates a symlink farm (i.e., a directory
hiearchy consisting of symlinks) from a given manifest. Mixed builds
need this tool the mixed builds (in particular, every Python applicaion
has a runtime symlink farm).

Bug: 232085015
Test: m USE_BAZEL_ANALYSIS=1 com.android.adbd
Change-Id: I9cfcb33cb7d0f63bd36ffd2b4101f53cfc6a42fc
2022-06-24 09:57:45 -07:00
jiajia tang
e5ddfcd004 Fix potential issues if str has spaces
Uniform the split() function
str.split() will return a list split all spaces in str,
while str.split(' ') will return a list might contain ''
which might have potential issues.

Signed-off-by: jiajia tang <tangjiajia@xiaomi.com>
Change-Id: I0961659b140f800bdbe285f63bb4f02b8459ff8b
Signed-off-by: jiajia tang <tangjiajia@xiaomi.com>
2022-06-23 23:56:45 +00:00
Dan Willemsen
6b2bd9d652 Merge "Fix potential error for sys.platform" am: 59f36c1950 am: fa7bb8304a
Original change: https://android-review.googlesource.com/c/platform/build/+/2127932

Change-Id: I9ab6f15da248c7be9b73df84da0a67787b291167
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2022-06-15 21:12:26 +00:00
jiajia tang
7e0c12b1ba Fix potential error for sys.platform
This CL is to fix gap between python2.x and 3.x.
For python2.x: sys.platform returns 'linux2'
For python3.x: sys.platform returns 'linux'

python3.x(https://docs.python.org/3.6/library/sys.html#sys.platform)
python2.x(https://docs.python.org/2.7/library/sys.html#sys.platform)
Both 2.x and 3.x recommend to use 'startswith' to identify sys.platform.

Change-Id: Id5a0a007518b329d1320baf96ecf1164cd3ebec1
Signed-off-by: jiajia tang <tangjiajia@xiaomi.com>
2022-06-15 15:06:38 +00:00
Treehugger Robot
ae68a90b54 Merge "Remove danielnorman@ from OWNERs." am: 6417dca03f am: 123a85e7fb
Original change: https://android-review.googlesource.com/c/platform/build/+/2118053

Change-Id: Ibcc6f866114a7c267aa89cd9b98c13dfcbe90f39
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2022-06-07 21:55:55 +00:00
Daniel Norman
731767c99d Remove danielnorman@ from OWNERs.
Bug: 235251309
Change-Id: I164b4fcd2abbe0bd5a0a73c29887d7804f87544a
2022-06-07 19:26:54 +00:00
Treehugger Robot
1ac559c009 Merge "Improve flags for compliance tools." am: 10714789a9 am: 4033d64768
Original change: https://android-review.googlesource.com/c/platform/build/+/2116674

Change-Id: I3c1bd5554060aad67d604fbbb15de3db53a1bad0
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2022-06-07 01:31:24 +00:00
Treehugger Robot
10714789a9 Merge "Improve flags for compliance tools." 2022-06-07 00:51:23 +00:00
Treehugger Robot
71880d9de6 Merge "[Bugfix] Check for presence before ZipWrite" am: bb8d9ed70f am: c3a915294b
Original change: https://android-review.googlesource.com/c/platform/build/+/2115654

Change-Id: Ifb49b4e8f30ea1da194e914c68cd01f63d8ccd4a
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2022-06-04 05:36:55 +00:00
Bob Badour
986a839161 Improve flags for compliance tools.
Test: m droid dist reportmissinglicenses

Change-Id: I4090dae3d5d33d1908d67dff31aeee92d2b261da
2022-06-03 12:44:22 -07:00
jiangxu5
b67b0d5bc0 [Bugfix] Check for presence before ZipWrite
There are vendor products but no *.map situation

Signed-off-by: jiangxu5 <jiangxu5@xiaomi.com>
Change-Id: I93a710fd10870b7406b1d7beb5a6f51b6badc104
2022-06-03 14:46:56 +08:00
Melisa Carranza Zúñiga
8e3198ace3 Revert "Merge "Adding flags and logic to sign updateable SEPolicy in APEX" am: 77c1dfa6d9 am: 5a0d81a0be am: 54e08307dc"
Test: mma and run sign_apex

This reverts commit 8fe9e3a2c7.

Reason for revert: Updateable SEPolicy is punted.

Change-Id: I28ae2afbb1083e215f4e0a27a9164133cd73dd76
Merged-In: I28ae2afbb1083e215f4e0a27a9164133cd73dd76
2022-06-01 12:01:03 +02:00
Melisa Carranza Zúñiga
da308bf773 Revert "Merge "Adding sepolicy sign params to sign_target_files_apks." am: fb042449e0 am: bd8c313275 am: a2db49becd"
Test: sign_target_files_apks target_files.zip target_out.zip

This reverts commit 6e6c22c70e.

Reason for revert: Updateable SEPolicy is punted.

Change-Id: I99524d13811028a4036aa85c74cf10974fe08165
Merged-In: I99524d13811028a4036aa85c74cf10974fe08165
2022-05-31 09:55:22 +00:00
Chih-hung Hsieh
2df67498e9 Merge "Add date time, checks on android_root, etc." am: c15b738a7c am: 13faf1ec21
Original change: https://android-review.googlesource.com/c/platform/build/+/2101076

Change-Id: Ie748e934ce2252b196bfa9d57de8e47f5c1cd705
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2022-05-24 23:57:02 +00:00
Chih-Hung Hsieh
dd58ab0dc1 Add date time, checks on android_root, etc.
* Dump current date/time to the output .html file
  to distinguish files from different build dates.
* Guess android_root from the path of warn_common.py,
  but only if it contains build/make and build/soong.
* Do not depend on the TOP variable,
  which is no longer emitted by build/soong make.
* Do not depend on the USE_RBE variables,
  which is not emitted in local makes.

Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Bug: 198657613
Change-Id: Ib75c60216f9e06f9a768e63b536495fb9658843f
2022-05-19 15:28:41 -07:00
Dmitrii Merkurev
03e324006b Merge "Introduce EROFS compress hints to the Android build system" am: 494358dec8 am: a13fbaa01f
Original change: https://android-review.googlesource.com/c/platform/build/+/2098052

Change-Id: I3ed091156c4b39dad6eaf578144ac3d070025988
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2022-05-19 11:41:19 +00:00
Dmitrii Merkurev
494358dec8 Merge "Introduce EROFS compress hints to the Android build system" 2022-05-19 11:00:11 +00:00
Chih-hung Hsieh
992d08602c Merge "Handle more mixed warning lines from RBE" am: b28e4c2235 am: c7638c668c
Original change: https://android-review.googlesource.com/c/platform/build/+/2094756

Change-Id: I42c5719052895460eb52e9e28e57c82d9ce02225
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2022-05-18 22:04:06 +00:00
Chih-hung Hsieh
b28e4c2235 Merge "Handle more mixed warning lines from RBE" 2022-05-18 21:31:22 +00:00
Chih-hung Hsieh
9fe03ee318 Merge "Add warning type into TopDirs and TopFiles index" am: 89703b6374 am: 3fc466adad
Original change: https://android-review.googlesource.com/c/platform/build/+/2100523

Change-Id: Idc5de621534beb043a942a0d1756894bfce3be66
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2022-05-18 21:30:40 +00:00
Dmitrii Merkurev
8ab6603a43 Introduce EROFS compress hints to the Android build system
Make it possible to provide a prebuilt hints file to specify different compression strategies on a file basis for all partitions using:

BOARD_EROFS_COMPRESS_HINTS := <path>/erofs-hints.txt

or separately for every partition using:

BOARD_SYSTEMIMAGE_EROFS_COMPRESS_HINTS := <path>/system-partition-erofs-hints.txt
BOARD_VENDORIMAGE_EROFS_COMPRESS_HINTS := <path>/vendor-partition-erofs-hints.txt

Hints format: Each line is defined by tokens separated by spaces in the following form:

<pcluster-in-bytes> <match-pattern>

pcluster-in-bytes can be specified as 0 which mean file will not be compressed

Bug: 231934752
Test: used dump.erofs -S system.img to verify the system image isn't compressed after specifying BOARD_SYSTEMIMAGE_EROFS_COMPRESS_HINTS := no-compress.txt, where no-compress.txt contains 0 .*
Test: used dump.erofs --nid=<inode-for-all-apk-files> vendor.img to verify apk files are not compressed (the rest of the partition content is compressed)
Test: used dump.erofs -S <all-images> to verify all images are not compressed after specifying BOARD_EROFS_COMPRESS_HINTS := <path>/no-compress.txt
Change-Id: I64054e26af5ea5781c45d2183b7796a9ad2b7927
2022-05-17 23:10:37 +00:00
Chih-Hung Hsieh
a55837c7d1 Handle more mixed warning lines from RBE
* Use android_root or __file__ path to find
  the source tree root and its sub directories.
* Use the sub directory list to clean up RBE-returned
  warning lines, removing all mixed prefix before
  the top sub directories.
* Change some re.match rules and order to save
  unnecessary comparisons.

Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Bug: 198657613
Change-Id: I78cc17d04b8ab9e12935ef04797f3272298d5267
2022-05-17 11:46:58 -07:00
Chih-Hung Hsieh
4b020cd1d3 Add warning type into TopDirs and TopFiles index
* The index was a directory or file path.
  This change adds additional entries that have
  warning type prefix in the index.
* In the "Directories/Files with at least 1% warnings"
  section, the list will include number of warnings for
  (1) "all warnings in a directory/file"
  (2) "warnings of a type in a directory/file"
  (3) "warnings of a type in all directories/files"
  Examples:
    4273 (8.7%) frameworks/av/...
    4130 (8.4%) packages/...
    2427 (5.0%) [google-explicit-constructor] */...
    1628 (3.3%) [cert-err34-c] */...
    1099 (2.2%) [google-runtime-int] bionic/tests/math_data/...
     608 (1.2%) [cert-err34-c] external/...

Bug: 231245501
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: I13bb54c846ad514334f78c5a71e994a131a92963
2022-05-17 11:46:15 -07:00