Commit Graph

8167 Commits

Author SHA1 Message Date
Hongguang Chen
0d6b727e03 Support chained init_boot partition signing
Bug: 256048561
Bug: 256237041
Test: sign_target_files_apks -d certs --avb_init_boot_algorithm \
        SHA256_RSA4096 --avb_init_boot_key init_boot_rsa4096.pem \
	xxx.zip signed.zip (Check signed.zip/META/misc_info.txt)
Change-Id: I65fc7913089ae318d90df55a533d3e2ebd93b029
2022-11-07 15:48:23 -08:00
Dennis Shen
f58e548943 add EROFS apex support.
Signing code uses deapexer to expand payload image, deapexer currently
only supports ext4 payload image expansion using debugfs or
debugfs_static. To support EROFS, need to supply deapexer with
fsck.erofs native binary for EROFS payload expansion.

cherrypick of ag/20154950BUG: b/195515562, b/240288941
Change-Id: Ibf707989f2502bfcf112202a3a6fdb74b6a8dbbc
2022-11-07 20:40:41 +00:00
Treehugger Robot
e32ad5e482 Merge "Convert soong_to_convert.py to python 3" 2022-11-05 03:34:03 +00:00
Cole Faust
5865420eb1 Convert soong_to_convert.py to python 3
Bug: 203436762
Test: m out/target/product/emulator_x86_64/soong_to_convert.txt
Change-Id: Ibf8d80a160315ed03a11fbeef66a630b224bea0a
2022-11-04 17:36:13 -07:00
Bob Badour
8eeff2b0b8 Merge "Fix final nits from noticeindex change." 2022-11-04 16:57:01 +00:00
Dennis Shen
44b714b7ca Update deapexer extract call with more input
BUG: b/255963179, b/240288941
Change-Id: Id7f5eb20e4c6f3c3e793860389f4be48f4f3938f
2022-11-03 17:39:04 +00:00
Ibrahim Kanouche
a68ed086ab Optimized project selection for getProjectMetadata
Test: m compliance_sbom

Change-Id: I9e60b7e138a4fca74170954d8bb83862677e750f
2022-11-03 16:01:13 +00:00
Bob Badour
3afc3004f2 Fix final nits from noticeindex change.
Test: m droid dist cts alllicensemetadata
Change-Id: Ie446e5693a09f4ae1c936cdae6f1e668105541f0
2022-11-02 10:24:04 -07:00
Bob Badour
88b02afbf0 Merge "Switch noticeindex from regex to proto." 2022-11-02 15:46:38 +00:00
Maciej Żenczykowski
458f90272c sbom_test - use UTC
Test: TreeHugger, m compliance_sbom
Signed-off-by: Maciej Żenczykowski <maze@google.com>
Change-Id: Id2abec8537a1dfa32d6816da4c34c86f46313086
2022-11-01 19:49:05 -07:00
Ibrahim Kanouche
1bb27903e8 Merge "Added SBOM generator module to implement the spdx utility bill of material" 2022-11-01 23:36:22 +00:00
Ibrahim Kanouche
bedf1a8a86 Added SBOM generator module to implement the spdx
utility bill of material

Added methods to graph.go to extract the annotation types

Test: m compliance_sbom

Test: tested against real targets, see go/paste/4932131502424064

Bug: 241591242
Change-Id: I344c15d64cd034d72076f9d9a677e593e288e3a6
2022-11-01 20:42:01 +00:00
Jihoon Kang
eb35b48df7 Merge "Create custom diff tool to compare stub contents" 2022-11-01 20:32:20 +00:00
Sam Delmerico
9fac7000ec Merge "move logtags rule from //build/make to //build/bazel" 2022-11-01 13:50:10 +00:00
Treehugger Robot
5667fe0a74 Merge "Allow RO build only for readonly mount points" 2022-10-31 22:58:32 +00:00
Bob Badour
5c12c66769 Replace nil-able *sync.Waitgroup with sync.Once
Simplifies synchronization and eliminates lock for nil waitroup.

Test: m droid

Test: m out/soong/.intermediates/packages/modules/StatsD/apex/com.android.os.statsd/android_common_com.android.os.statsd_image/NOTICE.html.gz

Change-Id: I381ee79e142214e7331241071f076db2f7960ba6
2022-10-29 22:48:13 -07:00
Bob Badour
d2c28ba897 Fix top-down resolve re-walking graph too much.
Firing off multiple concurrent walks of the same sub-tree with the same
conditions. Data race meant every walk would proceed. Move the logic to
entry of walk function and compare under lock.

Bug: 255526010

Test: m droid

Test: m out/soong/.intermediates/packages/modules/StatsD/apex/com.android.os.statsd/android_common_com.android.os.statsd_image/NOTICE.html.gz

Change-Id: Ie30edbb2ac9eaa9aa55badfc518d51eaadbb6be6
2022-10-29 22:27:49 -07:00
Jihoon Kang
3d38b6d9c8 Create custom diff tool to compare stub contents
Context:
- Create a tool to analyze loose equivalence of the stubs in two
  directories
- The tool can analyze strict equivalence of the directories of the
  stubs in two directories
- Analyze text to compare loose equivalence of the stub contents; Add a
  functionality to pass `skip_words` as arguments, which are optional
  list of words used to signal the tool which words are not considered
  diff
- The tool can be locally used to compare stub contents, and does not
  contribute to build process

Test: m
Change-Id: I74563a9a24ecdde939be2ce37b9096a9aeb4920a
2022-10-28 22:45:08 +00:00
Bob Badour
43daade15f Add some documentation and simplify data model.
Bug: 245562496

Test: m droid
Change-Id: Iae757a5767522e0734abbe3840ea0939620197e7
2022-10-28 12:06:08 -07:00
Bob Badour
ab5cfbd5b7 Switch noticeindex from regex to proto.
Bug: 254534552

Test: m droid dist cts alllicensemetadata

Test: repo forall -c 'echo -n "$REPO_PATH  " && $ANDROID_BUILD_TOP/out/host/linux-x86/bin/compliance_checkmetadata . 2>&1' | fgrep -v PASS

Change-Id: I34dfe143cdc88432d6e33abee096409a37ce3b7e
2022-10-27 18:13:24 -07:00
Bob Badour
d6574e5c97 Add AllMetadataFiles() method to track deps.
Improve METADATA parsing error message.

Example error message (for an unqoted string url starting with ssh:):

============
  error in project "<redacted>" METADATA "<redacted>/METADATA": proto: (line 4:12): invalid value for string type: ssh

METADATA and METADATA.android files must parse as text protobufs
defined by
   build/soong/compliance/project_metadata_proto/project_metadata.proto

* unknown fields don't matter
* check invalid ENUM names
* check quoting
* check unescaped nested quotes
* check the comment marker for protobuf is '#' not '//'

if importing a library that uses a different sort of METADATA file, add
a METADATA.android file beside it to parse instead
============

Bug: 254534552

Test: m droid dist cts alllicensemetadata

Change-Id: Ie5c0adc7362941e455dd522baaa31f2913d7db5f
2022-10-27 18:12:30 -07:00
Treehugger Robot
a2faaa8c45 Merge "Remove all ZIP64LIMIT hack" 2022-10-27 22:44:22 +00:00
Treehugger Robot
1fca7c5e7f Merge "Enable zip64 support in build image script" 2022-10-27 22:31:00 +00:00
Jooyung Han
e3862eacb6 Merge "Fix boot fail due to missing /apex in images" 2022-10-27 21:54:13 +00:00
Spandan Das
03b846ff37 Merge "Migrate check_elf_file.py to python3" 2022-10-27 16:46:18 +00:00
Jaegeuk Kim
551a2e66c3 Allow RO build only for readonly mount points
Should support RW for all other partitions.

Bug: 255454067
Change-Id: I9d87cf39195f8da0f6fe4153c703ba6a372aa2ea
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
2022-10-27 09:46:03 -07:00
Spandan Das
60b8195c83 Migrate check_elf_file.py to python3
Also, create a python_binary_host so that it runs using the hermetic
python toolchain.

Test: m check-elf-files
Bug: 203436762
Change-Id: I964342a27f6b6c9dcdbbe910d4dc6c9ec00c2213
2022-10-27 02:08:23 +00:00
Kelvin Zhang
37a4290909 Remove all ZIP64LIMIT hack
In the old days, we hacked values of ZIP64LIMIT to get around size
limitations of non-zip64 supported zip files. Now that we switched to
python3 + zip64, there's no point in keeping those hacks.

Test: th
Bug: 255683436
Change-Id: I913db33dad5503736c68a7a1f1321aa952019f60
2022-10-26 13:22:52 -07:00
Kelvin Zhang
1de1788dda Enable zip64 support in build image script
Images are getting bigger each year, we need zip64 support to store >4GB
files in .zip archive.

Test: th
Bug: 255683436
Change-Id: I9adbefa5ec72e357b02e46d6e629c054d466ddd7
2022-10-26 12:30:29 -07:00
Po Hu
4d2f64b6f0 Fix boot fail due to missing /apex in images
Do not move or rename any files in merge_target_files.py's temporary directory. They will be used to generate images further.

Bug: 254760332
Change-Id: I973a49b29234d49f5254a1403a0e236e69425d74
Test: check SYSTEM/apex in target-files.zip generated by merget_target_files.py
2022-10-26 02:34:12 +00:00
Treehugger Robot
1a6c0c5d86 Merge "Delete compare_fileslist.py" 2022-10-25 21:22:19 +00:00
Treehugger Robot
fe7277fdde Merge "The --why parameter in whichgit should be optional" 2022-10-25 20:08:59 +00:00
Cole Faust
29c032d260 Delete compare_fileslist.py
It was written in python 2, and is unused.

Bug: 203436762
Test: Presubmits
Change-Id: I3298ec06727ba1b72337753f52e3d09519fe0b39
2022-10-25 11:34:42 -07:00
Ibrahim Kanouche
bfa6c8e553 Merge "Added functions to projectmetadata to retrieve additional project info" 2022-10-25 17:25:11 +00:00
Cole Faust
3b9af315d2 Merge "Delete java-layers.py" 2022-10-25 17:00:52 +00:00
Cole Faust
c22b90c5d4 Merge "Delete parsedeps.py" 2022-10-25 17:00:11 +00:00
Treehugger Robot
f99edd9f67 Merge "Convert fileslist_util.py to python 3" 2022-10-25 15:58:19 +00:00
Treehugger Robot
aa3b322971 Merge "Convert findleaves.py to python 3" 2022-10-25 09:17:27 +00:00
Cole Faust
e0fab08a33 Convert fileslist_util.py to python 3
Also update it from getopt to argparse.

Bug: 203436762
Test: m $OUT/installed-files-root.txt
Change-Id: I006a36feab01cbbd76dc7d7c1c0675197848212a
2022-10-24 18:46:03 -07:00
Cole Faust
6a75c4f782 Delete java-layers.py
There are no usages of LOCAL_JAVA_LAYERS_FILE,
and java-layers.py was written in python 2.

Bug: 203436762
Test: Presubmits
Change-Id: I1be0f2547757a853e20811f28529479487037e94
2022-10-24 18:15:54 -07:00
Cole Faust
e995769d44 Delete parsedeps.py
This script is only ever manually run, and it's written
in python 2. Just delete it instead of updating to python
3 because it's unlikely anyone uses it anymore.

Bug: 203436762
Test: Presubmits
Change-Id: Ib55b3bf43a8066822308f49d1a948d27c520c5b2
2022-10-24 18:02:23 -07:00
Cole Faust
407d214b0d Convert findleaves.py to python 3
Kati special-cases findleaves.py to an internal
C++ implementation, so I don't think this python
script is even used. Update it anyways, and
test by manually running findleaves.py.

Bug: 203436762
Test: ./build/make/tools/findleaves.py --mindepth=2 --dir=build/make/target Android.mk
Change-Id: If27cc37d8f673388c0e31b9491e792b98ca59da4
2022-10-24 17:57:39 -07:00
Spandan Das
ce69e9c812 Convert check_radio_versions to python3
Additionaly,
1. Run it as a python_binary_host so that it uses the hermetic python
toolchain
2. Fix an existing bug where if `board_info_txt` was empty, $^ would
   evaluate to its own path, and it would try to parse the relevant
   configuration from the wrong file

Bug: 196300985
Test: m out/target/product/vsoc_arm64/android-info.txt
Change-Id: Ie1e51a93041591829da4351dd57f37106e15d764
2022-10-24 21:02:00 +00:00
Joe Onorato
8e908b221f The --why parameter in whichgit should be optional
Test: ./build/make/tools/whichgit --modules out/target/product/coral/system.img
Change-Id: Ibc6e208d75c8e68c8130dc9ba3fc7af51390691b
2022-10-24 08:03:20 -07:00
Ibrahim Kanouche
776ad80e68 Added functions to projectmetadata to retrieve
additional project info

Test: m droid

Bug: 254901942
Change-Id: I3de4bc528bd321c76900d277295bb10709035a9c
2022-10-22 01:22:53 +00:00
Treehugger Robot
e9d0a7f5d9 Merge "Convert fs_config_generator.py to python 3" 2022-10-21 21:17:23 +00:00
Treehugger Robot
775187f921 Merge "Fix ota build error when using boot variable files" 2022-10-21 20:43:21 +00:00
Cole Faust
216978e7c4 Convert fs_config_generator.py to python 3
Bug: 203436762
Test: ./test_fs_config_generator.py and m libc
Change-Id: I16ce5f5e273ade064acce8889a9eb3f2707910ae
2022-10-21 11:16:18 -07:00
Kelvin Zhang
eb147e051e Fix ota build error when using boot variable files
When performing a deep copy of info dict, input_file is serialized as
filename. Make ExtractFromInputFile support reading from filename of a
ZipFile.

Test: build OTA with partner provided target_files
Bug: 253549364
Change-Id: I366a076c32d638d61b83a1df9bf864807cf5257c
2022-10-21 10:53:21 -07:00
Treehugger Robot
e7bf5958b9 Merge "Fix broken fs_config_generator tests" 2022-10-21 01:33:41 +00:00