Commit Graph

8076 Commits

Author SHA1 Message Date
Alex Buynytskyy
5d205dc016 Change CI API level to keep linter happy.
Bug: 275409981
Test: run locally
Change-Id: I99d2ef4974a70f435f528c7a9358990fa41c9f5e
2023-04-27 15:30:19 -07:00
Alex Buynytskyy
d873b0e46b Update Android Studio Version Number.
Bug: 275409981
Test: run locally
Change-Id: I0d62ac8178f6d69f66adde8c5889ff86bcfad0ed
2023-04-27 18:18:57 +00:00
Treehugger Robot
99dc1341d1 Merge "Let the finalization script create ABI dumps for all architectures" 2023-04-27 05:52:25 +00:00
Alex Buynytskyy
d878cdf692 Merge "Mark master as VIC development branch." 2023-04-26 23:39:11 +00:00
Daniel Zheng
15c296958b Merge "Add --max_threads to OTA generation" 2023-04-26 21:53:53 +00:00
Alex Buynytskyy
0fc39526a1 Mark master as VIC development branch.
Test: ./build-step-1-and-m.sh
Bug: 279615150
Change-Id: Idb485846345ea146c04c5ecf6eb4ab6eea122f9b
Merged-In: Idb485846345ea146c04c5ecf6eb4ab6eea122f9b
2023-04-26 18:15:59 +00:00
Daniel Zheng
a30989bafa Add --max_threads to OTA generation
Adding an option to configure max number of threads used during
ota_from_target_files

Bug: 277794445
Test: tested ota_from_target_files
Change-Id: I9bae92ee43c21926434ab04563a2b4f0678b73eb
2023-04-26 10:52:12 -07:00
Hsin-Yi Chen
139cc54d81 Let the finalization script create ABI dumps for all architectures
The command that creates ABI dumps was written for testing purpose.
It creates the ABI dumps for aosp_arm64-user. The complete finalization
process should create ABI dumps for all supported architectures. This
commit removes the command parameters so that it will create all ABI
dumps by default.

Test: out/host/linux-x86/bin/create_reference_dumps
Bug: 276503801
Change-Id: Id6fbbac512a13d085ea74fec6e8b137485639467
2023-04-26 17:11:20 +08:00
Treehugger Robot
dd62b0da0f Merge "Support to merge PREBUILT_IMAGES" 2023-04-25 04:24:20 +00:00
Treehugger Robot
5ca02ff114 Merge "Fix incremental OTA generation failures" 2023-04-22 01:26:28 +00:00
Alex Buynytskyy
f5897ecdb6 Don't skip vndk finalization.
Bug: 275409981
Test: local run
Change-Id: I6651f29e3c5fd5f1ccb6ec07fd52491ba16f5086
2023-04-21 19:08:50 +00:00
Kelvin Zhang
ca45d7a85d Fix incremental OTA generation failures
source_file might be a directory, use the _ReadApexInfo() helper
function to read it instead of go straight to zipfiles.

Test: build incremental OTA for pixel
Bug: 279142680
Bug: 227848550
Bug: 277028723

Change-Id: Ia0041e2c72c1a57123519c970f6e734194a81cd2
2023-04-21 17:30:50 +00:00
Treehugger Robot
fad3925959 Merge "Update generateSPDXNamespace to generate a unique spdx doc namespace" 2023-04-21 16:50:00 +00:00
Treehugger Robot
269248add6 Merge "Cleanup the list of Soong prebuilt module types." 2023-04-21 01:40:01 +00:00
Wei Li
6f407ba0a8 Cleanup the list of Soong prebuilt module types.
Bug: 272356622
Test: CIs
Change-Id: I4031f0a73f5af1bbf1eb374bcd7d3a19b9609a46
2023-04-20 14:13:58 -07:00
Ibrahim Kanouche
397a9cf430 Update generateSPDXNamespace to generate a unique spdx doc namespace
Test: m compliance_sbom
Change-Id: Icc1e5aec0e1758ee32c43cfa79b5b49b3d1d9d30
2023-04-20 17:43:12 +00:00
Treehugger Robot
ece71703f8 Merge "Allow ota_from_target_file to work entirely on directories" 2023-04-20 15:17:50 +00:00
Bob Badour
ffba21bbae [LSC] Add LOCAL_LICENSE_KINDS to build/make
Added SPDX-license-identifier-Apache-2.0 to:
  tools/sbom/Android.bp

Bug: 68860345
Bug: 151177513
Bug: 151953481

Test: m all
Change-Id: I1f8e5fd8e787a4c53bf4caf80a4000d46d0291e4
2023-04-19 09:48:01 -07:00
Kelvin Zhang
9dbe2ce40f Allow ota_from_target_file to work entirely on directories
When building android, build system will first zip target_file directory
into a .zip file, and then invoke ota_from_target_files. We
ota_from_target_files can work directly on top of directories, we can
parallelize OTA generation and target_file zipping, making builds
faster.

Bug: 227848550
Bug: 277028723
Test: th
Change-Id: Id72bd7cca708af0e1bae2db754f39b27777cd601
2023-04-18 17:47:29 -07:00
Po Hu
81c3f4a884 Support to merge PREBUILT_IMAGES
For example, pvmfw.img comes from framework target files package
and dtbo.img comes from vendor target files package.

Test: merge_target_files
Change-Id: If2e6fac5ce3671e3387e801808a195fb7b2c55b5
2023-04-19 08:23:40 +08:00
Wei Li
49af939a23 Copy SBOM files of unbundled APKs to dist directory.
Bug: 266726655
Test: CIs
Test: tapas Browser2 Calendar arm userdebug && m dist
Change-Id: I82328dc4916451a99ad0bf6d7bb945c9b98d0bea
2023-04-18 15:57:34 -07:00
Treehugger Robot
67ca3c05c7 Merge "Fix duplicate super_empty.img" 2023-04-17 21:19:13 +00:00
Treehugger Robot
b7a45ea13d Merge "Try with search_path for some avb path args" 2023-04-17 20:43:12 +00:00
Kelvin Zhang
aecdc18222 Fix duplicate super_empty.img
When calling add_img_to_target_files twice, super_empty.img will be
written twice, and resulting zip would contain duplicate entries.

Test: th
Bug: 278522974
Change-Id: I2419d0c4f8b04f876a8cc69a1517f59cab94d75d
2023-04-17 09:52:39 -07:00
Alex Buynytskyy
2ba40b5be6 Merge "Finalization script fixes." 2023-04-17 15:26:16 +00:00
Alex Buynytskyy
f7a5e2fa4b Finalization script fixes.
- actually do the first step instead of just logging a message,
- make sure extension SDK bump gets picked up by the topic,
- fix the resources_sdk_int in first step, and unfix in the second.

Bug: 278246368
Fixes: 278246368
Test: local run
Change-Id: I0d0f4e2ec7f0b584b9fc14a545cfec73a9fcbdea
2023-04-16 21:15:18 -07:00
Treehugger Robot
3fb8d2bad8 Merge "Fix the following issues mentioned in Pixel SBOM review." 2023-04-15 20:34:52 +00:00
Wei Li
5290825452 Fix the following issues mentioned in Pixel SBOM review.
1) PackageSupplier should be NOASSERTION if there is no homepage
   information in METADATA file of source packages
2) PackageDownloadLocation of upstream packages should be NOASSERTION if
   there is no code repository URL in METADATA file of source packages

Test: CIs
Test: atest --host sbom_writers_test
Change-Id: I8a0298b7bacc2f96555f9d7dde0d21ada8c6b564
2023-04-15 06:05:32 +00:00
David Anderson
23c2df71bc Merge "ota_from_target_files: Disable XOR on <T vendor." 2023-04-15 04:52:46 +00:00
Treehugger Robot
7b640806da Merge "Revert "Remove all ZIP64LIMIT hack"" 2023-04-15 01:48:58 +00:00
Dennis Song
5da012adb0 Merge "Include META/* in the vendor item list when rebuild-sepolicy" 2023-04-15 01:15:44 +00:00
David Anderson
1c59617820 ota_from_target_files: Disable XOR on <T vendor.
Bug: N/A
Test: ota_from_target_files
Change-Id: Ia9093a3c3f79d88f98f0777e8a0f8c72741055b8
2023-04-14 16:01:55 -07:00
Kelvin Zhang
f92f7f046a Revert "Remove all ZIP64LIMIT hack"
This reverts commit 37a4290909.

Reason for revert: b/278156419

Change-Id: I67ea667619a9623be849d911993010ef0f0bfd88
2023-04-14 21:32:54 +00:00
Treehugger Robot
ebf41e9a91 Merge "Create separate python libraries for the following logic and refactor SBOM generation script accordingly." 2023-04-14 20:43:58 +00:00
Treehugger Robot
6fb4c8dcee Merge "Order entries in apex_info.pb" 2023-04-14 20:33:18 +00:00
Håkan Kvist
01e381954b Order entries in apex_info.pb
Adding the entries in apex_info.pb in sorted order makes it easier to
compare ota packages generated between the same source and target on
different computers/environments, ideally the generated ota zips should
be identical.

os.listdir() was used to find the apex files which should be
included in apex_info.pb. listdir() does not guarantee any order,
solution is to sort the result to ensure consistent order.

Bug: 278095305
Test: Manual. Confirm that apex_info.pb is generated sorted
    protoc --decode_raw < apex_info.pb
Change-Id: I2851d207f828a6ac31e63e20c9025965ac4b0eba
2023-04-14 10:36:04 +02:00
Dennis Song
09f5c5dc0b Include META/* in the vendor item list when rebuild-sepolicy
To ensure that `AddImagesToTargetFiles` can still be used with
vendor item list that do not specify all of the required META/
files, those files should be included by default.

Test: atest --host releasetools_test
Bug: 278133178
Change-Id: I5a04bfc43d405d6838028f2535ced2124dbd43aa
2023-04-14 12:13:42 +08:00
zhangyongpeng
707569711b Try with search_path for some avb path args
If we can't find relative signing_helper path, try with the OPTIONS.search_path dir prefix, and integrate the avb_*_key_path for the same logic.

Test: Build && releasetools_test

Signed-off-by: zhangyongpeng <zhangyongpeng@xiaomi.com>
Change-Id: Ifb1096ddea90693668f3344eb242bf9725113d11
2023-04-14 12:12:47 +08:00
Wei Li
dec97b1462 Create separate python libraries for the following logic and refactor SBOM generation script accordingly.
1) writer classes of generating SBOM in different SPDX formats
2) data classes to model the SBOM structure in SPDX

Bug: 272358880
Test: CIs
Test: build/soong/tests/sbom_test.sh
Test: atest --host sbom_writers_test

Change-Id: I1175cf0d99864bc4304559a59484ef0ba401cd64
2023-04-13 13:34:57 -07:00
Ibrahim Kanouche
8c745a436b Added validation to SBOM generator
Test: m compliance_sbom
Change-Id: I1ff2dfbc48361cfb785c64306112bc687ca51057
2023-04-13 16:10:12 +00:00
Treehugger Robot
64bfc41359 Merge "Fix s to t OTA" 2023-04-13 01:01:57 +00:00
Kelvin Zhang
010bbea401 Merge "Fix releasetools timeouts" 2023-04-12 15:37:59 +00:00
Kelvin Zhang
629bc8dc1c Fix s to t OTA
Android S update_engine has a smaller minor version, we should use
update_engine version number from source build when generating an
incremental OTA.

Test: th
Bug: 277837157
Change-Id: Ic1ccdccd26c19bf9fb13d146a9151e699e481201
2023-04-11 21:30:42 -07:00
Treehugger Robot
84031f7b49 Merge "Invoke delta_generator directly" 2023-04-12 01:00:43 +00:00
Treehugger Robot
b2be62551c Merge "Script updates." 2023-04-11 21:52:43 +00:00
Kelvin Zhang
ea84d42199 Fix releasetools timeouts
Release tools test uses sleep(5) to make sure mtime on files change in a
visible way. Use hardcoded mtime instead of sleeps.

Improvement: 145.6s -> 65s

Test: atest releasetools_test
Bug: 277782284
Fixes: 277782284

Change-Id: I4d7b04707f3b6c5843cde883f43e95a1e7a69879
2023-04-11 13:53:05 -07:00
Kelvin Zhang
fcd731e3d6 Invoke delta_generator directly
Currently, ota_from_target_files(a python script) calls
brillo_update_payload(a bash script), which then calls delta_generator(a
C++ binary) to do the actual diffing. Having brillo_update_payload in
the middle does not offer any additional flexibility, but it makes
maintaince more difficult. Bash code is less readable and harder to
debug. Further more, everytime we want to add a new flag to
delta_generator, we have to add the flag in three places:
ota_from_target_files, brillo_update_payload, delta_generator.

Historically, brillo_update_payload was there because we inherited from
ChromeOS side. This bash scripts extracts target_files.zip and invoke
delta_generator to generate the actual OTA. Any customizations we want
on OTA must be implemented by modifying the input target_files.zip ,
Manipuating big zip files is slow and inefficient.

To make thing simpler/faster, remove the intermdiary call to
brill_update_payload. ota_from_target_files will now extract target
files and call delta_generator directly.

Test: th
Bug: 227848550
Change-Id: I44b296e43bf7921bcf48ef6a1e4021a12669739c
2023-04-11 13:53:05 -07:00
Alex Buynytskyy
d98c43c8dd Script updates.
- add a mode to disable finalization steps if they were already merged,
- add the extension version into commit message.

Bug: 243966946
Test: local run
Change-Id: I51505e5d5715b56f3b385d714e1aabaae06f58ed
2023-04-11 16:51:03 +00:00
Dennis Song
a0ffed1fa1 Merge "Support merging target files from directory" 2023-04-11 05:40:05 +00:00
Treehugger Robot
52fdcf5457 Merge "Allow setting future api level before REL" 2023-04-11 00:09:23 +00:00