Commit Graph

3015 Commits

Author SHA1 Message Date
Daniel Zheng
6b558b3fe4 Merge "Add fastboot-info to updatepackage" am: 717591dd4e
Original change: https://android-review.googlesource.com/c/platform/build/+/2583539

Change-Id: I92a70b845d04e2f251eed2b96fbcfe58d57ffeea
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-12 05:19:25 +00:00
Treehugger Robot
96b51b54b6 Merge "parallelize add_partition_calls" 2023-05-12 04:54:11 +00:00
Daniel Zheng
717591dd4e Merge "Add fastboot-info to updatepackage" 2023-05-12 04:31:55 +00:00
Jeongik Cha
03d014a03f parallelize add_partition_calls
it made total time equal to the longest image build

1m10s->30s in local build

Bug: 281960217
Test: m dist
Change-Id: I13d4f45d9b46b39292a014e3b4e1913365d89b7a
2023-05-12 08:43:44 +09:00
Kelvin Zhang
06417aaa9e Merge "Fix non-AB ota generation for directories" am: ea70695d92
Original change: https://android-review.googlesource.com/c/platform/build/+/2585207

Change-Id: I7a6472946360238393a734640e27557f7623446f
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-10 04:46:20 +00:00
Kelvin Zhang
b036007592 Fix non-AB ota generation for directories
Test: ota_from_target_files -v --block yukawa-target_files-10097950.zip
test.zip

Change-Id: I4291010d66cc28bdb6b6b334cdbed06daf3de837
2023-05-09 20:30:53 -07:00
Daniel Zheng
380cb8065a Add fastboot-info to updatepackage
Modifying img_from_target_files to also add fastboot-info to
updatepackage

Test: m updatepackage
Bug: 194686221
Change-Id: I2e08c4269f0d83417b9d7079633bc28796d1cdd6
2023-05-09 13:44:22 -07:00
Treehugger Robot
5e4f59f9ca Merge "Support generating partial OTAs from extracted target_files" am: c3216a321b
Original change: https://android-review.googlesource.com/c/platform/build/+/2577071

Change-Id: I79fa834772dc12fe26678238bcf80ecffd2a34ce
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-04 00:54:58 +00:00
Kelvin Zhang
6b10e15acc Support generating partial OTAs from extracted target_files
This allows the build system to potentially paralleize generation of OTA
package and zipping of target files

Bug: 262185376
Bug: 227848550
Change-Id: I90b6c25761683ebe3803b22fc8e23540a5282c66
2023-05-02 20:48:14 -07:00
Treehugger Robot
8583eae383 Merge "Use misc_info.txt to check for VABC" am: ed1ab26f5d
Original change: https://android-review.googlesource.com/c/platform/build/+/2572975

Change-Id: I2ef6cf286ed259dafc20f8bb1f4b0388a32b5fda
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-01 22:41:40 +00:00
Kelvin Zhang
e634bde6bd Use misc_info.txt to check for VABC
Change-Id: I77f23a51ddb027254ccda8d7ab17e64205522300
2023-04-28 23:59:43 -07:00
Po Hu
09de31418f Fix merge_target_files in GRF case
aosp/2525591 remove SYSTEM/build.prop from vendor target_files,
and cause the same error reported in aosp/1836120.

Bug: 192422274
Change-Id: I3af33e44e395002a9cf26cf93677ff84c2c26207
2023-04-27 10:41:25 +08:00
Daniel Zheng
fe2a0816b6 Merge "Add --max_threads to OTA generation" am: 15c296958b
Original change: https://android-review.googlesource.com/c/platform/build/+/2563130

Change-Id: Ibd5d90c7661ecf3367485cf43b887c2cefcde544
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-04-26 22:27:31 +00:00
Daniel Zheng
15c296958b Merge "Add --max_threads to OTA generation" 2023-04-26 21:53:53 +00:00
Daniel Zheng
a30989bafa Add --max_threads to OTA generation
Adding an option to configure max number of threads used during
ota_from_target_files

Bug: 277794445
Test: tested ota_from_target_files
Change-Id: I9bae92ee43c21926434ab04563a2b4f0678b73eb
2023-04-26 10:52:12 -07:00
Treehugger Robot
5939a80763 Merge "Support to merge PREBUILT_IMAGES" am: dd62b0da0f
Original change: https://android-review.googlesource.com/c/platform/build/+/2512522

Change-Id: I67339cb1a8108577522bd3446e868adbe3385439
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-04-25 05:17:11 +00:00
Treehugger Robot
dd62b0da0f Merge "Support to merge PREBUILT_IMAGES" 2023-04-25 04:24:20 +00:00
Treehugger Robot
5970bf338b Merge "Fix incremental OTA generation failures" am: 5ca02ff114
Original change: https://android-review.googlesource.com/c/platform/build/+/2553374

Change-Id: I17880a243562b821a634715b242677de52ba6d12
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-04-22 02:04:12 +00:00
Kelvin Zhang
ca45d7a85d Fix incremental OTA generation failures
source_file might be a directory, use the _ReadApexInfo() helper
function to read it instead of go straight to zipfiles.

Test: build incremental OTA for pixel
Bug: 279142680
Bug: 227848550
Bug: 277028723

Change-Id: Ia0041e2c72c1a57123519c970f6e734194a81cd2
2023-04-21 17:30:50 +00:00
Xin Li
d6b6d0102a Merge Android 13 QPR3 tm-qpr-dev-plus-aosp-without-vendor@9936994
Bug: 275386652
Merged-In: I4b2ea3956c34aa56cc6f54bab7148f116c34759e
Change-Id: I900022a8e8f104b6dc52745211c428601be53ba6
2023-04-21 09:49:08 -07:00
Kelvin Zhang
9dbe2ce40f Allow ota_from_target_file to work entirely on directories
When building android, build system will first zip target_file directory
into a .zip file, and then invoke ota_from_target_files. We
ota_from_target_files can work directly on top of directories, we can
parallelize OTA generation and target_file zipping, making builds
faster.

Bug: 227848550
Bug: 277028723
Test: th
Change-Id: Id72bd7cca708af0e1bae2db754f39b27777cd601
2023-04-18 17:47:29 -07:00
Po Hu
81c3f4a884 Support to merge PREBUILT_IMAGES
For example, pvmfw.img comes from framework target files package
and dtbo.img comes from vendor target files package.

Test: merge_target_files
Change-Id: If2e6fac5ce3671e3387e801808a195fb7b2c55b5
2023-04-19 08:23:40 +08:00
Treehugger Robot
67ca3c05c7 Merge "Fix duplicate super_empty.img" 2023-04-17 21:19:13 +00:00
Treehugger Robot
b7a45ea13d Merge "Try with search_path for some avb path args" 2023-04-17 20:43:12 +00:00
Kelvin Zhang
aecdc18222 Fix duplicate super_empty.img
When calling add_img_to_target_files twice, super_empty.img will be
written twice, and resulting zip would contain duplicate entries.

Test: th
Bug: 278522974
Change-Id: I2419d0c4f8b04f876a8cc69a1517f59cab94d75d
2023-04-17 09:52:39 -07:00
David Anderson
23c2df71bc Merge "ota_from_target_files: Disable XOR on <T vendor." 2023-04-15 04:52:46 +00:00
Treehugger Robot
7b640806da Merge "Revert "Remove all ZIP64LIMIT hack"" 2023-04-15 01:48:58 +00:00
Dennis Song
5da012adb0 Merge "Include META/* in the vendor item list when rebuild-sepolicy" 2023-04-15 01:15:44 +00:00
David Anderson
1c59617820 ota_from_target_files: Disable XOR on <T vendor.
Bug: N/A
Test: ota_from_target_files
Change-Id: Ia9093a3c3f79d88f98f0777e8a0f8c72741055b8
2023-04-14 16:01:55 -07:00
Kelvin Zhang
f92f7f046a Revert "Remove all ZIP64LIMIT hack"
This reverts commit 37a4290909.

Reason for revert: b/278156419

Change-Id: I67ea667619a9623be849d911993010ef0f0bfd88
2023-04-14 21:32:54 +00:00
Håkan Kvist
01e381954b Order entries in apex_info.pb
Adding the entries in apex_info.pb in sorted order makes it easier to
compare ota packages generated between the same source and target on
different computers/environments, ideally the generated ota zips should
be identical.

os.listdir() was used to find the apex files which should be
included in apex_info.pb. listdir() does not guarantee any order,
solution is to sort the result to ensure consistent order.

Bug: 278095305
Test: Manual. Confirm that apex_info.pb is generated sorted
    protoc --decode_raw < apex_info.pb
Change-Id: I2851d207f828a6ac31e63e20c9025965ac4b0eba
2023-04-14 10:36:04 +02:00
Dennis Song
09f5c5dc0b Include META/* in the vendor item list when rebuild-sepolicy
To ensure that `AddImagesToTargetFiles` can still be used with
vendor item list that do not specify all of the required META/
files, those files should be included by default.

Test: atest --host releasetools_test
Bug: 278133178
Change-Id: I5a04bfc43d405d6838028f2535ced2124dbd43aa
2023-04-14 12:13:42 +08:00
zhangyongpeng
707569711b Try with search_path for some avb path args
If we can't find relative signing_helper path, try with the OPTIONS.search_path dir prefix, and integrate the avb_*_key_path for the same logic.

Test: Build && releasetools_test

Signed-off-by: zhangyongpeng <zhangyongpeng@xiaomi.com>
Change-Id: Ifb1096ddea90693668f3344eb242bf9725113d11
2023-04-14 12:12:47 +08:00
Treehugger Robot
858e08e0ca Merge "Fix s to t OTA" am: 64bfc41359 am: c4e1d12853
Original change: https://android-review.googlesource.com/c/platform/build/+/2531966

Change-Id: Ia20bdef8a788e1a4fbb8facf047c2bf211bd8495
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-04-13 01:46:13 +00:00
Kelvin Zhang
6c5f1c32a3 Merge "Fix releasetools timeouts" am: 010bbea401 am: d434bd4c02
Original change: https://android-review.googlesource.com/c/platform/build/+/2530977

Change-Id: I6c8c2360e17434139d2f0beffe5c0af807e9595d
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-04-12 16:39:03 +00:00
Kelvin Zhang
629bc8dc1c Fix s to t OTA
Android S update_engine has a smaller minor version, we should use
update_engine version number from source build when generating an
incremental OTA.

Test: th
Bug: 277837157
Change-Id: Ic1ccdccd26c19bf9fb13d146a9151e699e481201
2023-04-11 21:30:42 -07:00
Treehugger Robot
766eaa6d78 Merge "Invoke delta_generator directly" am: 84031f7b49 am: 651e2a94c5
Original change: https://android-review.googlesource.com/c/platform/build/+/2529019

Change-Id: I7471c375cf5dc05680e71f64435325fb96159f2f
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-04-12 02:49:54 +00:00
Kelvin Zhang
ea84d42199 Fix releasetools timeouts
Release tools test uses sleep(5) to make sure mtime on files change in a
visible way. Use hardcoded mtime instead of sleeps.

Improvement: 145.6s -> 65s

Test: atest releasetools_test
Bug: 277782284
Fixes: 277782284

Change-Id: I4d7b04707f3b6c5843cde883f43e95a1e7a69879
2023-04-11 13:53:05 -07:00
Kelvin Zhang
fcd731e3d6 Invoke delta_generator directly
Currently, ota_from_target_files(a python script) calls
brillo_update_payload(a bash script), which then calls delta_generator(a
C++ binary) to do the actual diffing. Having brillo_update_payload in
the middle does not offer any additional flexibility, but it makes
maintaince more difficult. Bash code is less readable and harder to
debug. Further more, everytime we want to add a new flag to
delta_generator, we have to add the flag in three places:
ota_from_target_files, brillo_update_payload, delta_generator.

Historically, brillo_update_payload was there because we inherited from
ChromeOS side. This bash scripts extracts target_files.zip and invoke
delta_generator to generate the actual OTA. Any customizations we want
on OTA must be implemented by modifying the input target_files.zip ,
Manipuating big zip files is slow and inefficient.

To make thing simpler/faster, remove the intermdiary call to
brill_update_payload. ota_from_target_files will now extract target
files and call delta_generator directly.

Test: th
Bug: 227848550
Change-Id: I44b296e43bf7921bcf48ef6a1e4021a12669739c
2023-04-11 13:53:05 -07:00
Dennis Song
7ffe8ab317 Merge "Support merging target files from directory" am: a0ffed1fa1 am: 609dd3b00f
Original change: https://android-review.googlesource.com/c/platform/build/+/2525591

Change-Id: I4fe623e0f07875ca112b593e83c00c048ceb436d
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-04-11 06:51:03 +00:00
Dennis Song
5bfa43e5eb Support merging target files from directory
Expand `merge_target_files.py` API capabilities so that
`--framework-target-files` and `--vendor-target-files`
can be either zip archives or directories.

Test: Create a merged package by vendor target files folder
Test: atest --host releasetools_test
Bug: 276068400
Change-Id: I200be2a458ae59a61e05bfd7c78ab66093db32eb
2023-04-10 17:07:03 +08:00
Treehugger Robot
579e5d2dcd Merge "Add comments explaining how secondary OTA works" am: d8e23f9519 am: a1c42f5b17
Original change: https://android-review.googlesource.com/c/platform/build/+/2527438

Change-Id: I56d60c850c851f47905d5c23a5543b3e67b29008
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-04-07 22:37:28 +00:00
Treehugger Robot
d8e23f9519 Merge "Add comments explaining how secondary OTA works" 2023-04-07 20:57:36 +00:00
Kelvin Zhang
bd111ccae2 Add comments explaining how secondary OTA works
Test: th
Change-Id: I72f890e3dcb6f8369dea8071da040b4c27877a2b
2023-04-07 09:47:04 -07:00
Kelvin Zhang
3dc5fb7392 Merge "Fix choosing wrong diff tool for recovery-from-boot patch" am: 3aa4644117 am: c8e58a10a0
Original change: https://android-review.googlesource.com/c/platform/build/+/2501818

Change-Id: Ic11be84be27ee80860202f387961de1e518f6ff4
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-04-05 17:46:57 +00:00
Oleg Lyovin
6d75a854ef Fix choosing wrong diff tool for recovery-from-boot patch
'bsdiff' tool should be used when recovery and boot image
have mismatching entries. This includes not only the case
when system_as_root enabled, but include_recovery_dtbo and
include_recovery_acpio too.

Add check if they are enabled to synchronize logic with
core/Makefile, which already takes this into account.

Test: make dist -j
Change-Id: Ie673403ae8eae446d006cc524dd48ac6c56194fc
Signed-off-by: Oleg Lyovin <ovlevin@sberdevices.ru>
2023-03-22 17:53:26 +03:00
Treehugger Robot
b5b28fae99 Merge "Add an automated tools for generating brick OTA package" am: 4fac9d776c am: ac4fb5a4d8
Original change: https://android-review.googlesource.com/c/platform/build/+/2488336

Change-Id: Iee4349d628ea8680135918bb4cbe71d6b047e199
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-03-15 10:42:24 +00:00
Treehugger Robot
4fac9d776c Merge "Add an automated tools for generating brick OTA package" 2023-03-15 09:24:40 +00:00
Kelvin Zhang
f91d74bd52 Add an automated tools for generating brick OTA package
In the past, we have been generating brick OTA packages manually.
Automate this process.

Bug: 273561331
Test: create_brick_ota --product oriole brick.zip and apply the package
Change-Id: I7f4cf3cacaedd9d376b4cc07cbb388930bca643b
2023-03-14 16:58:36 -07:00
Rob Seymour
9e52f880b9 Merge "Add test cases for APEX VINTF." am: 578e1bb073 am: 58d63254dc
Original change: https://android-review.googlesource.com/c/platform/build/+/2242995

Change-Id: Ibb3764a362c8b3ea38a9c5d6a8857615e4832166
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-03-10 01:57:02 +00:00