Commit Graph

8034 Commits

Author SHA1 Message Date
Kelvin Zhang
010bbea401 Merge "Fix releasetools timeouts" 2023-04-12 15:37:59 +00:00
Treehugger Robot
84031f7b49 Merge "Invoke delta_generator directly" 2023-04-12 01:00:43 +00:00
Treehugger Robot
b2be62551c Merge "Script updates." 2023-04-11 21:52:43 +00:00
Kelvin Zhang
ea84d42199 Fix releasetools timeouts
Release tools test uses sleep(5) to make sure mtime on files change in a
visible way. Use hardcoded mtime instead of sleeps.

Improvement: 145.6s -> 65s

Test: atest releasetools_test
Bug: 277782284
Fixes: 277782284

Change-Id: I4d7b04707f3b6c5843cde883f43e95a1e7a69879
2023-04-11 13:53:05 -07:00
Kelvin Zhang
fcd731e3d6 Invoke delta_generator directly
Currently, ota_from_target_files(a python script) calls
brillo_update_payload(a bash script), which then calls delta_generator(a
C++ binary) to do the actual diffing. Having brillo_update_payload in
the middle does not offer any additional flexibility, but it makes
maintaince more difficult. Bash code is less readable and harder to
debug. Further more, everytime we want to add a new flag to
delta_generator, we have to add the flag in three places:
ota_from_target_files, brillo_update_payload, delta_generator.

Historically, brillo_update_payload was there because we inherited from
ChromeOS side. This bash scripts extracts target_files.zip and invoke
delta_generator to generate the actual OTA. Any customizations we want
on OTA must be implemented by modifying the input target_files.zip ,
Manipuating big zip files is slow and inefficient.

To make thing simpler/faster, remove the intermdiary call to
brill_update_payload. ota_from_target_files will now extract target
files and call delta_generator directly.

Test: th
Bug: 227848550
Change-Id: I44b296e43bf7921bcf48ef6a1e4021a12669739c
2023-04-11 13:53:05 -07:00
Alex Buynytskyy
d98c43c8dd Script updates.
- add a mode to disable finalization steps if they were already merged,
- add the extension version into commit message.

Bug: 243966946
Test: local run
Change-Id: I51505e5d5715b56f3b385d714e1aabaae06f58ed
2023-04-11 16:51:03 +00:00
Dennis Song
a0ffed1fa1 Merge "Support merging target files from directory" 2023-04-11 05:40:05 +00:00
Treehugger Robot
52fdcf5457 Merge "Allow setting future api level before REL" 2023-04-11 00:09:23 +00:00
Dennis Song
5bfa43e5eb Support merging target files from directory
Expand `merge_target_files.py` API capabilities so that
`--framework-target-files` and `--vendor-target-files`
can be either zip archives or directories.

Test: Create a merged package by vendor target files folder
Test: atest --host releasetools_test
Bug: 276068400
Change-Id: I200be2a458ae59a61e05bfd7c78ab66093db32eb
2023-04-10 17:07:03 +08:00
Justin Yun
870ea2e188 Allow setting future api level before REL
At the dev stage, devices may set ro.board.(first_)api_level to the
future API level

Bug: 276927022
Test: test_post_process_props.py
Change-Id: I85c29af74ed8daa780278f64b023480bb6659781
2023-04-10 15:37:19 +09:00
Wei Li
3bcd0bca6f Check if an installed file exists and report error if not.
So far there is only one case happened in b/277039235 that an installed file in that product is moved to another directory in LOCAL_POST_INSTALL_CMD. The team agreed to convert to Android.bp, but better to add a check and report the error.

Resend this to check symlinks using os.path.islink(), which doesn't follow the link. os.path.isfile() follows symlinks and returns false since the symlinks are for devices and could not be resolved on host file systems.

Bug: 277039235
Test: m sbom
Test: build/soong/tests/sbom_test.sh
Change-Id: Ia9f1cd24dc974a3e41487bc17c9c76a26d419395
2023-04-07 16:36:30 -07:00
Wei Li
0bb86d0515 Merge "Revert "Check if an installed file exists and report error if not."" 2023-04-07 21:40:09 +00:00
Wei Li
fd559f89b5 Revert "Check if an installed file exists and report error if not."
This reverts commit f049389e2e.

Reason for revert: the check includes more files that should not be excluded.

Change-Id: I7f7d260218826cc8fd033400b2cfaf40e4c280d0
2023-04-07 21:35:13 +00:00
Treehugger Robot
d8e23f9519 Merge "Add comments explaining how secondary OTA works" 2023-04-07 20:57:36 +00:00
Kelvin Zhang
bd111ccae2 Add comments explaining how secondary OTA works
Test: th
Change-Id: I72f890e3dcb6f8369dea8071da040b4c27877a2b
2023-04-07 09:47:04 -07:00
Wei Li
f049389e2e Check if an installed file exists and report error if not.
So far there is only one case happened in b/277039235 that an installed file in that product is moved to another directory in LOCAL_POST_INSTALL_CMD. The team agreed to convert to Android.bp, but better to add a check and report the error.

Bug: 277039235
Test: m sbom
Test: build/soong/tests/sbom_test.sh
Change-Id: I9a2302ced6dd605e60e54c49a8778426444dcc4f
2023-04-07 07:33:11 +00:00
Alex Buynytskyy
0fce469f2d Merge "Minor renaming and documentation." 2023-04-06 23:38:33 +00:00
Alex Buynytskyy
307d9bcfdf Minor renaming and documentation.
Bug: 276752264
Test: local run
Change-Id: I75814313a81ca041fa1461e48733bd40a020b777
2023-04-06 15:28:39 -07:00
Alex Buynytskyy
f5b6dd80ef Merge "Copy ABI dumps from current/64 to FINAL_PLATFORM_SDK_VERSION/64" 2023-04-06 19:52:00 +00:00
Jihoon Kang
0c76cab34e Merge "Specify list_files to be built with embedded launcher" 2023-04-06 16:54:02 +00:00
Hsin-Yi Chen
ebce0d9501 Copy ABI dumps from current/64 to FINAL_PLATFORM_SDK_VERSION/64
Because prebuilts/abi-dumps/ndk/current contains additional ABI dumps
that are not included in the finalization process, the script copies
only the subdirectory to the finalized ABI dump directory.

Test: ./finalize-sdk-rel.sh
Bug: 276503801
Change-Id: I76287cf110323baf40035a957564a1e8f39e8015
2023-04-06 14:01:07 +00:00
Jihoon Kang
8f6edbbf04 Specify list_files to be built with embedded launcher
In the current use case of list_files, the module is built with a
possibly incompatible host machine's python version. This change fixes
this by specifying module to be built with embedded launcher.

Test: m list_files
Bug: 277124374
Change-Id: Id894271042136c9db50e63852075a8c4ef36a71e
2023-04-06 05:25:20 +00:00
Ibrahim Kanouche
aeeea61aa0 Merge "Added Document Fields to SBOM generator" 2023-04-05 20:37:04 +00:00
Kelvin Zhang
3aa4644117 Merge "Fix choosing wrong diff tool for recovery-from-boot patch" 2023-04-05 16:40:36 +00:00
Ibrahim Kanouche
f89fc4aa9b Added Document Fields to SBOM generator
Added functions to generate unique spdx doc namespace and generate a
clean document name

Test: m compliance_sbom

Bug: 265472710
Change-Id: I86ea9ddf50d066e139b757e8a093f98b8df8c81f
2023-04-04 18:53:10 +00:00
Jihoon Kang
155095a314 Merge "Create a python script to list files given a directory as an input" 2023-04-03 23:42:19 +00:00
Ibrahim Kanouche
960a4700e0 Merge "Revert "Revert "Updated SBOM generator module to generate JSON spdx utility bill of""" 2023-04-03 19:20:29 +00:00
Jihoon Kang
108379310d Create a python script to list files given a directory as an input
The script will be used in generating a file that lists generated stub
files from metalava, which will be passed to soong_zip to extract
selected stub files when generating stubs from java_api_library modules
with incomplete api surface text files.

Test: build java_api_library modules in aosp/2487335
Change-Id: I36b830f14c714315f81e3fd8608d84c29f673f83
2023-04-03 17:52:41 +00:00
Ibrahim Kanouche
91f2f9d851 Revert "Revert "Updated SBOM generator module to generate JSON spdx utility bill of""
This reverts commit 928ee9d9f7.

Reason for revert: Fixed the initial cause of the revert. Added spdx-tools to the missing branches. See b/276427351

Change-Id: I7bd0b3f194b27dc9a255ccadeb2a9a12a3d59f66
2023-04-01 21:24:30 +00:00
Alex Buynytskyy
5dfdbe12bc Move adb keys creation to CI-only step.
It's easily automatable build step, but it needs to be done way before
REL to allow enough time for manual key propagation.
As such, moving it to local step.

Bug: 243966946
Test: manual
Change-Id: I979e8500c3c0b768af30dff93c9b73ab998a0151
2023-03-31 22:20:29 +00:00
Bob Badour
00670e0c03 Merge "Revert "Updated SBOM generator module to generate JSON spdx utility bill of"" 2023-03-31 15:56:17 +00:00
Bob Badour
928ee9d9f7 Revert "Updated SBOM generator module to generate JSON spdx utility bill of"
This reverts commit e97adc5db9.

Reason for revert: Roll back until the kernel manifests are updated with the spdx-tools library.

Change-Id: Ib2327862acd928ebcea0f328daecc162b46d2c78
2023-03-31 14:51:36 +00:00
Treehugger Robot
09df4c6562 Merge "Updated SBOM generator module to generate JSON spdx utility bill of material" 2023-03-31 07:05:53 +00:00
Ibrahim Kanouche
e97adc5db9 Updated SBOM generator module to generate JSON spdx utility bill of
material

Test: m compliance_sbom

Bug: 265472710
Change-Id: Iad9ddbd2abf17ff0b034f1410c55dd99051f7127
2023-03-30 23:34:32 +00:00
Oleg Lyovin
6d75a854ef Fix choosing wrong diff tool for recovery-from-boot patch
'bsdiff' tool should be used when recovery and boot image
have mismatching entries. This includes not only the case
when system_as_root enabled, but include_recovery_dtbo and
include_recovery_acpio too.

Add check if they are enabled to synchronize logic with
core/Makefile, which already takes this into account.

Test: make dist -j
Change-Id: Ie673403ae8eae446d006cc524dd48ac6c56194fc
Signed-off-by: Oleg Lyovin <ovlevin@sberdevices.ru>
2023-03-22 17:53:26 +03:00
Treehugger Robot
7a40e97d74 Merge "Rename rbcrun/cmd to make go install work properly" 2023-03-16 20:26:22 +00:00
Dan Willemsen
ec5e3b8d28 Rename rbcrun/cmd to make go install work properly
The standard go tooling assumes that a package's binary name should be
the name of the containing folder. Before this change, `go install` or
`go build` (without `-o`) names this binary `cmd`.

Also run `go mod tidy` while we're here, so the tooling doesn't try to
download github.com/nbutton23/zxcvbn-go, which we don't have a local
copy of.

Test: treehugger
Change-Id: I8f79724e928720caf961cdad128655ef31c3e9b2
2023-03-15 20:06:59 -04:00
Wei Li
155377c1d6 Clean up the strings and use f-strings instead of string.format().
Bug: 272356600
Test: m sbom
Change-Id: Ifc399bacf25850b896ecb003703b7637a98c8907
2023-03-15 14:07:41 -07:00
Treehugger Robot
4fac9d776c Merge "Add an automated tools for generating brick OTA package" 2023-03-15 09:24:40 +00:00
Wei Li
15187b6f1a Merge "Generate SBOM of the target product in file sbom.spdx.json in product out directory." 2023-03-15 05:20:44 +00:00
Kelvin Zhang
f91d74bd52 Add an automated tools for generating brick OTA package
In the past, we have been generating brick OTA packages manually.
Automate this process.

Bug: 273561331
Test: create_brick_ota --product oriole brick.zip and apply the package
Change-Id: I7f4cf3cacaedd9d376b4cc07cbb388930bca643b
2023-03-14 16:58:36 -07:00
Wei Li
499333616c Generate SBOM of the target product in file sbom.spdx.json in product out directory.
Original aosp/2374663 was reverted, so this change cnotains the implementation but disabled by default and SBOM will not be built by default with "m dist".

The feature will be enabled later in small CLs after running tests successfully in forrest.

Test: m sbom
Test: m dist
Test: on aosp, lunch aosp_bluejay-userdebug && m dist
Bug: 266726655
Change-Id: I926d0f97f3a0330ef61d059f12ea660005d370e6
2023-03-14 15:14:06 -07:00
Kevin Han
afb2495f6f Merge "Revert "Generate SBOM of the target product in file sbom.spdx in..."" 2023-03-13 21:03:22 +00:00
Kevin Han
161d711d4d Revert "Generate SBOM of the target product in file sbom.spdx in..."
Revert submission 2481062-sbom

Reason for revert: b/273332655

Reverted changes: /q/submissionid:2481062-sbom

Change-Id: Ie8a73d8dc50592204acd489482e0e9d96ffbd5df
2023-03-13 20:57:37 +00:00
Wei Li
27d32b0957 Merge "Generate SBOM of the target product in file sbom.spdx in product out directory." 2023-03-13 17:43:59 +00:00
Wei Li
88ac7ae7df Generate SBOM of the target product in file sbom.spdx in product out directory.
Original aosp/2374663 was reverted, try to submit it again with some fixes in this.

Test: m sbom
Test: m dist
Test: on aosp, lunch aosp_bluejay-userdebug && m dist
Bug: 266726655
Change-Id: Icf305770473f0c448a1ad721cbe7addf737115e4
2023-03-13 17:38:04 +00:00
Wei Li
82308a21a3 Merge "Revert "Generate SBOM of the target product in file sbom.spdx in..."" 2023-03-10 06:40:54 +00:00
Android Culprit Assistant
95c3ea8704 Revert "Generate SBOM of the target product in file sbom.spdx in..."
This revert was created by Android Culprit Assistant. The culprit was identified in the following culprit search session (http://go/aca-get/5b65d203-d364-4ade-aabb-1330fe45236a).

Change-Id: I4c7d7645ec0c266eb4d591b3089ba02e78da0659
2023-03-10 06:18:03 +00:00
Wei Li
4e2aed055b Merge "Generate SBOM of the target product in file sbom.spdx in product out directory." 2023-03-10 04:47:47 +00:00
Wei Li
02797aa70e Generate SBOM of the target product in file sbom.spdx in product out directory.
Test: m sbom
Test: m dist
Bug: 266726655
Change-Id: I6464fa2dc416dea62c5cf688c923d6b496f044f5
2023-03-09 15:27:58 -08:00