Commit Graph

8104 Commits

Author SHA1 Message Date
Treehugger Robot
b1a7ba8735 Merge "Improve finalization perfomance by building only current version." 2023-05-10 15:19:12 +00:00
Kelvin Zhang
ea70695d92 Merge "Fix non-AB ota generation for directories" 2023-05-10 04:07:09 +00:00
Kelvin Zhang
b036007592 Fix non-AB ota generation for directories
Test: ota_from_target_files -v --block yukawa-target_files-10097950.zip
test.zip

Change-Id: I4291010d66cc28bdb6b6b334cdbed06daf3de837
2023-05-09 20:30:53 -07:00
Alex Buynytskyy
aac2a187d4 Improve finalization perfomance by building only current version.
Bug: 275409981
Test: run locally
Change-Id: If531178a17792166726ff7ed230592c392eeb899
2023-05-10 03:10:32 +00:00
Treehugger Robot
872023376e Merge "Remove unneeded rbcrun features" 2023-05-09 07:31:40 +00:00
Mårten Kongstad
98b0eeb2fc aconfig: simplify argument parsing in main
Improve readability by extracting how multiple input files are
transformed from a clap argument to an Input.

Bug: 279485059
Test: atest aconfig.test
Change-Id: I299179d39eca80e107342f6978984da79ee81e34
2023-05-08 12:47:34 +02:00
Mårten Kongstad
76adff2af0 aconfig: improve flag value tracing
Improve how the flag values (state, permission) are tracked when parsing
config and override files: migrate from a raw string to a proper struct,
as this will make it easier when aconfig will output this data in some
structured format for other tools to consume.

Also, rename Cache.debug to Cache.trace.

Bug: 279485059
Test: atest aconfig.test
Change-Id: Idad57c969515f697e9065429d8a44c38d8a512d2
2023-05-08 12:47:34 +02:00
Mårten Kongstad
80bff11aef Merge changes from topic "aconfig-part-2"
* changes:
  aconfig: change flag values to enabled/disabled enum
  aconfig: add read/write permission
  aconfig: Source: remove unnecessary #[derive(Serialize, Deserialize)]
2023-05-08 10:46:26 +00:00
Alex Buynytskyy
00a88ec3ec Script fixes and improvements.
Fix to mark all versions REL.

More robust patch application. Keep getting these build breakages, not sure why. Looks like git is
confusing one folder for the other?
https://android-build.googleplex.com/builds/submitted/10065142/test_suites_arm64/latest/logs/build_error.log

Bug: 275409981
Test: run locally
Change-Id: I6173b9ecae5e9ab12d9abf2eee3f2136dc353210
2023-05-06 20:55:10 +00:00
Cole Faust
a874f88cee Remove unneeded rbcrun features
- rblf_cli and rblf_env
- -c and -f

This is in preparation for making rbcrun able to function as a more
general purpose starlark interpreter.

Bug: 280685526
Test: go test, ./out/rbc ./build/make/tests/run.rbc, ./build/bazel/ci/rbc_dashboard.py --quick aosp_arm64
Change-Id: Ifff9ce7b4369422f39c5003bb85a168c78bde7cf
2023-05-05 11:52:02 -07:00
Mårten Kongstad
c68c4eabea aconfig: change flag values to enabled/disabled enum
Change the underlying type of a flag's value from bool to an explicit
enum (Disabled, Enabled): this will hopefully reduce future confusion on
how flags are intended to be used.

Bug: 279485059
Test: atest aconfig.test
Change-Id: I9535f9b23baf93ad5916ca06fb7d21277b4573eb
2023-05-05 16:20:09 +02:00
Mårten Kongstad
416330b060 aconfig: add read/write permission
Introduce the concept of flag read/write permissions: a read-only flag
can only have its value set during the build; a writable flag can by
updated in runtime.

Bug: 279485059
Test: atest aconfig.test
Change-Id: I3ec5c9571faa54de5666120ccd60090d3db9e331
2023-05-05 13:45:28 +02:00
Mårten Kongstad
2937566c55 aconfig: Source: remove unnecessary #[derive(Serialize, Deserialize)]
The Source struct is no longer serialized/deserialized. Remove the
unused derives.

Bug: 279485059
Test: atest aconfig.test
Change-Id: Ifd78988ed8134ab43013314b4437e428a8927981
2023-05-05 10:57:19 +02:00
Mårten Kongstad
0b6b92e5a5 Merge changes from topic "aconfig-part-1"
* changes:
  aconfig: add support for changing flag value based on build
  aconfig: introduce cache
  aconfig: define Aconfig proto
  aconfig: add support for cargo
2023-05-04 20:36:22 +00:00
Mårten Kongstad
09c28d1689 aconfig: add support for changing flag value based on build
Teach aconfig about build IDs (continuously increasing integers). Extend
the aconfig file format to allow flags to say "by default, my value is
X, but starting from build ID A, it's Y, and from build ID B, it's Z".

Bug: 279485059
Test: atest aconfig.test
Change-Id: Idde03dee06f6cb9041c0dd4ca917c8b2f2faafdd
2023-05-04 14:34:55 +02:00
Mårten Kongstad
4d2b4b047b aconfig: introduce cache
Introduce the Cache struct to represent parsed and verified aconfig and
override content. Most commands in aconfig will work of an existing
cache file, eliminating the need to re-read the input every time.

Restructure main.rs to use clap to create a proper command line
interface with support for sub-commands. main.rs is responsible for
parsing the command line, performing disk I/O and calling the correct
subcommand implementation (in commands.rs).

To simplify unit tests, subcommands never perform explicit I/O; instead
they only work with Read and Write traits.

Also add dependencies on clap, serde and serde_json.

Bug: 279485059
Test: atest aconfig.test
Change-Id: Ib6abf2eabd264009804f253874b6fba924fc391b
2023-05-04 10:40:26 +02:00
Mårten Kongstad
bb520729e8 aconfig: define Aconfig proto
Fill in aconfig.proto. Define Aconfig definitions (for introducing flags
and setting their values) and Overrides (for overriding flags regardless
of what their definitions say). More changes to the proto schema are
expected when more of the aconfig project is outlined.

Use proto2 instead of proto3: this will cause the protobuf text parser
to error on missing fields instead of returning a default value which is
ambiguous, especially for booleans. (Also, the text protobuf parser
doesn't provide good error messages: if the input is missing a field,
the error is always "1:1: Message not initialized").

Unfortunately the generated Rust wrappers around the proto structs land
in an external crate, which prevents aconfig from adding new impl
blocks. Circumvent this by converting the data read from proto into
structs defined in the aconfig crate.

Change main.rs to parse (static) text proto.

Also add dependency on anyhow.

Bug: 279485059
Test: atest aconfig.test
Change-Id: I512e3b61ef20e2f55b7699a178d466d2a9a89eac
2023-05-04 10:35:25 +02:00
Treehugger Robot
c3216a321b Merge "Support generating partial OTAs from extracted target_files" 2023-05-04 00:05:29 +00:00
Treehugger Robot
626b012556 Merge "Update sdk finalizer for change to api_levels_released_versions" 2023-05-03 17:24:31 +00:00
Mårten Kongstad
fe753f5365 aconfig: add support for cargo
Officially, aconfig is build using the Android tool-chain. However, to
speed up the local development cycle, add support for building with
cargo.

While it is possible to tell cargo to place the build artifacts outside
the source tree, there is no way to tell it to not generate the cargo
lock file in the same directory as Cargo.toml. Add a .gitignore to
ignore Cargo.lock and the target directory.

The way the Android build system and cargo generates code from the
protobuf files is slightly different. Tell cargo to enable the "cargo"
feature and introduce src/protos.rs to hide this difference from the
rest of the aconfig source.

Bug: 279485059
Test: m aconfig && aconfig
Test: atest aconfig.test
Test: cargo build
Test: cargo test
Change-Id: I85741f58cadae353ed95c124f566e4f4a7484186
2023-05-03 09:41:11 +02:00
Mårten Kongstad
e7a850e942 Merge "aconfig: add project scaffolding" 2023-05-03 06:18:02 +00:00
Kelvin Zhang
6b10e15acc Support generating partial OTAs from extracted target_files
This allows the build system to potentially paralleize generation of OTA
package and zipping of target files

Bug: 262185376
Bug: 227848550
Change-Id: I90b6c25761683ebe3803b22fc8e23540a5282c66
2023-05-02 20:48:14 -07:00
Mårten Kongstad
867a349ed9 aconfig: add project scaffolding
Introduce a new tool, aconfig, to manage build time configurations, such
as feature flags.

This CL adds a project skeleton. Later CLs will add the actual
implementation.

Note: there is no need for an explicit TEST_MAPPING file;
rust_test_host are automatically included in host-unit-tests and
explicitly adding these tests via TEST_MAPPING will cause an error.

Bug: 279485059
Test: m aconfig && aconfig
Test: atest aconfig.test
Change-Id: I94047a19a55ae4d45b4fd7a0c8105a91ddcdfd79
2023-05-02 17:36:30 +02:00
Treehugger Robot
ed1ab26f5d Merge "Use misc_info.txt to check for VABC" 2023-05-01 22:30:19 +00:00
Alex Buynytskyy
d03be09214 Set next sdk_int to previous sdk_int + 1.
OS and the finalization scripts use sdk_int - 1 in way too many places.

Bug: 275409981
Test: run locally
Change-Id: If4896a3eb0a6afadac5dbfd6ad455427d1c23d41
2023-05-01 18:56:26 +00:00
Kelvin Zhang
e634bde6bd Use misc_info.txt to check for VABC
Change-Id: I77f23a51ddb027254ccda8d7ab17e64205522300
2023-04-28 23:59:43 -07:00
Alex Buynytskyy
01db2a97da Correct if statement.
Bug: 275409981
Test: run locally
Change-Id: If6b8052748f816713cd16328ad1525dcd90ded69
2023-04-29 00:06:39 +00:00
Cole Faust
cdb1a45ac2 Update sdk finalizer for change to api_levels_released_versions
Change the starlark file, not the soong one.

Bug: 279095899
Test: manually ran this snippet
Change-Id: If7b93096743f79a5303db4330fc03cbc8fb5bae3
2023-04-28 14:23:46 -07:00
Alex Buynytskyy
5d205dc016 Change CI API level to keep linter happy.
Bug: 275409981
Test: run locally
Change-Id: I99d2ef4974a70f435f528c7a9358990fa41c9f5e
2023-04-27 15:30:19 -07:00
Alex Buynytskyy
d873b0e46b Update Android Studio Version Number.
Bug: 275409981
Test: run locally
Change-Id: I0d62ac8178f6d69f66adde8c5889ff86bcfad0ed
2023-04-27 18:18:57 +00:00
Treehugger Robot
99dc1341d1 Merge "Let the finalization script create ABI dumps for all architectures" 2023-04-27 05:52:25 +00:00
Alex Buynytskyy
d878cdf692 Merge "Mark master as VIC development branch." 2023-04-26 23:39:11 +00:00
Daniel Zheng
15c296958b Merge "Add --max_threads to OTA generation" 2023-04-26 21:53:53 +00:00
Alex Buynytskyy
0fc39526a1 Mark master as VIC development branch.
Test: ./build-step-1-and-m.sh
Bug: 279615150
Change-Id: Idb485846345ea146c04c5ecf6eb4ab6eea122f9b
Merged-In: Idb485846345ea146c04c5ecf6eb4ab6eea122f9b
2023-04-26 18:15:59 +00:00
Daniel Zheng
a30989bafa Add --max_threads to OTA generation
Adding an option to configure max number of threads used during
ota_from_target_files

Bug: 277794445
Test: tested ota_from_target_files
Change-Id: I9bae92ee43c21926434ab04563a2b4f0678b73eb
2023-04-26 10:52:12 -07:00
Hsin-Yi Chen
139cc54d81 Let the finalization script create ABI dumps for all architectures
The command that creates ABI dumps was written for testing purpose.
It creates the ABI dumps for aosp_arm64-user. The complete finalization
process should create ABI dumps for all supported architectures. This
commit removes the command parameters so that it will create all ABI
dumps by default.

Test: out/host/linux-x86/bin/create_reference_dumps
Bug: 276503801
Change-Id: Id6fbbac512a13d085ea74fec6e8b137485639467
2023-04-26 17:11:20 +08:00
Treehugger Robot
dd62b0da0f Merge "Support to merge PREBUILT_IMAGES" 2023-04-25 04:24:20 +00:00
Treehugger Robot
5ca02ff114 Merge "Fix incremental OTA generation failures" 2023-04-22 01:26:28 +00:00
Alex Buynytskyy
f5897ecdb6 Don't skip vndk finalization.
Bug: 275409981
Test: local run
Change-Id: I6651f29e3c5fd5f1ccb6ec07fd52491ba16f5086
2023-04-21 19:08:50 +00:00
Kelvin Zhang
ca45d7a85d Fix incremental OTA generation failures
source_file might be a directory, use the _ReadApexInfo() helper
function to read it instead of go straight to zipfiles.

Test: build incremental OTA for pixel
Bug: 279142680
Bug: 227848550
Bug: 277028723

Change-Id: Ia0041e2c72c1a57123519c970f6e734194a81cd2
2023-04-21 17:30:50 +00:00
Treehugger Robot
fad3925959 Merge "Update generateSPDXNamespace to generate a unique spdx doc namespace" 2023-04-21 16:50:00 +00:00
Treehugger Robot
269248add6 Merge "Cleanup the list of Soong prebuilt module types." 2023-04-21 01:40:01 +00:00
Wei Li
6f407ba0a8 Cleanup the list of Soong prebuilt module types.
Bug: 272356622
Test: CIs
Change-Id: I4031f0a73f5af1bbf1eb374bcd7d3a19b9609a46
2023-04-20 14:13:58 -07:00
Ibrahim Kanouche
397a9cf430 Update generateSPDXNamespace to generate a unique spdx doc namespace
Test: m compliance_sbom
Change-Id: Icc1e5aec0e1758ee32c43cfa79b5b49b3d1d9d30
2023-04-20 17:43:12 +00:00
Treehugger Robot
ece71703f8 Merge "Allow ota_from_target_file to work entirely on directories" 2023-04-20 15:17:50 +00:00
Bob Badour
ffba21bbae [LSC] Add LOCAL_LICENSE_KINDS to build/make
Added SPDX-license-identifier-Apache-2.0 to:
  tools/sbom/Android.bp

Bug: 68860345
Bug: 151177513
Bug: 151953481

Test: m all
Change-Id: I1f8e5fd8e787a4c53bf4caf80a4000d46d0291e4
2023-04-19 09:48:01 -07:00
Kelvin Zhang
9dbe2ce40f Allow ota_from_target_file to work entirely on directories
When building android, build system will first zip target_file directory
into a .zip file, and then invoke ota_from_target_files. We
ota_from_target_files can work directly on top of directories, we can
parallelize OTA generation and target_file zipping, making builds
faster.

Bug: 227848550
Bug: 277028723
Test: th
Change-Id: Id72bd7cca708af0e1bae2db754f39b27777cd601
2023-04-18 17:47:29 -07:00
Po Hu
81c3f4a884 Support to merge PREBUILT_IMAGES
For example, pvmfw.img comes from framework target files package
and dtbo.img comes from vendor target files package.

Test: merge_target_files
Change-Id: If2e6fac5ce3671e3387e801808a195fb7b2c55b5
2023-04-19 08:23:40 +08:00
Wei Li
49af939a23 Copy SBOM files of unbundled APKs to dist directory.
Bug: 266726655
Test: CIs
Test: tapas Browser2 Calendar arm userdebug && m dist
Change-Id: I82328dc4916451a99ad0bf6d7bb945c9b98d0bea
2023-04-18 15:57:34 -07:00
Treehugger Robot
67ca3c05c7 Merge "Fix duplicate super_empty.img" 2023-04-17 21:19:13 +00:00