Commit Graph

8345 Commits

Author SHA1 Message Date
Mårten Kongstad
f73b963283 aconfig: first iteration of Rust codegen
Add a new `create-rust-lib` command to generate Rust code. The output is
a src/lib.rs file; the build system is assumed to set the generated
crate's name.

For READ_ONLY flags, the generated code returns a hard-coded true or false.

For READ_WRITE flags, the generated code reaches out to DeviceConfig via
the cc_library server_configurable_flags via the
libprofcollect_libflags_rust Rust bindings. The build system is assumed
to add this to the generated crate's dependencies.

Note: libprofcollect_libflags_rust seems generic enough that it should
be moved to an official Rust wrapper for server_configurable_flags. This
is tracked in b/284096062.

Summary of module the built system is assumed to wrap the auto-generated
code in:

  rust_library {
      name: "lib<namespace>_rs",
      crate_name: "<namespace>_rs",
      edition: "2021",
      clippy_lints: "none",
      no_stdlibs: true,
      lints: "none",
      srcs: ["src/lib.rs"],
      rustlibs: [
          "libprofcollect_libflags_rust",
      ],
  }

Also add a set of test input to be used in the unit tests for a more
coherent test strategy. A follow-up CL will migrate the code in
commands.rs, codegen_java.rs and codegen_cpp.rs.

Bug: 279483360
Bug: 283907905
Test: atest aconfig.test
Test: manual: create cache from files in testdata, create rust lib, add to module template above, verify the module builds
Change-Id: I02606aa3686eda921116e33f7e2df8fd1156a7aa
2023-05-25 16:18:59 +02:00
Treehugger Robot
26fda7e335 Merge "Fix merge_target_files in GRF case" am: b3adc24301 am: 592ad2cb13 am: 7c629dbb53
Original change: https://android-review.googlesource.com/c/platform/build/+/2564310

Change-Id: I5a0f0236e07f50b7d8cfa05b476734ed3f98300a
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-25 13:06:05 +00:00
Treehugger Robot
592ad2cb13 Merge "Fix merge_target_files in GRF case" am: b3adc24301
Original change: https://android-review.googlesource.com/c/platform/build/+/2564310

Change-Id: I564bb6d43bfa5b9cb79463f0ba50038d03ef6abb
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-25 11:37:45 +00:00
Treehugger Robot
b3adc24301 Merge "Fix merge_target_files in GRF case" 2023-05-25 10:46:32 +00:00
Mårten Kongstad
993111f91c aconfig: improve dump --format=debug output
Use Rust's {:#?} formatter for more readable output.

Bug: 279485059
Test: atest aconfig.test
Change-Id: I127f413e3d7aebfba96cad1dd58d9e261dd613a4
2023-05-24 14:56:45 +02:00
Treehugger Robot
4f4da804bf Merge "Unsparse images before generating OTA" am: b4406d06f0 am: 74ad39125c am: 8b55fd52a0
Original change: https://android-review.googlesource.com/c/platform/build/+/2598008

Change-Id: Ib062901d1b91380b98da95f38666c66f1205f587
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-23 23:11:17 +00:00
Treehugger Robot
74ad39125c Merge "Unsparse images before generating OTA" am: b4406d06f0
Original change: https://android-review.googlesource.com/c/platform/build/+/2598008

Change-Id: I40ac20dd7b30d11e1eda77688718566af4c1c416
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-23 21:22:54 +00:00
Treehugger Robot
b4406d06f0 Merge "Unsparse images before generating OTA" 2023-05-23 20:51:10 +00:00
Treehugger Robot
2f727af4b9 Merge "Add 16K artifacts to PREBUILT_IMAGES/ dir of target_files" am: 8485145bd7 am: f6d61bd3c1 am: af0cfd9e98
Original change: https://android-review.googlesource.com/c/platform/build/+/2595883

Change-Id: I4f6bac39abfd1ec19d21b763201770c92a2e0c3f
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-23 04:55:01 +00:00
Treehugger Robot
f6d61bd3c1 Merge "Add 16K artifacts to PREBUILT_IMAGES/ dir of target_files" am: 8485145bd7
Original change: https://android-review.googlesource.com/c/platform/build/+/2595883

Change-Id: I36815cfa877645e242d40a3b4b4cadd0e50a6ef5
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-23 03:30:56 +00:00
Treehugger Robot
8485145bd7 Merge "Add 16K artifacts to PREBUILT_IMAGES/ dir of target_files" 2023-05-23 02:36:17 +00:00
Yihan Dong
355b6b4a52 Merge "Remove GTS-Verifier license check" into udc-dev am: a75b82f9d2
Original change: https://googleplex-android-review.googlesource.com/c/platform/build/+/23343092

Change-Id: I41e817b323c93d103486825200936b41fdfac8d5
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-23 02:23:02 +00:00
Yihan Dong
a75b82f9d2 Merge "Remove GTS-Verifier license check" into udc-dev 2023-05-23 02:00:54 +00:00
Kelvin Zhang
2268091521 Unsparse images before generating OTA
Test: th
Bug: 283172692
Change-Id: Ie6d3dc704fd9a8c107e2888222e4c2bf804dad3e
2023-05-22 10:22:59 -07:00
Treehugger Robot
1ad09e0bfa Merge changes from topic "aconfig-stable-flag-order" am: f469fd6cc6
Original change: https://android-review.googlesource.com/c/platform/build/+/2595206

Change-Id: Ic66ea4047487b6f8c4c51237bf583d035b7032d1
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-22 12:30:11 +00:00
Mårten Kongstad
2f95444281 aconfig: sort items in cache by name
Introduce a builder pattern for constructing a cache from flag
declarations and flag values. Teach the builder to sort the flags by
name as the last step. This will ensure consistent dump output
regardless of the order flags are specified in the input files.

Bug: 279485059
Test: atest aconfig.test
Change-Id: Icdd62f51fa3761a469663f17581a83d9909e9ffe
2023-05-22 10:01:05 +02:00
Mårten Kongstad
af677038b3 aconfig: dump: support multiple caches
Teach `aconfig dump` to read multiple caches at the same time.

A cache represents a single namespace, and the Android build creates
multiple caches for different namespaces. By passing in all those cache
files to `aconfig dump`, aconfig will create a complete overview of all
flags in the Android tree.

Caches are traversed in order of namespace to produce the same output
regardless of the order the cache files are given on the command line.
If two caches use the same namespace, their order with respect to each
other is undefined.

Bug: 279485059
Test: atest aconfig.test
Change-Id: I54c3950bbb7b2be7d96c8928e78ae83aa626c2e2
2023-05-22 09:57:48 +02:00
Kelvin Zhang
decee4adb7 Add 16K artifacts to PREBUILT_IMAGES/ dir of target_files
Test: th
Bug: 283506035
Change-Id: I9be3982b5a4324ec839937d10c33266c7d27c525
2023-05-19 14:31:00 -07:00
Yihan Dong
58c428e336 Remove GTS-Verifier license check
The whole GTS-Verifier will be removed. Before that, remove GTS-V
license check first.

Ignore-AOSP-First: This CL must be submit with other CLs in the topic to avoid breaking the test. This CL will be cherry pick to AOSP after submitting.

Bug: 283384166
Test: m gts
Change-Id: I5bc8c1f4609f55cedd5bfbed014b65de164e2b16
2023-05-19 10:38:55 +00:00
Wei Li
233d5b97f8 Some changes to support SBOM generation for b build unbundled APEXs.
1) Use output file path of installed files in build system since there is no PRODUCT_OUT in Bazel
2) Use CONTAINS to describe the relationship between a APEX and files it contains
3) Generate SBOM of APEXs, which is similar to SBOM of products

Bug: 275472038
Test: CIs
(cherry picked from https://android-review.googlesource.com/q/commit:fd7e6517d345d3b8d4af12dae345434d968b83b9)
Merged-In: I41622366e5e6ed9dc78cca7bc7bb69a1f8f9bd9f
Change-Id: I41622366e5e6ed9dc78cca7bc7bb69a1f8f9bd9f
2023-05-19 00:35:20 +00:00
Daniel Zheng
b87d46cb1a Merge "Add --max_threads to OTA generation" into udc-dev 2023-05-18 19:53:12 +00:00
Treehugger Robot
97f144f4f7 Merge "aconfig: Add first iteration of cpp codegen to aconfig" am: 4266df817d
Original change: https://android-review.googlesource.com/c/platform/build/+/2585679

Change-Id: Ic76387fcc26f48db6417cbf564a450d6acc6a184
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-17 19:56:49 +00:00
Treehugger Robot
4266df817d Merge "aconfig: Add first iteration of cpp codegen to aconfig" 2023-05-17 19:37:28 +00:00
Wei Li
802310c4fd Merge "Some changes to support SBOM generation for b build unbundled APEXs." am: eb035dc89f
Original change: https://android-review.googlesource.com/c/platform/build/+/2585677

Change-Id: I0f302bf5cf4e6313aedc836034ccd9111f4ec534
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-17 19:33:20 +00:00
Baligh Uddin
f04ce4eb36 Merge "Add upstream package of a prebuilt fork package, which will have the package information from the METADATA file." am: 82d450e501
Original change: https://android-review.googlesource.com/c/platform/build/+/2590485

Change-Id: I2b8dace3a3b2f257919c4f2a1ca775d766e0aff9
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-17 19:32:56 +00:00
Wei Li
eb035dc89f Merge "Some changes to support SBOM generation for b build unbundled APEXs." 2023-05-17 19:19:47 +00:00
Satoshi Futenma
6c8f896589 Support custom payload signer in merge_ota
Add options to handle custom payload signer as it is required in
merge_ota.py as well if the original OTA packages are signed by
the signer.

If input is only one OTA, clone apex_info.pb to the target.

Use common.ZipWriteStr instead of zipfile.writestr, this ensures
that the same permission for the entries as done by
ota_from_target_files.

Bug: 282189563
Test: manual. pass single OTA to merge_ota, with same signing
    parameters as originally used. Confirm that output zip is
    binary identical to input.
(cherry picked from https://android-review.googlesource.com/q/commit:1f93ce20c5a5f3e7c2f6b5e1f9fdc4d2572220cc)
Merged-In: I3b926b8cd69bc74dff6ccf8b5ccc72bedffcac6f
Change-Id: I3b926b8cd69bc74dff6ccf8b5ccc72bedffcac6f
2023-05-17 17:22:51 +00:00
Daniel Zheng
0d3c15c2ae Add --max_threads to OTA generation
Adding an option to configure max number of threads used during
ota_from_target_files

Bug: 277794445
Test: tested ota_from_target_files
(cherry picked from https://android-review.googlesource.com/q/commit:a30989bafa820faf24eca7e11e48f8f122d7991f)
Merged-In: I9bae92ee43c21926434ab04563a2b4f0678b73eb
Change-Id: I9bae92ee43c21926434ab04563a2b4f0678b73eb
this patch is necessary for partner to improve OTA generation b/277794445
2023-05-17 16:14:05 +00:00
Dennis Shen
1dc9ad4662 aconfig: Add first iteration of cpp codegen to aconfig
The general idea to reuse java codegen's very neat tiny template idea.
For generated cpp code, it is in the form of a collection of classes
inside a namespace. The reason we choose a collection of classes rather
than a collection of static functions is because gmock test technology
only supports mocking virtual method.

Bug: b/279483801
Test: atest aconfig.test
Change-Id: I9ba00667437ff7c3e147ff2828171fc95528bebf
2023-05-17 14:57:58 +00:00
Wei Li
16e7aa3c2e Add upstream package of a prebuilt fork package, which will have the package information from the METADATA file.
Test: CIs
Test: lunch barbet-user && m sbom
Change-Id: Ic8eb42c369de8c94c7977b9631ff4b9084dfef01
2023-05-17 01:38:05 +00:00
Joe Onorato
6ee16caf99 Merge "Use the namepace as the java package name in aconfig and call the class "Flags"" am: 7833547153
Original change: https://android-review.googlesource.com/c/platform/build/+/2588271

Change-Id: Id42b087eea8e723d2dc66534760d00842263dda6
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-16 23:14:30 +00:00
Treehugger Robot
e1623bb140 Merge "Allow source build to be an extracted target_files" am: fce28c1b97
Original change: https://android-review.googlesource.com/c/platform/build/+/2589846

Change-Id: I0a4ebf623140c02b39c7d3cb2e9bd060b7ea4090
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-16 23:14:19 +00:00
Joe Onorato
7833547153 Merge "Use the namepace as the java package name in aconfig and call the class "Flags"" 2023-05-16 23:13:52 +00:00
Treehugger Robot
fce28c1b97 Merge "Allow source build to be an extracted target_files" 2023-05-16 22:56:29 +00:00
Treehugger Robot
a2ebcf5e03 Merge "Support custom payload signer in merge_ota" am: bb404f2b27
Original change: https://android-review.googlesource.com/c/platform/build/+/2587127

Change-Id: Idd3456e86e1785b5611810f71a42042b66d5610c
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-16 22:31:54 +00:00
Treehugger Robot
bb404f2b27 Merge "Support custom payload signer in merge_ota" 2023-05-16 21:39:58 +00:00
Joe Onorato
0c4ef0f88c Use the namepace as the java package name in aconfig and call the class "Flags"
Test: aconfig.test
Change-Id: I15032262aad2c80d1ec1bd8fa81664a846b307ec
2023-05-16 14:27:25 -07:00
Treehugger Robot
1be8ad0258 Merge changes from topic "aconfig-cleanup-1" am: d4a32797c9
Original change: https://android-review.googlesource.com/c/platform/build/+/2591069

Change-Id: Ic657d8f404dc001ba645ef443e8ae6fd94f575ac
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-16 17:20:00 +00:00
Mårten Kongstad
e17ba5f423 aconfig: cache: reject empty namespace and name fields
Add invariant to struct Cache: all flag namespace and name fields added
to the cache are required to be non-empty strings.

Bug: 279485059
Test: atest aconfig.test
Change-Id: I5ff34ec8feccc19e52241d4221fc87699518f3ff
2023-05-16 12:52:43 +02:00
Mårten Kongstad
6b9e382eed aconfig: remove calls to unwrap (outside tests)
Do not call unwrap outside tests: replace existing uses with Result
return values or infallible alternatives.

Bug: 279485059
Test: atest aconfig.test
Change-Id: Ie5919b704b23a0f96bbef84ffbe9270d667cecd8
2023-05-16 11:19:58 +02:00
Mårten Kongstad
ba94e6a6b2 aconfig: rename enum Format -> enum DumpFormat
Rename enum Format to enum DumpFormat to make it more apparent what it
refers to.

Bug: 279485059
Test: atest aconfig.test
Change-Id: I869be020b69618b036fa05247f155d9e35ff85e2
2023-05-16 11:00:16 +02:00
Satoshi Futenma
1f93ce20c5 Support custom payload signer in merge_ota
Add options to handle custom payload signer as it is required in
merge_ota.py as well if the original OTA packages are signed by
the signer.

If input is only one OTA, clone apex_info.pb to the target.

Use common.ZipWriteStr instead of zipfile.writestr, this ensures
that the same permission for the entries as done by
ota_from_target_files.

Bug: 282189563
Test: manual. pass single OTA to merge_ota, with same signing
    parameters as originally used. Confirm that output zip is
    binary identical to input.
Change-Id: I3b926b8cd69bc74dff6ccf8b5ccc72bedffcac6f
2023-05-16 10:21:05 +02:00
Kelvin Zhang
b97416e09e Allow source build to be an extracted target_files
common.LoadInfoDict() already supports loading from extracted
directories, just use it.

Test: generate an incremental OTA where both inputs are directories
Bug: 227848550
Bug: 277028723
Change-Id: Iedba831bb4d65d971df6b2ac95279e3234a02e2f
2023-05-15 11:13:26 -07:00
Mårten Kongstad
bdd924b335 Merge "aconfig: use bail! and ensure! where applicable" am: 7890fa3e84
Original change: https://android-review.googlesource.com/c/platform/build/+/2588765

Change-Id: Iaec315775f38b17825d4c98f077773a6023a1259
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-15 14:48:02 +00:00
Mårten Kongstad
e66b89f635 aconfig: use bail! and ensure! where applicable
The Android Rust style guide recommends anyhow's bail! and ensure!
macros where possible. Update the code accordingly.

Bug: 279485059
Test: atest aconfig.test
Change-Id: I7db7cc2d8ec15e1d3450657ad000e879adb19e08
2023-05-15 11:10:12 +02:00
Mårten Kongstad
011f6d42b2 Merge changes from topic "aconfig-part-5" am: 71f9dabe47
Original change: https://android-review.googlesource.com/c/platform/build/+/2584221

Change-Id: I0520bb7fce7cf788cd45bec7b1897b902dd86bc9
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-12 13:03:15 +00:00
Mårten Kongstad
71f9dabe47 Merge changes from topic "aconfig-part-5"
* changes:
  aconfig: follow Java conventions for Java file paths
  aconfig: separate flag declarations and flag values
2023-05-12 12:21:02 +00:00
Treehugger Robot
531125c397 Merge "parallelize add_partition_calls only if output_zip is None" am: 8d5d2257d9
Original change: https://android-review.googlesource.com/c/platform/build/+/2584696

Change-Id: Ida95b894dc799345b02aa64b58f4d6392db92d7f
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2023-05-12 11:39:56 +00:00
Treehugger Robot
8d5d2257d9 Merge "parallelize add_partition_calls only if output_zip is None" 2023-05-12 10:56:23 +00:00
Mårten Kongstad
d42eeeba3d aconfig: follow Java conventions for Java file paths
Update codegen_java to write the generated Java file(s) to
"java/package/File.java" instead of just "File.java".

Also generalize codegen_java::GeneratedFile to commands::OutputFile in
preparation for the upcoming C++ and Rust codegen.

Also change Java package name to 'com.android.internal.aconfig'.

Bug: 279485059
Test: atest aconfig.test
Change-Id: I13978697e35010fe6be8637aa495d4b852dbed7e
2023-05-12 10:01:00 +02:00