Commit Graph

7456 Commits

Author SHA1 Message Date
Ramji Jiyani
13a4137929 core & tools: system_dlkm: add dynamic partition
Converts existing static partition support to a
dynamic partition.

Bug: 200082547
Test: TH
Signed-off-by: Ramji Jiyani <ramjiyani@google.com>
Change-Id: Ifd6d0c2a04e947b16f8b241e99cca594a1d315ae
2022-01-30 08:43:08 +00:00
Colin Cross
e4442b7df6 Merge changes I3db6310e,I061be2b9,I7e81936d,If0684cf1,I404b138f
* changes:
  Move compliance commands into subdirectories
  Give compliance a pkgPath
  Move policy subdirectory into package directory
  Fix errors caught by go vet in compliance package
  Run gofmt on compliance
2022-01-28 21:24:26 +00:00
Bob Badour
d8724181a9 Merge changes I73975ca7,I98e2c1ee
* changes:
  license metadata bill of materials list
  license metadata shipped libraries list
2022-01-28 04:09:05 +00:00
Treehugger Robot
b816e5863d Merge "license metadata html notice files" 2022-01-28 03:59:51 +00:00
Colin Cross
d0f05c9eda Move compliance commands into subdirectories
Move each compliance command into its own directory to avoid Go tooling
considering all the files to be a single package and complaining about
conflicting global names.

Test: go test ./...
Change-Id: I3db6310e7368bcc7fe6a3704b40a84078ed44209
2022-01-27 17:14:03 -08:00
Colin Cross
38a6193180 Give compliance a pkgPath
Change the pkgPath of compliance to android/soong/tools/compliance
to distinguish it from the standard library.

Also add a go.mod file so that the Go tools can find the package.

Test: go test ./...
Change-Id: I061be2b91378db6508e9c88be2548648af8be4e7
2022-01-27 17:14:03 -08:00
Colin Cross
2c51bd1d49 Move policy subdirectory into package directory
Go's tooling expects that all files in the same package are in the
same directory.  Move the policy/*.go files to policy_*.go.

Test: go test ./...
Change-Id: I7e81936ddd20a5ffb4770ae23bdb6e411d6924cc
2022-01-27 17:14:03 -08:00
Colin Cross
179ec3e55e Fix errors caught by go vet in compliance package
`go test` implicitly runs `go vet` and fails the tests if vet
errors are found.  Fix all the issues found by vet.

Test: go test build/make/tools/compliance/...
Change-Id: If0684cf124ece4931af440008cd44a61c22de475
2022-01-27 17:14:02 -08:00
Colin Cross
35f79c37aa Run gofmt on compliance
Test: builds
Change-Id: I404b138faac0db9a92201be6163b19d9e6bff810
2022-01-27 17:14:02 -08:00
Treehugger Robot
29429f928f Merge "license metadata text notice files" 2022-01-28 00:23:47 +00:00
Treehugger Robot
766c5343e2 Merge "Add a version code and version name to fsverity manifest APK" 2022-01-27 23:48:24 +00:00
Treehugger Robot
c1edfc3b29 Merge "Fix bug where VABC xor is used on unsupported builds" 2022-01-27 22:30:57 +00:00
Colin Cross
658172d982 Add a version code and version name to fsverity manifest APK
The post-build signing step expects to find a version code in every
APK, add the version code and version name.

Bug: 216584708
Test: manual
Change-Id: Ic43730df50ef32611d5b4159b46c8bbf9906e009
2022-01-27 12:46:22 -08:00
Iavor-Valentin Iftime
b837b7170b Skip generating care_map for non-AB updates
Bug: 186097910
Test: build a non-AB update merged target using Forrest,
      No errors reported and no META/care_map.pb is generated.

Change-Id: I74990b0aa7f46d4c4d40098a4b102f7e04166c39
2022-01-27 18:35:43 +00:00
Kelvin Zhang
f175801f62 Fix bug where VABC xor is used on unsupported builds
Test: th
Bug: 216426344
Change-Id: I352136eb5f261ee1ca1567530109e7623c5f978b
2022-01-27 10:23:10 -08:00
Bob Badour
2546febca7 license metadata bill of materials list
Introduce the below command-line tool:

bom outputs a text file listing 1 installed path per line.

Bug: 68860345
Bug: 151177513
Bug: 151953481
Bug: 213388645
Bug: 210912771

Test: m all
Test: m systemlicense
Test: m bom; out/soong/host/linux-x85/bom ...

where ... is the path to the .meta_lic file for the system image. In my
case if

$ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD)

... can be expressed as:

${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic

Change-Id: I73975ca7b161945a62ff83888527ce01fb47d75a
2022-01-26 20:58:24 -08:00
Bob Badour
00c8a38714 license metadata shipped libraries list
Introduce the below command-line tool:

shippedlibs outputs a text file listing 1 library per line containing
the libraries the shipped image is derived from.

Bug: 68860345
Bug: 151177513
Bug: 151953481
Bug: 213388645
Bug: 210912771

Test: m all
Test: m systemlicense
Test: m shippedlibs; out/soong/host/linux-x85/shippedlibs ...

where ... is the path to the .meta_lic file for the system image. In my
case if

$ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD)

... can be expressed as:

${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic

Change-Id: I98e2c1eec94ad7878e911eee2458a26e12ee2b19
2022-01-26 17:21:39 -08:00
Bob Badour
6ea1457c9b license metadata html notice files
Introduce the below command-line tool:

htmlnotice outputs a NOTICE.html file constructed from the license
texts of the transitive closure of dependencies.

Bug: 68860345
Bug: 151177513
Bug: 151953481
Bug: 213388645
Bug: 210912771

Test: m all
Test: m systemlicense
Test: m htmlnotice; out/soong/host/linux-x85/htmlnotice ...

where ... is the path to the .meta_lic file for the system image. In my
case if

$ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD)

... can be expressed as:

${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic

Change-Id: Idbbeb2939d8cbf497237516fe468004fcd2d72a1
2022-01-26 16:02:17 -08:00
Bob Badour
e6fdd1403b license metadata text notice files
Introduce the below command-line tool:

textnotice outputs a NOTICE text file constructed from the license
texts of the transitive closure of dependencies.

Bug: 68860345
Bug: 151177513
Bug: 151953481
Bug: 213388645
Bug: 210912771

Test: m all
Test: m systemlicense
Test: m textnotice; out/soong/host/linux-x85/textnotice ...

where ... is the path to the .meta_lic file for the system image. In my
case if

$ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD)

... can be expressed as:

${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic

Change-Id: Ia691869fd8e58ef008024f48c23b1a4b4435677a
2022-01-26 16:02:11 -08:00
Treehugger Robot
ef8f4426ef Merge "releasetools: system_dlkm: add image to dist zip" 2022-01-25 20:09:57 +00:00
Bob Badour
d2c7c08e7c Fix licenses and license texts.
Use Android-Apache-2.0 for all Apache 2.0

Remove legacy_restricted (no longer appears in code).

Remove Android.bp which did not build anything but was a placeholder
for the missing license texts.

Test: m nothing
Change-Id: Ia491a59c735ca20afda9b4bbac9d8b33dbf8874e
2022-01-24 16:19:41 -08:00
Ramji Jiyani
bdef28f2ee releasetools: system_dlkm: add image to dist zip
Bug: 200082547
Test: Manual builds:
      1. Set BOARD_USES_SYSTEM_DLKM_PARTITION
      2. Set BOARD_PREBUILT_SYSTEM_DLKM_IMAGE to external image
      - Check that "m dist" put the system_dlkm image in
      out/dist/aosp_cf_x86_64_phone-img-eng.ramjiyani.zip
Signed-off-by: Ramji Jiyani <ramjiyani@google.com>
Change-Id: I8b7dd40e178db7fe542253bff19040095847fa57
2022-01-24 21:27:52 +00:00
Chih-Hung Hsieh
02f0df83fc Lower severity of 3 tidy warnings
* bcmp/bcopy/bzero are not insecureAPI on Android.
* Android projects may enable/disable clang-tidy checks locally;
  warn/*patterns.py keep global classification of all warnings.

Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: I7720f7babd0a33c1b6602c9a8bb67565c45db94e
2022-01-20 16:13:12 -08:00
Kelvin Zhang
6fe2fc031f Add erofs compressor type param
Test: th
Bug: 206729162
Change-Id: I3ddd1a55e1544a8a4d24ed445562f05ad0aaad2c
2022-01-18 10:17:49 -08:00
Kelvin Zhang
554c8bee14 Fix formatting error
Test: th
Bug: 214388458
Change-Id: I2dc72884aad4f440b4b68566e2e29f9f6f5aa73e
2022-01-18 10:17:49 -08:00
Kelvin Zhang
4a376986ea Merge "Fix check_tf failure when duplicate package names are present" 2022-01-18 18:14:02 +00:00
Jingwen Chen
2a31f3df87 Migrate java-event-logs-tags.py to python3.
These are used in the Bazel Android app builds.

Test: m droid
Test: CI
Test: b run //build/make/tools:java-event-log-tags

Change-Id: Iaffe6f974008d1a0532a849353d25df02197afd2
2022-01-17 13:02:53 +00:00
Yi-yo Chiang
8b4e2fd6c0 Merge changes from topic "gki-avb-vts"
* changes:
  releasetools: Android T GKI certification scheme
  Android T GKI certification scheme
2022-01-17 04:52:28 +00:00
Yi-Yo Chiang
36054e2daf releasetools: Android T GKI certification scheme
Companion change of Iaf48a6e3d4b97fa6bfb5e1635a288b045baa248f
To support new GKI certification scheme for boot.img and
init_boot.img on upgrading and launching device combinations.

Bug: 210367929
Bug: 211741246
Bug: 203698939
Test: atest --host releasetools_test:test_common
Test: unpack_bootimg --boot_img boot.img
Test: unpack_bootimg --boot_img init_boot.img
Test: avbtool info_image --image out/boot_signature
Change-Id: I3749297c09c3899046550e4be776acbeea37ef2e
2022-01-14 19:50:32 +08:00
Treehugger Robot
2f1f985480 Merge "Add option to enable lz4diff" 2022-01-13 20:33:31 +00:00
Kelvin Zhang
67553739e3 Fix check_tf failure when duplicate package names are present
Test: check_target_files_signatures sdk_gphone64_x86_64-target_files-8069754.zip
Bug: 214388458

Change-Id: I0a8a0d0596c04a9da4283d4d4caeb3bb62762ccc
2022-01-13 12:30:22 -08:00
Kelvin Zhang
f2728d615e Add option to enable lz4diff
When lz4diff is enabled, we inject the host copy of liblz4.so in
target_file to delta_generator. This is to ensure that host tooling can
produce same compressed output as the device.

Test: th
Bug: 206729162

Change-Id: I2d8206f7ec54fadedea16bf3d811b8353bc9414d
2022-01-12 17:21:00 -08:00
Treehugger Robot
3a0656307c Merge "Regression testcase for shipped vs nonshipped." 2022-01-12 23:35:17 +00:00
Treehugger Robot
710813e871 Merge "Fix typo in doc: code segment for container dot" 2022-01-12 22:31:40 +00:00
Bob Badour
00bf3d9b29 Regression testcase for shipped vs nonshipped.
Bug: 68860345
Bug: 151177513
Bug: 151953481

Test: m all
Test: m systemlicense
Test: m listshare; out/soong/host/linux-x86/bin/listshare ...
Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ...
Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ...
Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ...

where ... is the path to the .meta_lic file for the system image. In my
case if

$ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD)

... can be expressed as:

${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic

Change-Id: I17221ec8fe429b5f745c0733f8bb1ab0ea2f4567
2022-01-11 16:38:55 -08:00
Treehugger Robot
49380ebecf Merge changes Ia2ec1b81,Ibc831ae8
* changes:
  Performance and scale.
  Use struct{}
2022-01-12 00:27:41 +00:00
Treehugger Robot
b3f3e7b3f7 Merge "Fold annotations." 2022-01-12 00:27:35 +00:00
Bob Badour
99dba55b4b Fix typo in doc: code segment for container dot
Test: not applicable
Change-Id: Ibaadc6e8fd4a267b50a46feb2dd6278ff1f5ace1
2022-01-11 14:46:22 -08:00
Bob Badour
103eb0f9bc Performance and scale.
Defer edge creation.

Don't create edges until the count is known to avoid repeated allocate+
copy operatios.

Limit resolutions.

Allow only a single resolution condition set per target, and overwrite
intermediate results. Reduces memory and obviates allocations.

Propagate fewer conditions.

Instead of propagating notice conditions to parents in graph during
initial resolve, leave them on leaf node, and attach to ancestors in
the final walk. Reduces copies.

Parallelize resolutions.

Use goroutines, mutexes, and waitgroups to resolve branches of the
graph in parallel. Makes better use of available cores.

Don't accumulate resolutions inside non-containers.

During the final resolution walk, only attach actions to ancestors from
the root down until the 1st non-aggregate. Prevents an explosion of
copies in the lower levels of the graph.

Drop origin for scale.

Tracking the origin of every potential origin for every restricted
condition does not scale. By dropping origin, propagating from top
to bottom can prune many redundant paths avoiding an exponential
explosion.

Conditions as bitmask.

Use bit masks for license conditions and condition sets. Reduces maps
and allocations.

Bug: 68860345
Bug: 151177513
Bug: 151953481

Test: m all
Test: m systemlicense
Test: m listshare; out/soong/host/linux-x86/bin/listshare ...
Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ...
Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ...
Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ...

where ... is the path to the .meta_lic file for the system image. In my
case if

$ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD)

... can be expressed as:

${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic

Change-Id: Ia2ec1b818de6122c239fbd0824754f1d65daffd3
2022-01-11 10:40:50 -08:00
Bob Badour
5446a6f8e1 Use struct{}
Using struct{}{} as the payload for set maps reduces memory use for
large sets.

Bug: 68860345
Bug: 151177513
Bug: 151953481

Test: m all
Test: m systemlicense
Test: m listshare; out/soong/host/linux-x86/bin/listshare ...
Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ...
Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ...
Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ...

where ... is the path to the .meta_lic file for the system image. In my
case if

$ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD)

... can be expressed as:

${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic

Change-Id: Ibc831ae80fc50f35e1000348fb28fc0167d0ebed
2022-01-11 10:40:50 -08:00
Bob Badour
67d8ae390a Fold annotations.
Use annotation slices looked up from map instead of read from disk to
reduce copies.

Bug: 68860345
Bug: 151177513
Bug: 151953481

Test: m all
Test: m systemlicense
Test: m listshare; out/soong/host/linux-x86/bin/listshare ...
Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ...
Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ...
Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ...

where ... is the path to the .meta_lic file for the system image. In my
case if

$ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD)

... can be expressed as:

${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic

Change-Id: Ibf6f7d319092e0d54ab451ffbbd35dcd7d0080e0
2022-01-11 10:40:39 -08:00
Treehugger Robot
9d83209f12 Merge "Do not append .meta_lic to meta_lic" 2022-01-11 06:11:14 +00:00
Bob Badour
63a281cc29 Do not append .meta_lic to meta_lic
Test: manual
Change-Id: I8d7162dc55f057d542faaa7feb546050d3a872e7
2022-01-10 18:00:03 -08:00
Jiyong Park
efbb6ffa00 Add min_sdk_version to BuildManifest.apk
sign_target_files_apks demands it.

Bug: 213823227
Test: run com.google.android.tradefed.ota.SigningToolTest#Testcase1_Signingtest
Change-Id: I1bba2b25a71449a57d284024e77fb76d1b79a0ee
2022-01-10 20:14:05 +09:00
Treehugger Robot
dcd7e283d2 Merge "Add option to enable zucchini" 2022-01-09 20:38:18 +00:00
Martin Stjernholm
76537be0a9 Merge "Add an option to specify the path to aapt2." 2022-01-09 19:11:49 +00:00
Treehugger Robot
555c3a7dc7 Merge "Write signature size even when it's empty" 2022-01-08 02:20:11 +00:00
Martin Stjernholm
58472e8609 Add an option to specify the path to aapt2.
Thin manifests (e.g. master-art) may not have the aapt2 sources and
instead relies on the prebuilt in prebuilts/sdk/tools/linux/bin. In
that case there's no aapt2 in the search path, so it needs to be
specified.

Test: Heavy presubmit build on mainline_modules_bundles on git_master
Test: vendor/google/modules/ArtGoogle/build-art-module.sh
  with http://ag/16584845 on git_master-art, where only the prebuilt
  aapt2 is present
Bug: 212462945
Change-Id: I41a22e8146f5a88534c2721345b4d9d64f76698c
2022-01-07 23:08:53 +00:00
Kelvin Zhang
410bb389d8 Add option to enable zucchini
Change-Id: Id952826c9c5af912fba679af61f2a2e1645641dd
2022-01-07 14:59:01 -08:00
Kelvin Zhang
98ef7bb157 Fix GetCareMap for small images
Change-Id: Iac518e36dbb2fe9303e84d502cd7dfe74621c0af
2022-01-07 14:59:01 -08:00