When lz4diff is enabled, we inject the host copy of liblz4.so in
target_file to delta_generator. This is to ensure that host tooling can
produce same compressed output as the device.
Test: th
Bug: 206729162
Change-Id: I2d8206f7ec54fadedea16bf3d811b8353bc9414d
Bug: 68860345
Bug: 151177513
Bug: 151953481
Test: m all
Test: m systemlicense
Test: m listshare; out/soong/host/linux-x86/bin/listshare ...
Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ...
Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ...
Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ...
where ... is the path to the .meta_lic file for the system image. In my
case if
$ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD)
... can be expressed as:
${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic
Change-Id: I17221ec8fe429b5f745c0733f8bb1ab0ea2f4567
Defer edge creation.
Don't create edges until the count is known to avoid repeated allocate+
copy operatios.
Limit resolutions.
Allow only a single resolution condition set per target, and overwrite
intermediate results. Reduces memory and obviates allocations.
Propagate fewer conditions.
Instead of propagating notice conditions to parents in graph during
initial resolve, leave them on leaf node, and attach to ancestors in
the final walk. Reduces copies.
Parallelize resolutions.
Use goroutines, mutexes, and waitgroups to resolve branches of the
graph in parallel. Makes better use of available cores.
Don't accumulate resolutions inside non-containers.
During the final resolution walk, only attach actions to ancestors from
the root down until the 1st non-aggregate. Prevents an explosion of
copies in the lower levels of the graph.
Drop origin for scale.
Tracking the origin of every potential origin for every restricted
condition does not scale. By dropping origin, propagating from top
to bottom can prune many redundant paths avoiding an exponential
explosion.
Conditions as bitmask.
Use bit masks for license conditions and condition sets. Reduces maps
and allocations.
Bug: 68860345
Bug: 151177513
Bug: 151953481
Test: m all
Test: m systemlicense
Test: m listshare; out/soong/host/linux-x86/bin/listshare ...
Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ...
Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ...
Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ...
where ... is the path to the .meta_lic file for the system image. In my
case if
$ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD)
... can be expressed as:
${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic
Change-Id: Ia2ec1b818de6122c239fbd0824754f1d65daffd3
Using struct{}{} as the payload for set maps reduces memory use for
large sets.
Bug: 68860345
Bug: 151177513
Bug: 151953481
Test: m all
Test: m systemlicense
Test: m listshare; out/soong/host/linux-x86/bin/listshare ...
Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ...
Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ...
Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ...
where ... is the path to the .meta_lic file for the system image. In my
case if
$ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD)
... can be expressed as:
${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic
Change-Id: Ibc831ae80fc50f35e1000348fb28fc0167d0ebed
Use annotation slices looked up from map instead of read from disk to
reduce copies.
Bug: 68860345
Bug: 151177513
Bug: 151953481
Test: m all
Test: m systemlicense
Test: m listshare; out/soong/host/linux-x86/bin/listshare ...
Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ...
Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ...
Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ...
where ... is the path to the .meta_lic file for the system image. In my
case if
$ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD)
... can be expressed as:
${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic
Change-Id: Ibf6f7d319092e0d54ab451ffbbd35dcd7d0080e0
sign_target_files_apks demands it.
Bug: 213823227
Test: run com.google.android.tradefed.ota.SigningToolTest#Testcase1_Signingtest
Change-Id: I1bba2b25a71449a57d284024e77fb76d1b79a0ee
Thin manifests (e.g. master-art) may not have the aapt2 sources and
instead relies on the prebuilt in prebuilts/sdk/tools/linux/bin. In
that case there's no aapt2 in the search path, so it needs to be
specified.
Test: Heavy presubmit build on mainline_modules_bundles on git_master
Test: vendor/google/modules/ArtGoogle/build-art-module.sh
with http://ag/16584845 on git_master-art, where only the prebuilt
aapt2 is present
Bug: 212462945
Change-Id: I41a22e8146f5a88534c2721345b4d9d64f76698c
Normally this has no effect, but when we generate metadata for small
files (<=4KB), merkle tree isn't generated. In such case, writing zero
will make the metadata format simpler and unconditional.
Test: manual
Change-Id: Ibe18175b580af3409c896a8bb97323792ad9c459
build_image.py has been handling fsverity metadata generation in the
packing step, but it can cause issues because the metadata files are
missing in the $OUT directory, and they only exist in result system.img.
This change moves the generation logic into Makefile, and makes the
metadata tracked by ninja graph.
Bug: 206326351
Test: PRODUCT_SYSTEM_FSVERITY_GENERATE_METADATA := true and build
Change-Id: I1f910d8ac6e2cc3c54f35916871733c632f18e44
If init_boot.img is present, the GKI boot-*.img should
not include the ramdisk because the ramdisk will be packed
into the init_boot.img instead.
The "has_ramdisk" flag incorrectly checks the condition of:
prebuilt_name != "boot.img" to see if it is a boot.img.
Because "has_ramdisk" was added before we packed multiple
GKI boot-*.img, e.g., boot-5.10.img, boot-5.10-lz4.img, etc.,
into the target files.
Fix this by checking the partition_name is "boot" or not.
Also moving the logic into a new function with comments for each
condition.
Bug: 203698939
Bug: 213028932
Test: sign_target_files_apks \
--gki_signing_key=external/avb/test/data/testkey_rsa4096.pem \
--gki_signing_algorithm=SHA256_RSA4096 \
./out/dist/*-target_files-eng.*.zip signed.zip, then
unpack_bootimg to checks the signed boot-*.img has no ramdisk
Change-Id: I5354669feb54d547dbe797e51b1b1baa187cb7cf
* Now with 4 sections that can be shown/hidden independently.
* After the 'selected_projects' section, add two more sections:
* 'top_directory_section' contains
* a table of directories with at least 1% warnings.
* a 'selected_directory_warnings' subsection to show all warnings
in the clicked directory from the directory table
* 'top_file_section' contains
* a table of files with at least 100 or 1% warnings.
* a 'selected_file_warnings' subsection to show all warnings
in the clicked file from the file table
* Adjust button/section spaces and reduce button font size to 100%.
* Rename drawTable to genTables.
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: I765b09a46adc111cfe781719ba7aa0f917aa6ffc
Equivalent to PRODUCT_EXTRA_RECOVERY_KEYS but for A/B OTA.
Bug: 211848136
Test: set PRODUCT_EXTRA_OTA_KEYS and check otacerts.zip
Change-Id: I81e27d12a22b405f6227b09c01ed684dfcede19e
Currently when running sign_target_files_apks on a no-system-image
target, it will raise the following error:
ValueError: max() arg is an empty sequence
This is because there is no APK files in the target_files.zip.
Fixing this by setting maxsize to zero in this case.
Bug: 213028932
Test: lunch gki_arm64-userdebug; make dist
Test: sign_target_files_apks \
--gki_signing_key=external/avb/test/data/testkey_rsa4096.pem \
--gki_signing_algorithm=SHA256_RSA4096 \
--gki_signing_extra_args="--prop gki:prop1 --prop gki:prop2" \
./out/dist/*-target_files-eng.*.zip signed.zip
Change-Id: I40daecbc2ff3f89d3e635d1a4a1c1dea31ba9a27
Test: Add a VENDOR_BLOBS_LICENSE entry to an existing product, then `m with-license`
Bug: 203436762
Change-Id: I4bbe77bda7789b6c44bea141518b1cd2e699d326
For not virtApex, this results in a KeyError.
line 151, in <module>\n', ' File
"/usr/local/google/home/baligh/clients/goog/master/out/host/linux-x86/bin/sign_apex/sign_apex.py",
line 144, in main\n', "KeyError: 'sign_tool'\n"]
BUG: 193504286
Test: TH
Change-Id: Id982e5c57086ada78168163d2293813df121847d
Avoid writing test files in the build output directory which fails when
run with Bazel. This happens because Bazel's sandboxing environment
ensures that the test's working directory is unwritable.
See https://docs.bazel.build/versions/main/sandboxing.html for more
information.
Bug: 209687942
Test: atest --bazel-mode zipalign_tests
Test: atest zipalign_tests
Change-Id: Ie22f464830c1ffe4d38a94a16dbd39dafa7fe317
This new init_boot.img contains the ramdisk that used to reside in the
boot.img file.
Test: set BOARD_PREBUILT_INIT_BOOT_IMAGE to an external init_boot.img
- Check that "m" pulls in the init_boot.img to
out/target/product/vsoc_x86_64/
- Check that "m dist" adds the init_boot.img to
aosp_cf_x86_64_phone-img-eng.devinmoore.zip
Test: atest --host releasetools_test
Bug: 203698939
Change-Id: If7ef2cf093e5e525529c7c44333c0f40f6ba0764
When we have a PEM key, we don't need the process converting a DER key
to PEM format, but we just need to use the PEM key as-is.
Bug: 205987437
Test: build and manual test
Change-Id: I6f61a9088efc0f7193737d3c33b8cfde399b2b6f
Making this a host tool will help users generate their own fsverity
metadata easily.
Bug: 205987437
Test: m fsverity_metadata_generator and run it
Change-Id: Iafd228815a74d298d87ca1466c6909c0d24c5874
Performance optimization means not every path will be traversed.
Instead of updating parents via the path, perform a 2nd bottom-up walk
after the top-down walk to propagate the new resolutions to parents.
Note: the 2nd walk method will add resolutions to statically linked
libraries etc. at deeper levels, but those do not affect what gets
reported. In particular, note that test data for dumpresolutions
changes, but none of the test data for listshare, checkshare etc.
changes.
Test: m all systemlicense listshare checkshare dumpgraph dumpresolutions
Bug: 68860345
Bug: 151177513
Bug: 151953481
Change-Id: I76361c4e33bbadbbea38cbec260430e8f9407628
Tune the top-down walk to avoid needlessly walking the same subtree
over and over again with the same condition(s).
Takes walking system image down from 3m to 1.5s.
Test: m all systemlicense listshare checkshare dumpgraph dumpresolutions
Bug: 68860345
Bug: 151177513
Bug: 151953481
Change-Id: I4354800cd8dfc42efd4df274d2ce45eaa3e0a99f
Bug: 68860345
Bug: 151177513
Bug: 151953481
Test: m all
Test: m systemlicense
Test: m listshare; out/soong/host/linux-x86/bin/listshare ...
Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ...
Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ...
Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ...
where ... is the path to the .meta_lic file for the system image. In my
case if
$ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD)
... can be expressed as:
${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic
Change-Id: I5d48eababce7bba39795d3668eee86b332cbe43d
package to read, consume, and analyze license metadata and dependency
graph.
Includes the below command-line tools:
listshare outputs csv of projects to share to meet restricted and
reciprocal license requirements with one project per line. The first
field is the path to the project, and subsequent fields identify the
license resolutions as colon-separated target:annotations tuples.
checkshare outputs error messages to stderr for any targets where
policy dictates both sharing and not sharing the source-code, and PASS
or FAIL to stdout. exit status indicates success 0 or conflict found 1
Bug: 68860345
Bug: 151177513
Bug: 151953481
Test: m all
Test: m systemlicense
Test: m listshare; out/soong/host/linux-x86/bin/listshare ...
Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ...
Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ...
Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ...
where ... is the path to the .meta_lic file for the system image. In my
case if
$ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD)
... can be expressed as:
${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic
Change-Id: I4ff3f98848f7e6d03a35734300d763ef5f245d53
package to read, consume, and analyze license metadata and dependency
graph.
Includes the below command-line tool:
dumpresolutions outputs the resulting set of resolutions after the
bottom-up and top-down resolves, or after joining 1 or more condition
walks.
Bug: 68860345
Bug: 151177513
Bug: 151953481
Test: m all
Test: m systemlicense
Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ...
Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ...
where ... is the path to the .meta_lic file for the system image. In my
case if
$ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD)
... can be expressed as:
${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic
Change-Id: I9869400126cd7ad4b7376b0bab31b46aad732f5d
package to read, consume, and analyze license metadata and dependency
graph.
Includes testdata/ and the the below command-line tool:
dumpgraph outputs edges of the graph as "target dependency annotations"
Bug: 68860345
Bug: 151177513
Bug: 151953481
Test: m all
Test: m systemlicense
Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ...
where ... is the path to the .meta_lic file for the system image. In my
case if
$ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD)
... can be expressed as:
${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic
Change-Id: I5fe57d361da5155dbcb2c0d369626e9200c9d664
