The apex_util -> deapexer transition still passes signing args in as 1
huge string, just without the double qoutes. Once deapexer receives
the arugment, deapexer would attempt to shlex it, but double
quotes prevents proper shlexing, hence remove the double quotes.
As we added to log to monitor args parser when run apexer from
RepackApexPayload
2024-07-30 12:42:03 - common.py - INFO : Individual args list:
['/home/hieu/aosp/out/host/linux-x86/bin/apexer', '--force',
'--payload_only', '--do_not_check_keyname', '--apexer_tool_path',
'/home/hieu/aosp/out/host/linux-x86/bin:/home/hieu/aosp/out/host/linux-x86/bin:/home/hieu/aosp/out/host/linux-x86/bin:/home/hieu/aosp/prebuilts/clang/host/linux-x86/llvm-binutils-stable:/home/hieu/aosp/prebuilts/asuite/acloud/linux-x86:/home/hieu/aosp/prebuilts/asuite/aidegen/linux-x86:/home/hieu/aosp/prebuilts/asuite/atest/linux-x86:/home/hieu/aosp/prebuilts/jdk/jdk17/linux-x86/bin:/home/hieu/aosp/build/bazel/bin:/home/hieu/aosp/development/scripts:/home/hieu/aosp/prebuilts/devtools/tools:/home/hieu/aosp/prebuilts/misc/linux-x86/dtc:/home/hieu/aosp/prebuilts/misc/linux-x86/libufdt:/home/hieu/aosp/prebuilts/android-emulator/linux-x86_64:/home/hieu/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/lib/wsl/lib:/mnt/c/Program Files/WindowsApps/MicrosoftCorporationII.WindowsSubsystemForLinux_2.2.4.0_x64__8wekyb3d8bbwe:/mnt/c/WINDOWS/system32:/mnt/c/WINDOWS:/mnt/c/WINDOWS/System32/Wbem:/mnt/c/WINDOWS/System32/WindowsPowerShell/v1.0/:/mnt/c/WINDOWS/System32/OpenSSH/:/mnt/c/Program Files/7-Zip:/mnt/c/Users/hnguy140/AppData/Local/Microsoft/WindowsApps:/snap/bin', '--manifest', '/tmp/tmpjk8as91m/apex_manifest.pb', '--build_info', '/tmp/tmpjk8as91m/apex_build_info.pb',
'--key', 'key_dir/avb.pem',
'--signing_args', '--signing_helper_with_files external/fnv/production-sign-tools/production-signing-scripts/IVIMB/signing_helper/avb_signing_helper.py',
'/tmp/tmpji2gvei4', '/tmp/tmpjk8as91m/apex_payload.img', '-v']
As monitor from the above log '--signing_helper_with_files
.../avb_signing_helper.py' is considered as a single argument which make
apexer to be confused. The correct ways of parsing arguments is
'--signing_helper_with_files' and '.../avb_signing_helper' should be
separate to 2 arguments instead merge it as one like current
implementation.
Change-Id: I913b6b3f2ff03d8ce2333b1c2c38178967766778
Signed-off-by: Hieu Nguyen <hnguy140@ford.com>
Previously, META/apex_info.pb contained only /system/apex apexes. Now,
it has all apexes from all possible partitions.
The main purpose of this file is to caculate the decompressed apex size
when applying OTA. Hence it should have all apexes, not just system
apexes.
Bug: 320228659
Test: m dist # check META/apex_info.pb
Change-Id: I3428dc502e4fe3336d1fc5ca941f1fbc332985cd
Adding the entries in apex_info.pb in sorted order makes it easier to
compare ota packages generated between the same source and target on
different computers/environments, ideally the generated ota zips should
be identical.
os.listdir() was used to find the apex files which should be
included in apex_info.pb. listdir() does not guarantee any order,
solution is to sort the result to ensure consistent order.
Bug: 278095305
Test: Manual. Confirm that apex_info.pb is generated sorted
protoc --decode_raw < apex_info.pb
Change-Id: I2851d207f828a6ac31e63e20c9025965ac4b0eba
To support erofs apex extract via deapexer, we need blkid to tell the filesystem type of underlying payload image. If it is ext4, debugfs_static will be used, if it is erofs, then we should use fsck.erofs. Thus we now need explicit blkid input.
Also, remove dependency on fsck.erofs for other deapexer calls. Only extract deapexer call needs blkid and fsck.erofs.
BUG: b/255963179, b/240288941
Change-Id: I8cea0f2def664f9cabf8b14c9a7ecc47bbddfbdd
Signing code uses deapexer to expand payload image, deapexer currently
only supports ext4 payload image expansion using debugfs or
debugfs_static. To support EROFS, need to supply deapexer with
fsck.erofs native binary for EROFS payload expansion.
cherrypick of ag/20154950BUG: b/195515562, b/240288941
Change-Id: Ibf707989f2502bfcf112202a3a6fdb74b6a8dbbc
In the old days, we hacked values of ZIP64LIMIT to get around size
limitations of non-zip64 supported zip files. Now that we switched to
python3 + zip64, there's no point in keeping those hacks.
Test: th
Bug: 255683436
Change-Id: I913db33dad5503736c68a7a1f1321aa952019f60
Test: mma and run sign_apex
This reverts commit 8fe9e3a2c7.
Reason for revert: Updateable SEPolicy is punted.
Change-Id: I28ae2afbb1083e215f4e0a27a9164133cd73dd76
Merged-In: I28ae2afbb1083e215f4e0a27a9164133cd73dd76
apex_utils.SignApex() has `signing_args` argument which is supposed to
be passed to avbtool to set `--signing_helper` or
`--signing_helper_with_files`.
A custom APEX signing tool which relies on avbtool's signing should pass
the same arguments when signing_args is set.
This change passes signing_args verbatim to the custom APEX signing tool
via --signing_args argument.
Bug: 216971834
Test: tbd
Change-Id: I8aaec9440c4536e11e1e4a1dd4ae5f4fe32d4f23
When an APEX specifies its custom signing tool (custom_sign_tool:),
apexkeys.txt contains the info and sign_target_files_apks pass the value
to apex_util.
For now the Virt APEX has its own custom signing tool (sign_virt_apex),
which is added to OTATOOLS.
Bug: 193504286
Test: sign_target_files_apks invokes sign_virt_apex
Change-Id: Iba845723fe3e18f542963324b9c58cd00914c5ba
A new argument is a custom signing tool for APEX contents. When
specified, apex_util invokes the tool with payload's key and payload
directory.
For now, the Virt APEX has its own custom signing tool (sign_virt_apex)
to re-sign filesystem images in it.
Bug: 193504286
Test: atest releasetools_test
Test: m sign_apex sign_virt_apex
Test: sign_apex --sign_tool sign_virt_apex --payload_key ..
--container_key .. resigned.apex
adb install resigned.apex
reboot & vm run-app
Change-Id: Ic4d369c2ba42a8295044a0a75e054dc8def93208
`zipalign` before `signapk -a 4096` is unnecessary step since `signapk
-a 4096` does the same thing.
And, compressed APEX files don't need entries to be aligned.
Bug: 193362313
Test: atest --host releasetools_test
Change-Id: I0758d1162f7c3eb8cc9c73978b5ae5484f007a3a
A new flag tells that the resulting file size should be aligned as well.
For example, signapk -a 4096 --align-file-size ... generates the output
file of 4K-aligned sized.
Bug: 192991318
Test: check built APEX file size.
APEX files should be sized as 4K-aligned.
Change-Id: I1c287e7219b4551bbb3e5957bdb64d80adfd3c39
Uses apex_utils.GetApexInfoFromTargetFiles to find and parse APEX files
in the target files partition dirs. Raises an error on failure to parse
or duplicate package names.
Bug: 177225446
Test: releasetools_test
Test: Create a merged build that provides the VNDK APEX on both vendor
and system. Observe failure.
Change-Id: I1356e263b7b32d6063129e079f3ba7ab4ff132a7
This is needed for partial builds that do not have the apex directory in
their target files package.
Test: build target files for a partial vendor-only build.
Change-Id: I076bfbd1a81cccddcef795f5edeaf2b51538cdec
If an OTA contains compressed APEX inside it, then the device will need
to allocate space on /data partition for their decompression. In order
to calculate how much space the OTA process needs to allocate, the
process needs more information about the APEX contained inside the OTA.
In this CL, we are adding functionality to the OTA generation script
that allows us to gather information about the APEX stored inside the
target-file zip. However, we did not integrate the new functionality
with the ota_from_target_files.py scrip yet. That will be done on follow
up CL.
Bug: 172911822
Test: atest releasetools_py3_test
Change-Id: I2ac42018f628c2c21527b3e086be1f4e7e7247ad
Logic for signing compressed apex will be added in the follow-up cl.
Test: python -m unittest test_sign_apex
Bug: 172912232
Change-Id: I8dc1d334c17e11f9eed8fe0a575b6dfcf337ab5d
When opening an zip file through zipfile.ZipFile(), python2 by default
disables zip64 support. To support update files >4GB, we manually add
allowZip64 to override the setting.
Test: generate && serve an OTA
Change-Id: I9645e963ced830cc2d3a4b72bc63b9369a1cefe8
When signing APks on build server, there might not be an android repo.
Usually deapexer.py will be run from otatools.zip after exctration. So
we should look for debugfs_static in search path
Bug: 165347497
Test: Use sign_apk_from_target_files without android repo
Change-Id: Ifaf91764ca0dc08d010f43a24bfe975a07a71e7f
When signing APks on build server, there might not be an android repo.
Usually deapexer.py will be run from otatools.zip after exctration. So
we should look for debugfs_static in search path
Bug: 165347497
Test: Sign using self-service
Change-Id: I608870b0184bd8f5f07afa53355b39de47a97e1c
When sign_apex.py re-signs an apex, the hash algorithm is extracted and
it is used when re-packaging the file.
Bug: 155771970
Test: releasetools_py3_test doesn't regress
(some tests were failed, but they failed even without this change:
test_ApexApkSigner_apkKeyNotPresent (test_apex_utils.ApexUtilsTest) ... ERROR
test_ApexApkSigner_noApkPresent (test_apex_utils.ApexUtilsTest) ... ERROR
test_ApexApkSigner_noAssetDir (test_apex_utils.ApexUtilsTest) ... ERROR
test_ApexApkSigner_signApk (test_apex_utils.ApexUtilsTest) ... ERROR)
Merged-In: I996a33e0208d3dd3a04a31b8dafb27ef995297f2
Change-Id: I996a33e0208d3dd3a04a31b8dafb27ef995297f2
(cherry picked from commit a1887f326c)
When sign_apex.py re-signs an apex, the hash algorithm is extracted and
it is used when re-packaging the file.
Bug: 155771970
Test: releasetools_py3_test doesn't regress
(some tests were failed, but they failed even without this change:
test_ApexApkSigner_apkKeyNotPresent (test_apex_utils.ApexUtilsTest) ... ERROR
test_ApexApkSigner_noApkPresent (test_apex_utils.ApexUtilsTest) ... ERROR
test_ApexApkSigner_noAssetDir (test_apex_utils.ApexUtilsTest) ... ERROR
test_ApexApkSigner_signApk (test_apex_utils.ApexUtilsTest) ... ERROR)
Change-Id: I996a33e0208d3dd3a04a31b8dafb27ef995297f2
When doing apk-in-apex signing, the package name is not reserved
during the apex repacking. As a result, the name accidentally
reverts to 'com.android.wifi' from 'com.google.android.wifi'.
This cl changes the behavior to call 'apexer' by passing the
'--payload_only' argument. So we don't build the apex file from
scratch and the old AndroidManifest.xml will be reused.
BUG: 152084536
Test: unit tests pass
Change-Id: I8332b2ee84832fb196f2e1c4309abac5ab92e153
Merged-In: I8332b2ee84832fb196f2e1c4309abac5ab92e153
When doing apk-in-apex signing, the package name is not reserved
during the apex repacking. As a result, the name accidentally
reverts to 'com.android.wifi' from 'com.google.android.wifi'.
This cl changes the behavior to call 'apexer' by passing the
'--payload_only' argument. So we don't build the apex file from
scratch and the old AndroidManifest.xml will be reused.
Test: 152084536
Bug: unit tests pass
Change-Id: I8332b2ee84832fb196f2e1c4309abac5ab92e153
We should add quote when passing the signing arguments to apexer. So
the final argument whould look like
apexer ... --signing_args "--signing_helper_with_files=%path"
And the argument parser in avbtool will eventually parse the
"--signing_helper_with_files" correctly.
Bug: 148627666
Test: unittest pass, run a smoke sign_target_files_apk with signing arguments
Change-Id: Ie203d26a508cb98ca94e6c364a2640b681ad79f2
In the build signing environment, we don't necessary have access to the
physical private key. And we work around it by passing an extra
signing_helper to the avbtool instead of a private key. Since we need
to perform signing to add the hashtree footer when rebuilding the apex
image. We should propogate the signing_helper to apexer and eventually
to avbtool.
Bug: 148627666
Test: unit tests pass. Smoke test by pass a public key to apexer and
hardcode the private key in signing helper script.
Change-Id: I56d57485493a5ba2cbe4c49e493ee9604f68088b
We should only provide the --assets_dir option when the assets directory
is available in the original apex.
Bug: 148452817
Test: unit tests pass
Change-Id: I44308b80a43ff3f441223f0ecc248991d1d83f31
So when $ANDROID_BUILD_TOP is missing, the error message print out the
path of the missing file; instead of a type error.
Bug: 148452817
Test: check result with missing ANDROID_BUILD_TOP
Change-Id: I5f841f6626eb3843364f20e78b278d3244505396
Some apex payload images contain apk files. And these apks need to be
signed during the signing processed when sign_target_files_apks is
called. To support the signing, we can extract the payload and repack
the apex file with the (de)apexer tool. Add the signing support in the
apex_util.
Bug: 146508800
Test: unit tests pass, run sign_apex, sign_target_files_apks
Change-Id: If6d58975248709a144b07dbabf47c27916e5695e
Bug: 139957269
Test: Sign a target_files.zip. Extract a re-signed APEX and check the
hashtree size (being zero).
Test: Use sign_apex to sign an APEX file. Check the hashtree size (not
being zero).
Test: python -m unittest test_apex_utils
Test: python -m unittest test_sign_apex
Change-Id: I927b7681d66920d7732b700ec3a8f7a65b4cb351
The avb_pubkey may not be present, if the apex was initially
unsigned or generated from a bundle. In this case, running
sign_apex to generate a signed apex binary would result in an error.
This fix checks for presense of avbpubkey before attempting the
deletion
BUG: 139994107
Change-Id: I3cb2e88a11ad8797e38ba5fb98c96a4ec4135fc8
