Previously, META/apex_info.pb contained only /system/apex apexes. Now,
it has all apexes from all possible partitions.
The main purpose of this file is to caculate the decompressed apex size
when applying OTA. Hence it should have all apexes, not just system
apexes.
Bug: 320228659
Test: m dist # check META/apex_info.pb
Change-Id: I3428dc502e4fe3336d1fc5ca941f1fbc332985cd
Checking ro.build.date.utc to detect a downgrade is not enough in the
case where several target files are merged, as timestamps can differ
for each partition.
The solution is to check the build timestamp for each partition and
ro.build.date.utc.
With this change ota generation will be aborted during creation
instead of rejected when installing on device.
This also fixes the --override_timestamp so it is possible to generate
a package where timestamps are reversed, as was added in:
3e6161a3b3.
Issue: 315913966
Test: Manual, confirm that ota generation is aborted when timestamp
on post vendor partitions is newer than on pre.
Ota created successfully with --override_timestamp parameter and
installation successful
Test: atest --host releasetools_test
Change-Id: I275e67a3840f4ef2263381c253231068e72f47d2
Move some depency on global OPTIONS to parameters. This makes it easier
for other modules to re-use these functions w/o having to modify OPTIONS
variables before/after calling.
Test: th
Bug: 227536004
Change-Id: I3bab292f65a4222f0c9502bcf04fa7a85f8124c6
There's also a Payload class in system/update_engine/scripts. To avoid
name collision, rename the one in releasetools to PayloadGenerator.
No functional changes in this CL.
Bug: 227848550
Test: th
Change-Id: Ib7d4c7ad9839d99416d965e3a3661b8cee7b7693
This makes it easier for other otatools to re-use these logic without
having to pull in lots of dependencies.
Test: th
Bug: 227848550
Change-Id: I81ed01c5cea4b934a074650731b6f89752221de9
So updaters can streaming download the file, and query the apex info
inside the file.
Bug: 190244686
Test: generate an OTA package, check the streaming property
Change-Id: I17078d3f8d60ca53c6afe82f74b232e2fb242467
We have already logged the compressed apexes in the target-files.
Because we want to support the apex metrics during OTA update, also
include the uncompressed apexes in the META/apex_info.pb.
For incremental OTA packages, include the source apex version for
each apex package as well.
Bug: 190244686
Test: unit test
Change-Id: I5cf2647c56c4feb5517f9a81aa1e9abc52515bf1
Uses apex_utils.GetApexInfoFromTargetFiles to find and parse APEX files
in the target files partition dirs. Raises an error on failure to parse
or duplicate package names.
Bug: 177225446
Test: releasetools_test
Test: Create a merged build that provides the VNDK APEX on both vendor
and system. Observe failure.
Change-Id: I1356e263b7b32d6063129e079f3ba7ab4ff132a7
In aosp/1581143 , we banned generation of SPL downgrade OTAs. However,
caller of OTA script can still force an SPL downgrade OTA by passing
--spl_downgrade flag. If this flag is specified, we propagate it to OTA
metadata so that GOTA server can properly honor this flag.
Test: th
Change-Id: Ic8cdc850d2210f4149ad9121fa4ed2e5a4f59bcc
This is needed for partial builds that do not have the apex directory in
their target files package.
Test: build target files for a partial vendor-only build.
Change-Id: I076bfbd1a81cccddcef795f5edeaf2b51538cdec
If an OTA contains compressed APEX inside it, then the device will need
to allocate space on /data partition for their decompression. In order
to calculate how much space the OTA process needs to allocate, the
process needs more information about the APEX contained inside the OTA.
In this CL, we are adding functionality to the OTA generation script
that allows us to gather information about the APEX stored inside the
target-file zip. However, we did not integrate the new functionality
with the ota_from_target_files.py scrip yet. That will be done on follow
up CL.
Bug: 172911822
Test: atest releasetools_py3_test
Change-Id: I2ac42018f628c2c21527b3e086be1f4e7e7247ad
Add a new custom_image option to configure which custom images to use to
update custom partitions in A/B update.
This change also moves oem_settings to common option as A/B update will
use it to set oem properties too.
BUG: 171225290
Test: unittest pass, generate OTAs, flash to devices and check results
Change-Id: I279477d6b2954fb3705d7efede0a8bcd330c108b
In aosp/1459042, init has updated it's logic to use
ro.build.version.release_or_codename for fingerprint calculation.
The ota script needs the same update for ota targeting to work
correctly.
As a best effort to support pre-S dev stage builds, use
ro.build.version.release for sdk version < 30.
Bug: 170968068
Bug: 158483506
Test: unit tests, generate an OTA for S build
Change-Id: I01ca8a3b7b8b58f94b10f90ed0d27e688a72b866
Allow the ota generation script to take a list of partitions;
and thus generating a partial ota update package. This allow the
testing of partial update functionality, e.g. system-only updates.
Bug: 170921953
Test: unit tests, generate and apply a partial update on coral with system, system_ext, vbmeta_system
Change-Id: I0d8f93806dfbf7b781ea911117b02fc5c0971434
When opening an zip file through zipfile.ZipFile(), python2 by default
disables zip64 support. To support update files >4GB, we manually add
allowZip64 to override the setting.
Test: generate && serve an OTA
Change-Id: I9645e963ced830cc2d3a4b72bc63b9369a1cefe8
As part of the ota metadata work to support partial update, we should
add the per-partition device name, fingerprint, and version to the ota
metadata.
Bug: 151088567
Bug: 166154730
Test: unit tests pass, generate a package
Change-Id: I7a71a4860aad651daf9e62df39c60d72727c14ff
Background in http://go/android-partial-updates. For partial update
(e.g. system-only) or devices with mixed build, the current
fingerprint & device name no longer suffice as the precondition to
install the package.
Therefore, we need to additionally include the per-partition build
props into the ota metadata. We also define a protobuf for the metadata
so it can be extended later. The metadata of the legacy format is also
kept for backward compatibility.
Bug: 151088567
Test: unittest pass, generate an OTA and check the result
Change-Id: I716f7da54a393cd340280dbddc3c92b3460f8ef8
The values of the ro.boot* variables are not part of the image files
and are provided (e.g. by bootloaders) at runtime. Meanwhile, their
values may affect some of the device build properties, as a different
build.prop file can be imported by init during runtime.
This cl adds an option to accepts a list of possible values for some
boot variables. The OTA generation script later use these values to
calculate the alternative runtime fingerprints of the device; and
list the device names and fingerprints in the OTA package's metadata.
The OTA metadata is verified by the OTA server or recovery to ensure
the correct OTA package is used for update. We haven't made any
restrictions on what ro.boot* variables can be used for fingerprint
override. One possible candidate can be the skus listed in
ODM_MANIFEST_SKUS.
Bug: 152167826
Test: unittests pass, generate an OTA file with the new option
Change-Id: I637dea3472354236d2fd1ef0a3306712b3283c29
This reverts commit b21e48b499.
In practice, some partners use the 'import' statement to override
the device fingerprint at runtime. The runtime fingerprint will
later add to the metadata of OTA package, so that the OTA server
can deliver the package to corresponding devices correctly.
This CL supports parsing a subset of import statement that the init
process recognizes. And we loose the restriction based on how the
dynamic fingerprint is used in practice. Right now, we only searches
for the override of brand, name and device. And the placeholder
format should be ${placeholder}, with its value supplied by the
script caller.
As part of the implementation, we generate all the possible
combinations of the input boot variables. And recalculate the
fingerprint for each of the combination. Though we load the
build.prop multiple times, the logic is easier to follow. Also,
it's more convenient to enhance the logic if we only want to
allow some of the boot variables combination later.
Bug: 152167826
Change-Id: I4a9fa35c7ac037ff1cf4f9a4bdff602beac3894b
Test: unittests pass
This reverts commit 6022545272.
The build prop for a partition used to be a simple key:value
dictionary. But we need more fields to hold the alternative build
props overriden by the 'import' statement. Therefore, add a new
class as a wrapper for these props first.
Bug: 152167826
Change-Id: I5c952cd2a976ba1a09ddc66d56c2b8b55a61986b
Test: unittests pass
In practice, some partners use the 'import' statement to override
the device fingerprint at runtime. The runtime fingerprint will
later add to the metadata of OTA package, so that the OTA server
can deliver the package to corresponding devices correctly.
This CL supports parsing a subset of import statement that the init
process recognizes. And we loose the restriction based on how the
dynamic fingerprint is used in practice. Right now, we only searches
for the override of brand, name and device. And the placeholder
format should be ${placeholder}, with its value supplied by the
script caller.
As part of the implementation, we generate all the possible
combinations of the input boot variables. And recalculate the
fingerprint for each of the combination. Though we load the
build.prop multiple times, the logic is easier to follow. Also,
it's more convenient to enhance the logic if we only want to
allow some of the boot variables combination later.
Bug: 152167826
Test: unittests pass
Change-Id: Iac093a40dc6f873c5e1858efa44cb2bd6082508a
The build prop for a partition used to be a simple key:value
dictionary. But we need more fields to hold the alternative build
props overriden by the 'import' statement. Therefore, add a new
class as a wrapper for these props first.
Bug: 152167826
Test: unittests pass
Change-Id: I2fe7e93a2f4de8e55f5f8051b000b96b5efdc85a
The signature size is needed during payload hashing and signing.
We used to sign the update_engine's payload with RSA keys only. In
this case, the signature size always equals the key size. But the
assumption is no longer true for EC keys, whose DER-encoded signature
size is a variant with a maximum size.
Therefore, we always give the maximum signature size to the delta
generator, who then add paddings to the real signature if necessary.
The maximum signature size is calculated by calling the delta_generator
with the new option '--calculate_signature_size'. For custom payload
signers, we also deprecate the '--payload_signer_key_size' and replace
it with '--payload_signer_maximum_signature_size'.
The EC key in the test is generated with:
The EC key in the unittest is generated with the command:
openssl ecparam -name prime256v1 -genkey -noout -out prime256v1-key.pem
openssl pkey -in prime256v1-key.pem -out testkey_EC.key
Bug: 141244025
Test: sign and verify a payload
Change-Id: Ife6e269d8aa3d870405aca20086330f1795e167f
There is no change to module functionalities. Testcases are moved around
accordingly.
Bug: 134525174
Test: TreeHugger
Test: lunch a target; atest --host releasetools_test releasetools_py3_test
Change-Id: I7bc8f49cc239e7c6655fe5e375508f01c1743b94
This CL additionally removes boot/dtbo/modem/vbmeta_* images out of
secondary payload. We essentially only keep system_other.img and
bootloader images there.
For Pixel devices, this additionally saves ~80MiB (mostly because of
the removal of boot and radio images).
Bug: 140771390
Test: ota_from_target_files \
--include_secondary --skip_postinstall \
-i input-target_files-1.zip \
input-target_files-2.zip \
output-ota.zip
Test: python -m unittest test_ota_from_target_files
Change-Id: If47b27c52b3547a4cc86223a988c53960bc6af40
We used to carry the full OTA for vendor and product partitions when
generating the secondary payload. Since they are not actually used, we
can remove them to reduce the size of factory OTA packages. For example,
the size for a sargo package reduces from 454M to 283M after the change.
Bug: 140771390
Test: generate and apply a second payload on Pixel
Change-Id: I091dc07db8c345b182d9f5dd8d48f5c209245641
They used to be disabled due to the assertion of search_path in setUp()
function, which is not a prerequisite for most of the tests.
Bug: 112080715
Test: `atest releasetools_test`
Test: TreeHugger
Change-Id: I3cbaf42aa09dba0b87a64e11d97de9b3f7af7a47
About half of the testcases rely on external tools (i.e. the ones in
`otatools.zip`, which are external to releasetools module, but still
built by Android). It's WAI as releasetools scripts are mostly for
gluing purpose.
However, the current support in Soong doesn't allow packing the helper
modules as part of the built releasetools_test. This CL adds a decorator
that allows declaring external dependencies in testcases, which will be
skipped while running in presubmit. It doesn't affect local invocation
of `atest releasetools_test`.
Fixes: 112080715
Test: `atest releasetools_test`
Test: TreeHugger; check that releasetools_test is invoked (and test
passes).
Change-Id: I8fdeb6549023cf5ddeb79d610c7c37cf9f13d3cc
All the unittests will be built into releasetools_test. One can run the
tests with `atest releasetools_test` or the traditional way
`test_utils.py`. The atest way is recommended, which additionally builds
the required tools.
With the current support in Soong, we can't pack the built tools into
releasetools_test yet. So running `releasetools_test` alone in clound
would fail. Follow-up CLs will address the issue in order to deploy the
tests with TEST_MAPPING.
Bug: 112080715
Test: `atest releasetools_test`
Change-Id: Ica95517a5ab326f4e58fc57c6c2c276cfe882f3c
The signature size will be 512 bytes when signing the payload
with 4096 bits key. This cl determines the key size with
"openssl rsa -modulus"
The new key in testdata is generated by
"openssl genrsa -out testkey 4096"
Bug: 129163830
Test: generate and verify an OTA package
Change-Id: I6662b0a0c553dc0fd84711312a1256b887e332fd
common.RunAndCheckOutput() checks the exit code and will raise on errors.
Test: python -m unittest test_ota_from_target_files
Test: Call ota_from_target_files.py with aosp_marlin target_files zip
Test: Inject an error to the command. Repeat the call without verbose
flag. It dumps the stack trace on error.
Change-Id: I85b765a33b9087bcbcb0571d6e632a07bb86c65c
All the releasetools unittests extend the common base class of
test_utils.ReleaseToolsTestCase. Define tearDown() in the base class to
do the clean-up works.
Test: `pylint --rcfile=pylintrc test_*.py`
Test: `python -m unittest discover .`
Change-Id: I51775d964ef032dcdf3bb89c55e1a31371cde708
stdout and stderr will default to subprocess.PIPE and subprocess.STDOUT
respectively (which is the expected behavior from most of the existing
callers), unless caller specifies any of them.
Test: `m dist`
Test: python -m unittest \
test_common \
test_add_img_to_target_files \
test_ota_from_target_files \
test_validate_target_files
Change-Id: I43b3f08edfa8a9bcfe54baf9848dc705c048e327
The new suffix distinguishes the new care_map from the ones in plain
text format; and thus the old update_verifier won't report an error
upon parsing failures.
Bug: 115740187
Test: Generate OTA files for Pixels
Change-Id: Ia782afd8cbb0f4bb8c363edaa00e92ab302d5d1b
Commit d572632f3d added support in build
system to blacklist given vendor properties. When
ro.vendor.build.fingerprint gets blacklisted, the OTA generation script
can't rely on those properties to determine a change in vendor images.
This CL considers such a case as "vendor images must have changed"
between the two builds.
Bug: 113892939
Test: Generate an incremental package with builds not having
ro.vendor.build.fingerprint.
Test: python -m unittest test_ota_from_target_files
Change-Id: I188de9c3cbeecf26132c92b9356e9d5fef75205e
The change in [1] removed duplicating radio images from RADIO/ to
IMAGES/. When getting the target_files zip for generating secondary
payload, we need to additionally copy the files under RADIO/ over.
[1] commit 5277d1015f
Test: python -m unittest test_ota_from_target_files
Test: Generate an OTA with secondary payload (--include_secondary).
Change-Id: I096f1642a905fb764e63f5df2edc1396aa6befd8
BuildInfo already has dict-like getters. This CL adds the matching
setter method to keep it backward compatible with device-specific
scripts that expect a dict.
It also adds items() method that allows iterating the items.
Bug: 111087332
Test: python -m unittest test_ota_from_target_files.BuildInfoTest
Test: Generate an incremental OTA that sets info_dict in device-specific
releasetools script.
Change-Id: Idd033f98a9186740f9da1a300d4c2bdddd9c1345
We used to write a fake post-timestamp into package metadata for
"--override_timestamp", which allows a package to be pushed based on
the fake timestamp. This CL stops doing that by using the designated
"ota-downgrade=yes" instead, but keeping the "--override_timestamp" flag
for backward compatibility.
Now both "--override_timestamp" and "--downgrade" allow pushing packages
with downgrade timestamps. The only differenece is that "--downgrade"
enforces a data wiping, because it's intended a real downgrade (based on
Android version or security patch level). "--override_timestamp" serves
the path of a legit "upgrade", but unfortunately with build timestamps
in reverse order (e.g. the two builds are cut from different branches).
With this CL, we write "post-timestamp" to all packages.
Bug: 77811049
Test: Generate an incremental pakcage with "--override_timestamp".
Test: Generate an incremental pakcage with "--downgrade".
Test: python -m unittest test_ota_from_target_files
Change-Id: I4ae4ce77cb600a06ffd9b15b6d4c730314aa0b79
ota_from_target_files.py:
Rename PropertyFiles._GetPropertyFilesString to
PropertyFiles.GetPropertyFilesString.
Currently only PropertyFiles.Compute and
PropertyFiles.Finalize gives access to _GetPropertyFilesString.
But Compute force sets reserve_space=True,
and Finalize requires reserved_length.
_GetPropertyFilesString is useful method that can
be used outside releasetools.
One of the usage is in bootable/recovery/updater_sample/.
Test: python -m unittest test_ota_from_target_files
Change-Id: I2cc44ec46a0e68ba071531b003af8cdbfe90e588
Signed-off-by: Zhomart Mukhamejanov <zhomart@google.com>
