This property is not device-generic, as it's used for things like
whether the product is phone/tablet/other and whether it has an
sdcard or not.
Bug: 118618261
Test: adb shell getprop ro.build.characteristics for aosp_blueline
Change-Id: I6bbf1d4c36d7f37f92c67bebde58d72aba9d0852
Regression introduced by c3d408e3c1
Found a case where 1 + 1/59 reduction in size based on free space
failed when right sizing an ext4 image. Reduced the math to 1 + 1/60.
Test: compile
Bug: 111302946
Change-Id: I6a0f3919e8d45d0b3e7cd558a6ad4d3799012f2c
Add ext4 dedupe to system_other, product, product_services, odm and
oem images. Experimental savings without any other configuration
changes was 0%, .75%, 3.4% respectively for the raw image file size
for the first three.
Test: manual
Bug: 111302946
Change-Id: Ia8fb5696151acad59bb144ea93f2c2ddac962bbd
To more quickly settle on a no free space result, recognize that
there is roughly 1/58.5 overhead managing the free space pool as
determined experimentally. This algorithm carries with it some
risk of being sensitive to any alterations in ext4.
This also addresses the issue of a much larger apparent free space
available calculated on the sample pass when deploying ext4 dedupe,
and resolves the poorer estimation that happens without accounting
for the overhead.
The alternative of adding a second pass works to a similar, and
likely more comforting algorithm and result. But doing so adds
a minute to the typical incremental build time, or +50% to the
time it takes to determine ext4 right sized images.
Test: manual
Bug: 111302946
Change-Id: Id5ae6645b9aa2d036e6fefe2fb17672e8f8ef6f0
This avoids showing a message as below:
raise BuildImageError("Failed to get tune2fs usage:\n{}".format(output))
UnboundLocalError: local variable 'output' referenced before assignment
The `output` won't be set under such a case, which unfortunately hides
the real cause. This CL addresses the issue by letting it throw
implicitly, which contains the command and outputs.
Test: `m dist`
Test: `python -m unittest test_build_image`
Change-Id: Icf015bab8869d150516246ed73552f6502127cdf
If partition_reserved_size is 0 or undefined, and
use_dynamic_partition_size is true, we should approach no space
and no free inodes automatically.
Estimate the space and number of inodes required, then do a first
pass build to see how much space actually used, and use those values
to refine the estimate.
Depends on tune2fs to report the characteristics of the filesystem,
so only support for ext filesystems. In the future if there has to
be a more generic ability, either a tool per a filesystem has to be
found, or we will need root capabilities to mount the filesystem to
acquire the characteristics live from the host system.
Test: manual + python -m unittest test_build_image
Bug: 111302946
Change-Id: I933a388be43516b6de7b5007b296765bd5556fde
Chained vbmeta images should be included into the top-level vbmeta.img
as chained partitions. It's done in the path of `m`, but not `m dist`.
Bug: 118115607
Test: `m dist` a target that uses chained vbmeta.
a) Check that the `vbmeta.img` in target-img.zip contains the image
descriptor of `vbmeta_system.img` (`avbtool info_image`).
b) `avbtool verify_image --image vbmeta.img \
--expected_chain_partition \
vbmeta_system:1:/path/to/vbmeta_system/pubkey`
c) `avbtool verify_image --image vbmeta_system.img`
Change-Id: I064e583b247c44b9b2f19355838550bb5dbb8f26
Recovery can now parse the pem encoded x509 keys from a zipfile. So
instead of dumping the keys into a text file with some intermediate format,
we can simply create a zipfile with the keys.
Bug: 116655889
Test: make bootimage and check the generated zipfile, run sign_target_files_apks
Change-Id: Ib76feecfb26d6be713a07644e80ec96133759004
This sets up the root logger while running tests. It also logs to
stdout, as our Python test result parser doesn't like outputs from
stderr.
Test: `python -m unittest discover build/make/tools/releasetools \
> /dev/null` doesn't print log lines.
Change-Id: Ic99711bd458bc4b67b38226786fed604c2168476
Converts the following files to Python logging.
add_img_to_target_files.py
blockimgdiff.py
build_image.py
check_ota_package_signature.py
common.py
img_from_target_files.py
make_recovery_patch.py
ota_from_target_files.py
sparse_img.py
verity_utils.py
This separates logging outputs from normal outputs, and allows easier
switching between different verbosity levels. It also supports adjusting
logging config via environment variable (LOGGING_CONFIG).
Test: `m dist`
Test: `python -m unittest discover build/make/tools/releasetools`
Change-Id: Idfc3f7aff83e03f41f3c9b5067d64fd595d5192d
common.RunAndCheckOutput() checks the exit code and will raise on errors.
Test: python -m unittest test_ota_from_target_files
Test: Call ota_from_target_files.py with aosp_marlin target_files zip
Test: Inject an error to the command. Repeat the call without verbose
flag. It dumps the stack trace on error.
Change-Id: I85b765a33b9087bcbcb0571d6e632a07bb86c65c
This prepares for upcoming changes that refactor verity-related
functions into classes.
This CL makes minimal changes to the moved functions, by creating a new
BuildVerityImageError class in verity_utils.py replacing the former
BuildImageError that's specific to build_image.py. As part of the
change, it also moves the tests for AVBCalcMinPartitionSize into
test_verity_utils.py.
Test: python -m unittest test_verity_utils
Test: `m dist` with aosp_marlin-userdebug (Verified Boot 1.0)
Test: `m dist` with aosp_taimen-userdebug (Verified Boot 2.0)
Change-Id: I19b52714d8980705ea1f9484ac03eb0af9483240
All the releasetools unittests extend the common base class of
test_utils.ReleaseToolsTestCase. Define tearDown() in the base class to
do the clean-up works.
Test: `pylint --rcfile=pylintrc test_*.py`
Test: `python -m unittest discover .`
Change-Id: I51775d964ef032dcdf3bb89c55e1a31371cde708
There's a naming convention between the testcase and the class/method
being tested.
Also remove VerifiedBootVersion2HashtreeInfoGenerator as it's merely a
placeholder without telling anything much (e.g. whether it's to be
implemented or working as intended).
Test: python -m unittest test_verity_utils
Change-Id: Ieb156765c865aa551e3882f381e3a6db2cac4cbc
build_image.RunCommand and common.Run are essentially doing the same
work, but with different interfaces and return values. This CL
consolidates them by moving build_image.RunCommand into common.py as
common.RunAndCheckOutput, with the former calling common.Run internally.
common.RunAndCheckOutput checks the exit code: it returns the output
string on success, or raises common.ExternalError otherwise.
Test: `m dist` with aosp_marlin-userdebug
Test: python -m unittest test_build_image
Test: python -m unittest test_common
Change-Id: I3cd0e9301c576572194618eba8dc00db06c3c35f
I really only care about code that's built for Windows, but I may as
well clean up anywhere that's easy to clean up too...
Bug: N/A
Test: builds
Change-Id: I3ef34fb12ac90e9411b6421e9c23dd8524f056ae
stdout and stderr will default to subprocess.PIPE and subprocess.STDOUT
respectively (which is the expected behavior from most of the existing
callers), unless caller specifies any of them.
Test: `m dist`
Test: python -m unittest \
test_common \
test_add_img_to_target_files \
test_ota_from_target_files \
test_validate_target_files
Change-Id: I43b3f08edfa8a9bcfe54baf9848dc705c048e327
We don't need verity_root_hash or verity_salt to be in the prop dict.
Test: `m dist` with aosp_marlin-userdebug
Test: python -m unittest test_verity_utils
Change-Id: I5a9c50f4741dfb1083b3f590136335b6bc0e5216
Generate the transfer command "compute_hash_tree" for incremental
updates of the non-A/B devices that enable verified boot 1.0
Other changes include:
i. factor out verity_utils to use both in build_image and blockimgdiff
ii. add a new flag 'hashtree_info_generator' in sparse_image to generate
the hashtree information.
Bug: 25170618
Test: generate a package for aosp_angler; and run simulator
Change-Id: I4d4d7a3e41dc3da810d2cbf8988e85d187c9ab0e
The image building functions in build_image.py have been returning
(success, result) or special values to indicate the validity of the
result. The same logic can be better expressed by raising exceptions
instead, because
a) using a special value relies on caller to check for that magic
value;
b) exceptions can carry additional messages other than a boolean does,
e.g. the output from the failing command;
c) caller can have cleaner code flow without explicitly checking for
the validity of the result.
This CL changes such functions to raise on errors. The majority of these
functions are internal to build_image.py only, except for BuildImage()
that has a few callers in add_img_to_target_files.py (which all die upon
error anyway).
Test: `m dist`
Test: python -m unittest test_build_image
Test: python -m unittest test_add_img_to_target_files
Test: python -m unittest test_validate_target_files
Test: Inject an error to the depended binaries (e.g. avbtool), and check
that build_image.py exits with error messages.
Change-Id: Ibe4d51e267756bb1a00fa9238a213f9d55fd9b58
When AVB is enabled with PRODUCT_USE_DYNAMIC_PARTITION_SIZE, AVB
metadata (e.g., hash tree, fec metadata) will consume _RESERVED_SIZE,
resulting in smaller reserved size in file system (e.g., ext4).
Adding additional space for AVB signing and keep the _RESERVED_SIZE only
for file system. This is done by adding a function to binary search an
optimal partition size for a given image size (disk usage + _RESERVED_SIZE).
Bug: 112322265
Test: Build aosp_arm64-userdebug, calculate the running time of
AVBCalcMinPartitionSize() is about 0.3-0.4 seconds.
Test: python -m unittest test_build_image
Change-Id: I8f0051b57701d6fbba6a9db3756dd194066c74b8
Unless using dynamic partitions, `partition_size` should be a fixed
value that equals to the partition size in BoardConfig. It should stay
the same when building an image for that partition. Only the actual
image size that's used to hold the filesystem could be adjusted. This CL
cleans up the uses of `partition_size` and `image_size` to better
reflect such logic.
With dynamic partitions, the only thing that changes is the need to
compute `partition_size` upfront. Once that's done, `partition_size`
should remain unchanged.
Test: `m dist`
Test: `python -m unittest test_add_img_to_target_files`
Test: `python -m unittest test_validate_target_files`
Change-Id: Idedb3e018c95e8f63dc4d9c423be27f30ebb584f
Reads the fingerprint of a partition from the build props and embeds it
into the care_map. Later, the update_verifier will compare the
fingerprint before it performs the verification.
Bug: 114778109
Test: generate care_map for sailfish and new pixel
Change-Id: I06afa01bf6dd3de4456a08ee7960954facc775fb
The recovery image already contains a copy of first stage init, so we
can boot unconditionally to the recovery image and instruct first
stage init whether or not to boot to Android or to recovery. In this
case, we need neither the kernel to mount /system as / nor a separate
partition for recovery, so this change modifies the build scripts to
allow this combination.
Bug: 114062208
Test: Boot pixel from recovery to Android with BOARD_USES_RECOVERY_AS_BOOT
But without BOARD_BUILD_SYSTEM_ROOT_IMAGE
Change-Id: Icd047afb7f22d2724b3bcaca1aa0c837426dcce7
The new suffix distinguishes the new care_map from the ones in plain
text format; and thus the old update_verifier won't report an error
upon parsing failures.
Bug: 115740187
Test: Generate OTA files for Pixels
Change-Id: Ia782afd8cbb0f4bb8c363edaa00e92ab302d5d1b
Migrate off of one target onto another that should be identical.
"conscrypt" is about to be refactored for on-device use and signapk
needs to be unaffected.
Bug: 113148576
Test: make checkbuild
Change-Id: I3cecb4cc8fb8349383a82c8e430c936651e0835a
Bug: 112007947
Test: `m dist` with aosp_taimen-userdebug.
Test: Set up a target that uses chained VBMeta images of
`vbmeta_mainline` and `vbmeta_vendor`. `m dist` and check the
build log, as well as outputs from
`avbtool info_image --image vbmeta.img`,
`avbtool info_image --image vbmeta_mainline.img`,
`avbtool info_image --image vbmeta_vendor.img`.
Change-Id: Ib1d4e97f583b65245703eae15d211adcd9e83741
Also removing the requirement of BOARD_BOOTIMAGE_PARTITION_SIZE when
BOARD_AVB_ENABLE is set. Some targets (e.g., emulator) doesn't build
boot.img when AVB is enabled.
Bug: 112293933
Test: build a failure case and checks the error output is expected
Change-Id: I290c707719193ddaedcd9fd0b31de566ad17078c
This is in line with the change of mkuserimg_mke2fs.sh to python binary.
Bug: 112555072
Bug: 63866463
Test: unittests pass
Change-Id: I82c0be1e5bbc685edc15120da73aa43fdc9f2f05
Commit d572632f3d added support in build
system to blacklist given vendor properties. When
ro.vendor.build.fingerprint gets blacklisted, the OTA generation script
can't rely on those properties to determine a change in vendor images.
This CL considers such a case as "vendor images must have changed"
between the two builds.
Bug: 113892939
Test: Generate an incremental package with builds not having
ro.vendor.build.fingerprint.
Test: python -m unittest test_ota_from_target_files
Change-Id: I188de9c3cbeecf26132c92b9356e9d5fef75205e
