When AVB is enabled with PRODUCT_USE_DYNAMIC_PARTITION_SIZE, AVB
metadata (e.g., hash tree, fec metadata) will consume _RESERVED_SIZE,
resulting in smaller reserved size in file system (e.g., ext4).
Adding additional space for AVB signing and keep the _RESERVED_SIZE only
for file system. This is done by adding a function to binary search an
optimal partition size for a given image size (disk usage + _RESERVED_SIZE).
Bug: 112322265
Test: Build aosp_arm64-userdebug, calculate the running time of
AVBCalcMinPartitionSize() is about 0.3-0.4 seconds.
Test: python -m unittest test_build_image
Change-Id: I8f0051b57701d6fbba6a9db3756dd194066c74b8
Unless using dynamic partitions, `partition_size` should be a fixed
value that equals to the partition size in BoardConfig. It should stay
the same when building an image for that partition. Only the actual
image size that's used to hold the filesystem could be adjusted. This CL
cleans up the uses of `partition_size` and `image_size` to better
reflect such logic.
With dynamic partitions, the only thing that changes is the need to
compute `partition_size` upfront. Once that's done, `partition_size`
should remain unchanged.
Test: `m dist`
Test: `python -m unittest test_add_img_to_target_files`
Test: `python -m unittest test_validate_target_files`
Change-Id: Idedb3e018c95e8f63dc4d9c423be27f30ebb584f
Also removing the requirement of BOARD_BOOTIMAGE_PARTITION_SIZE when
BOARD_AVB_ENABLE is set. Some targets (e.g., emulator) doesn't build
boot.img when AVB is enabled.
Bug: 112293933
Test: build a failure case and checks the error output is expected
Change-Id: I290c707719193ddaedcd9fd0b31de566ad17078c
This is in line with the change of mkuserimg_mke2fs.sh to python binary.
Bug: 112555072
Bug: 63866463
Test: unittests pass
Change-Id: I82c0be1e5bbc685edc15120da73aa43fdc9f2f05
Support a first stage ramdisk which will include the targets placed
into TARGET_RAMDISK_OUT. This replaces the existing ramdisk on
existing devices.
All system images are now built to be mounted as the root dir.
Devices with a first stage ramdisk will switch root to the system
partition.
BOARD_BUILD_SYSTEM_ROOT_IMAGE remains and is used to specify if the
system partition is going to be directly used as rootfs without the
ramdisk.
Bug: 79173823
Bug: 79758715
Test: hikey boots, sailfish boots
Test: OTA walleye from P to master
Change-Id: Idbb2dccc6340b0235a4bef03e11e420a9ed154b6
Attempting to reduce the number of different spellings we have for
"product services" partition in the codebase.
Bug: 112431447
Test: m
Change-Id: I6debd3f5f5abc4bd1e22f2ef11c5fe131915976c
Merged-In: I6debd3f5f5abc4bd1e22f2ef11c5fe131915976c
When trying to support sparse format for GSIs, aosp/720949, a bug in
e2fsck was triggered, and hence failed the build for sparse GSI.
This workaround disables the detection of ASAN ODR so we can build
sparse GSIs successfully like we previously did for the vendors,
some of which can't handle unsparse images.
Bug: 112062612
Test: $ lunch aosp_x86_64-eng
$ m SANITIZE_HOST=address ASAN_OPTIONS=detect_odr_violation=0 -j
$ emulator
Change-Id: Ifafd27aa1e57d785b914f78a4d9bfef6a0c33fc9
Split the flag into PRODUCT_USE_DYNAMIC_PARTITION_SIZE
and PRODUCT_BUILD_SUPER_PARTITION. More sub-flags can be
added with the same way.
Also change some checks with USE_LOGICAL_PARTITIONS with
the appropriate sub-flags.
This allows easier device bring-up to fulfill the requirements
separately.
This also enables dynamic partition size without logical patition.
Originally, to enable BOARD_*_PARTITION_RESERVED_SIZE must also
enable PRODUCT_USE_LOGICAL_PARTITIONS. The patch fix the rule to
let dynamic partition size is able to be used independently.
Fixes: 111966003
Test: make superimage with PRODUCT_USE_LOGICAL_PARTITIONS
Change-Id: I45b29c87bf94356d8416bbd58a14c8982039c222
Make names consistent.
Some things that does not change:
- productservicesimage target and intermediates
Test: build product services image
Bug: 111609632
Change-Id: I4c2b975e194577aad3d51b908b103880625deb5e
When building a system image with system_root_image enabled, the size
computation should include files under both of in_dir (i.e. /system
files) and root (pointed by 'root_dir'). Because files from both
locations will end up into the built image. The files under root are
usually only a few MiBs, but should be accounted for especially in the
context of logical partitions (where the partition size will be
allocated based on the actual need). Note that we will still need some
"reserved space" (defined via BOARD_*_PARTITION_RESERVED_SIZE) to cover
the cost for filesystem and/or verity metadata.
This CL moves the combination of the two dirs up, before parsing and
computing other properties. This doesn't affect anything for a
successful image building path. It may however increase the time to
error out in certain error path, since it copies the files earlier now.
Test: python -m unittest test_build_image
Test: `make dist`
Test: Setup a target with PRODUCT_USE_LOGICAL_PARTITIONS == true and
BOARD_SYSTEMIMAGE_PARTITION_RESERVED_SIZE == 20MiB. Build system
image successfully.
Test: Setup a target with PRODUCT_USE_LOGICAL_PARTITIONS == true and
BOARD_SYSTEMIMAGE_PARTITION_RESERVED_SIZE == 20MiB. Write a large
file to root dir (PRODUCT_OUT_ROOT). The image building fails, but
reporting a size that accounts for both of /system and root.
Change-Id: Idfb26b8e259626ba57ec3bd4f85d357c30e56163
When verity is enabled, `partition_size` has the "adjusted" partition
size for holding the filesystem files (i.e. excluding verity hashtree,
and FEC metadata if applicable), whereas `original_partition_size` has
the one for the actual image returned by build_image.py.
`partition_size` must be smaller than `original_partition_size`. As a
result, the later assertion in Makefile (e.g. assert-max-image-size) may
fail to hold when using `partition_size`.
This CL addresses the issue by returning the value in
`original_partition_size` instead, if available. It also changes to
include both values when dumping the size info on image creation
failure.
Fixes: 79106666
Test: `m dist` with aosp_walleye-userdebug
Test: Set up walleye to use logical partitions, with a reserved size of
20MiB. `m systemimage` no longer fails.
Test: Set up walleye that doesn't use verified boot. Use `m systemimage`
to build an oversized image. Check the error message.
Change-Id: I41d77bffa85e8eadfbf4a2a7dcd2e798fadfdab7
This property indicates the path to the root directory (i.e.
TARGET_ROOT_OUT at build time, or ROOT/ in a target_files.zip).
build_image.py will only use this property when building a target using
system_root_image. Under such a configuration, files under this dir
never land into a ramdisk, but as part of system.img. This CL renames
the property name to avoid confusion.
Note that with this change releasetools keeps working with older
target_files zips that contain 'ramdisk_dir'. Because common.py doesn't
read the value of that property, but sets it to ROOT/ directly.
Test: `m dist` on aosp_marlin-userdebug.
Test: `m dist` on aosp_angler-userdebug.
Test: Run add_img_to_target_files.py with a new target_files.zip and an
old target_files.zip respectively.
Change-Id: Ie9e19dd17d8c2abb4c4a97449e436774d5dd2a36
This CL is largely an adaptation of Change-Id
I774e6a38003734421591e51bed103802ff84f432
It adds the following variables:
- BOARD_AVB_PRODUCT_SERVICES_KEY_PATH
- BOARD_AVB_PRODUCT_SERVICES_ALGORITHM
- BOARD_AVB_PRODUCT_SERVICES_ROLLBACK_INDEX_LOCATION
- BOARD_PRODUCT_SERVICESIMAGE_FILE_SYSTEM_TYPE
- BOARD_PRODUCT_SERVICESIMAGE_EXTFS_INODE_COUNT
- BOARD_PRODUCT_SERVICESIMAGE_EXTFS_RSV_PCT
- BOARD_PRODUCT_SERVICESIMAGE_PARTITION_SIZE
- BOARD_PRODUCT_SERVICESIMAGE_JOURNAL_SIZE
- BOARD_PRODUCT_SERVICESIMAGE_SQUASHFS_BLOCK_SIZE
- BOARD_PRODUCT_SERVICESIMAGE_SQUASHFS_COMPRESSOR
- BOARD_PRODUCT_SERVICESIMAGE_SQUASHFS_COMPRESSOR_OPT
- BOARD_PRODUCT_SERVICESIMAGE_SQUASHFS_DISABLE_4K_ALIGN
- BOARD_PREBUILT_PRODUCT_SERVICESIMAGE
- BOARD_USES_PRODUCT_SERVICESIMAGE
- LOCAL_PRODUCT_SERVICES_MODULE
- PRODUCT_PRODUCT_SERVICES_BASE_FS_PATH
- PRODUCT_PRODUCT_SERVICES_VERITY_PARTITION
- PRODUCT_PRODUCT_SERVICES_PROPERTIES
- TARGET_COPY_OUT_PRODUCT_SERVICES
- TARGET_OUT_PRODUCT_SERVICES
- TARGET_OUT_PRODUCT_SERVICES_*
Bug: 80741439
Test: Successfully built product-services.img with one module in it, and flashed
on device. Also successfully built image with /system/product-services directory
and no /product-services partition.
Change-Id: I5d229f6ac729ea6df9ff1f14cee2e28972cd9b4d
Test: `make vendorimage`
Test: `make vendorimage` with the following:
- install a large file to vendor image
fails as expected (because _PARTITION_SIZE is exceeded)
Test: `make vendorimage` with the following:
- set PRODUCT_USE_LOGICAL_PARTITIONS to true
- set BOARD_VENDORIMAGE_PARTITION_RESERVED_SIZE
fails as expected (BOARD_VENDORIMAGE_PARTITION_SIZE needs
to be undefined)
Test: `make vendorimage` with the following:
- install a large file to vendor image
- set PRODUCT_USE_LOGICAL_PARTIIONS to true
- add a small BOARD_VENDORIMAGE_PARTITION_RESERVED_SIZE
- remove BOARD_VENDORIMAGE_PARTITION_SIZE
build succeeds.
Bug: 79106666
Change-Id: Ica8fdce64e8f37d91e66e3d49c0c74fadd15a832
- Copy "use_logical_partitions" to *_image_info.txt before sending
it to build_image.py, so that the script can use this variable.
- build_image.py emits an additional properties file to inform
the build system about the system image size.
Test: `make systemimage`
Test: `make systemimage` with the following:
- install a large file to system image
fails as expected (because _PARTITION_SIZE is exceeded)
Test: `make systemimage` with the following:
- set PRODUCT_USE_LOGICAL_PARTITIONS to true
- set BOARD_SYSTEMIMAGE_PARTITION_RESERVED_SIZE
fails as expected (BOARD_SYSTEMIMAGE_PARTITION_SIZE needs
to be undefined)
Test: `make systemimage` with the following:
- install a large file to system image
- set PRODUCT_USE_LOGICAL_PARTIIONS to true
- add a small BOARD_SYSTEMIMAGE_PARTITION_RESERVED_SIZE
- remove BOARD_SYSTEMIMAGE_PARTITION_SIZE
build succeeds.
Test: same for systemotherimage
Bug: 79106666
Change-Id: I574062882acd1ecd633ac38c5a8c5351b90a32d8
This CL defaults the journal size to 0 for system images (i.e. system,
vendor, system_other, oem, product). We used to do this by explicitly
defining BOARD_{SYSTEM,VENDOR,OEM,PRODUCT}IMAGE_JOURNAL_SIZE to 0 in
device-specific BoardConfig. Targets can still specify a non-zero
journal size as needed.
With this CL, marlin/sailfish gets 32MB / 16MB free space back for
system and vendor respectively.
Bug: 75975085
Test: `m dist`. Check the journal size in the generated images.
Change-Id: Ib3185d07c49be2b4f0fac5fe17ec1a82093ba0c0
(cherry picked from commit 965542fe69)
Defaults the reserved blocks for root to 0% on read only partitions
(system, system_other, vendor, oem). It also adds support for
explicitly specifying the extfs reserved percentage via
BOARD_{SYSTEM,VENDOR,OEM,PRODUCT}IMAGE_EXTFS_RSV_PCT.
This eventually translates down to the -m option for mkfs.
Removing the reserved space can save at least 5% from the default.
dumpe2fs system:
Reserved blocks uid: 0 (user root)
Reserved blocks gid: 0 (group root)
Bug: 75975085
Test: Build, verify reserved space is changed accordingly
Change-Id: I212d82741908b636db0d658a1c4847bbaadfd5ba
(cherry picked from commit 5ff758799c)
build_image.py used to invoke build_verity_metadata.py with a hard-coded
path of 'system/extras/verity/build_verity_metadata.py', which makes it
hard to run unittests from non-$(ANDROID_BUILD_TOP) directory.
This CL adds the dependency on the tool, so that it gets installed to
$(HOST_OUT_EXECUTABLES), then removes the hard-coded path.
Bug: 74544459
Bug: 76015688
Test: `m dist`
Test: python -m unittest test_validate_target_files
Change-Id: I0dcf4eb067a0db6f099cb589eb99a151a05c7f2b
The output (stdout and stderr combined) would be helpful in understanding
the cause of the failure.
Not changing other occurrences in build_image.py yet, since it's less
obvious if a failed call should be considered fatal (thus whether to
dump such outputs). For example, GetVeritySize() simply returns 0 on
failed calls. They deserve further clean-ups in later CLs (with unit
tests).
Bug: 71864688
Test: Inject errors into mke2fs/e2fsck/simg2img. `m snod` prints outputs.
Test: `m snod` remains quiet on successful runs.
Change-Id: I172403f6cd05bce93767265dbcb110271a68e2a8
Mostly cosmetic changes, such as moving print statement to print
function. The only functional change is to wrap the file opening in
Append() with try statement, which would dump the error message as
needed.
Test: `m dist`
Test: python -m unittest test_build_image
Change-Id: I4e6c593517cf737bfcbe51bc533b22a247b10e44
This simplifies the work to track mkdtemp()'d dirs. This CL also cleans
up the temporary file creation in build_image.py.
Test: `m dist`
Test: python -m unittest test_common
Change-Id: Ied7d86126714b9b26e5df7bcca1dfdea21cfcf60
The test is mostly trivial, but it ensures the result parsing from
actual calls to mke2fs.
Test: python -m unittest test_build_image
Test: `m dist` on aosp_marlin-userdebug (w/ and w/o defining
PRODUCT_SYSTEM_HEADROOM respectively).
Change-Id: I8b9964213950e76f6d7d5518414a1bab888b4706
This is only used in file-based OTA, where we may reserve space on
/system partition. With the deprecation of file-based OTA since O, the
code has become obsolete.
Test: `m dist`
Change-Id: I2bd686b292f93b566fe28a2bcd74c564ffc75dac
BuildImage can return false for lots of reasons. Log two of the more
plausible ones. You'll still need to edit this file and pass True to
RunCommand to work out *what* went wrong, but at least this logging will
tell you *where* things went wrong.
Bug: http://b/63142920
Test: debugged my actual problem
Change-Id: If84427058108dcac0a1b5e2216b8bb11eef522ca
This also updates mkf2fsuserimg.sh to call sload.f2fs.
Change-Id: I245f6b3dfcd966fbc6783329d781375879fcd065
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
We have added support in mkuserimg_mke2fs.sh that allows specifying
filesystem UUID and hash seed UUID. This CL generates and passes the
UUIDs based on the image name and build fingerprint. This way we can
rebuild and get identical images.
Note that this only applies to images generated with `m dist` and with
signing tools (sign_target_files_apks.py and
add_img_to_target_files.py). Images under $OUT (built with `make`) are
not affected.
Bug: 64994964
Test: lunch marlin-userdebug; run `m dist` twice (after deleting the
intermediate files under $OUT/obj/PACKAGING), and compare the
generated images.
Change-Id: I41cf4e5869582bb930af2f35a8e9c79bff43b2a2
(cherry picked from commit 3aa21e6bb9)
Adding verified boot metadata with a "disable magic". The resulting
metadata at the end of each image (e.g., system.img, vendor.img) will
be the same as triggering an "adb disable-verity" on an USERDEBUG image.
This can help simplify the code on fs_mgr, which won't have to check if
current image is an ENG build or not.
Bug: 63056044
Test: boot sailfish eng/userdebug builds
Change-Id: I95d23ac7b76c04d6d4483c9c4dc1de16bf0d9c3a
We want the generated images being identical for the same source files.
Currently the generated ext4 image (either from make_ext4fs or mke2fs)
is reproducible, but the AVB footer added by avbtool contain changes
because of the random salt being used.
This CL changes the avbtool invocation to specify "--salt <hexstring>"
(already supported by avbtool) to use reproducible salt that's computed
based on fingerprints (or thumbprints if applicable).
Bug: 67023482
Test: Regenerate images from the same source as follows:
Use a target_files.zip from an AVB-enabled target.
$ zip -d target_files.zip IMAGES/\*
$ ./build/make/tools/releasetools/add_img_to_target_files.py \
-v target_files.zip
Repeat the above commands and compare the generated images.
Change-Id: Id9db17ae0132ca3a820b4be5a5ef06ca3fef71ed
(cherry picked from commit 8f05cca1d9)
Make the build log less verbose; also print the result for
AdjustPartitionSizeForVerity().
Bug: 65292710
Test: Run add_img_to_target_files with and without "-v" and check
the logs.
Change-Id: Icdc86197ce69db036658ea9e77cc4ebe1750463b
Current build system will include AVB metadata from each partition and
store them into /vbmeta partiton when BOARD_AVB_ENABLE is set, which makes
each partition tightly-coupled.
Add the support for 'chain partition':
- The vbmeta of each partition is stored on the same partition itself.
- The public key used to verify each partition is stored in /vbmeta.
For example, the following build variables are required to enable chain
partition for system partition:
- BOARD_AVB_SYSTEM_KEY_PATH := path/to/system_private_key
- BOARD_AVB_SYSTEM_ALGORITHM := SHA512_RSA8192
- BOARD_AVB_SYSTEM_ROLLBACK_INDEX := 1
- BOARD_AVB_SYSTEM_ROLLBACK_INDEX_LOCATION := 2
The corresponding settings will be added into META/misc_info.txt for
build_image.py and/or add_img_to_target_files.py:
- avb_system_key_path=path/to/system_private_key
- avb_system_algorithm=SHA512_RSA8192
- avb_system_add_hashtree_footer_args=--rollback_index 1
- avb_system_rollback_index_location=2
To enable chain partition for other partitions, just replace SYSTEM with
BOOT, VENDOR and/or DTBO in the build variables.
Also switch from `avbtool make_vbmeta_image --setup_rootfs_from_kernel system.img ...`
to `avbtool add_hashtree_footer --image system.img --setup_as_rootfs_from_kernel...`
when BOARD_BUILD_SYSTEM_ROOT_IMAGE is true. This works for both chained
and no-chained:
- chained: `avbtool add_hashtree_footer --setup_as_rootfs_from_kernel` will
add dm-verity kernel cmdline descriptor to system.img
- no-chained: `avbtool make_vbmeta_image --include_descriptors_from_image
system.img` will include the kernel cmdline descriptor from system.img into
vbmeta.img
Bug: 38399657
Test: `make` pass, flash images from $OUT and boot device without chain partitions
Test: `make` pass, flash images from $OUT and boot device with chain partitions
Test: `make dist` pass, flash images from TF.zip and boot device without chain partitions
Test: `make dist` pass, flash images from TF.zip and boot device with chain partitions
Test: follow the same steps in
https://android-review.googlesource.com/#/c/407572/
Change-Id: I344f79290743d7d47b5e7441b3a21df812a69099
When AVB is enabled, generate care_map.txt and add it to the target
files. Also copy it into the OTA package where it will later be used
by the update_verifier.
Bug: 62208947
Test: \
1. Run add_img_to_target_files on the TF of a new pixel device,
and care_map.txt generates successfully.
2. Make dist in oc-dr1-release and find care_map.txt in the OTA package.
3. update_verifier succeeds in reading all the blocks on the care_map,
and fails to read out-of-bound blocks.
Change-Id: I2881711e6f87789cb7de150dbeca18b756fed68a
`make custom_images` supports to build different kinds of *non-droid* images,
e.g., odm.img. Adding the support of signing them with either AVB HASH footer
or AVB HASHTREE footer. The user can use HASH for small images and
HASHTREE for large images.
Sample signing configurations:
* AVB HASH footer:
- CUSTOM_IMAGE_AVB_HASH_ENABLE := true
- CUSTOM_IMAGE_AVB_ADD_HASH_FOOTER_ARGS := --append_to_release_string my_odm_image
* AVB HASHTREE footer:
- CUSTOM_IMAGE_AVB_HASHTREE_ENABLE := true
- CUSTOM_IMAGE_AVB_ADD_HASHTREE_FOOTER_ARGS := --fec_num_roots 8
* Using custom signing key:
- CUSTOM_IMAGE_AVB_ALGORITHM := SHA256_RSA2048
- CUSTOM_IMAGE_AVB_KEY_PATH := external/avb/test/data/testkey_rsa2048.pem
Bug: 36701014
Test: `make custom_images` with AVB HASH footer
Test: `make custom_images` with AVB HASHTREE footer
Test: `make droid` to check system.img is still properly signed with AVB HASHTREE
Test: `make droid` to check vendor.img is still properly signed with AVB HASHTREE
Change-Id: I8dc420e12e37e9a631345c0cd883339db05d489f
This is useful for devices with low disk space with different
build variants.
Bug: 37469715
Test: Regular image builds successfully, errors occur when
the headroom size is greater than available partition space.
Change-Id: I526cdd0f84981bbd16e3afcfe1cd7fc43dce98ef
