14 Commits

Author SHA1 Message Date
Yi-Yo Chiang
18650c7be0 releasetools: Remove deprecated BOARD_BUILD_SYSTEM_ROOT_IMAGE
This variable is always false or empty now, so clean up remaining
references to the variable.

Bug: 241346584
Test: Presubmit
Change-Id: I5dd3d356729d0bd2639633580bbb85f8b03582cf
2023-12-01 09:03:08 +00:00
Jooyung Han
c4b7b34b4b fix releasetools_test
Some test cases are failing because tests are not updated accordinly.

Bug: n/a
Test: atest releasetools_test
Change-Id: I3df071f72f01dedd6df4fa462ca52b8a0b1ffd4e
2021-10-27 16:06:10 +09:00
Tao Bao
82490d3de1 relesetools: Enable releasetools_test in presubmit.
About half of the testcases rely on external tools (i.e. the ones in
`otatools.zip`, which are external to releasetools module, but still
built by Android). It's WAI as releasetools scripts are mostly for
gluing purpose.

However, the current support in Soong doesn't allow packing the helper
modules as part of the built releasetools_test. This CL adds a decorator
that allows declaring external dependencies in testcases, which will be
skipped while running in presubmit. It doesn't affect local invocation
of `atest releasetools_test`.

Fixes: 112080715
Test: `atest releasetools_test`
Test: TreeHugger; check that releasetools_test is invoked (and test
      passes).
Change-Id: I8fdeb6549023cf5ddeb79d610c7c37cf9f13d3cc
2019-04-12 15:26:35 -07:00
Mark Salyzyn
780f595fac build_image: improvements to right size for dynamic partitions
If partition_reserved_size is 0 or undefined, and
use_dynamic_partition_size is true, we should approach no space
and no free inodes automatically.

Estimate the space and number of inodes required, then do a first
pass build to see how much space actually used, and use those values
to refine the estimate.

Depends on tune2fs to report the characteristics of the filesystem,
so only support for ext filesystems.  In the future if there has to
be a more generic ability, either a tool per a filesystem has to be
found, or we will need root capabilities to mount the filesystem to
acquire the characteristics live from the host system.

Test: manual + python -m unittest test_build_image
Bug: 111302946
Change-Id: I933a388be43516b6de7b5007b296765bd5556fde
2018-10-23 15:14:00 -07:00
Tao Bao
71197515bb releasetools: Move verity related functions into verity_utils.py.
This prepares for upcoming changes that refactor verity-related
functions into classes.

This CL makes minimal changes to the moved functions, by creating a new
BuildVerityImageError class in verity_utils.py replacing the former
BuildImageError that's specific to build_image.py. As part of the
change, it also moves the tests for AVBCalcMinPartitionSize into
test_verity_utils.py.

Test: python -m unittest test_verity_utils
Test: `m dist` with aosp_marlin-userdebug (Verified Boot 1.0)
Test: `m dist` with aosp_taimen-userdebug (Verified Boot 2.0)
Change-Id: I19b52714d8980705ea1f9484ac03eb0af9483240
2018-10-12 12:09:34 -07:00
Tao Bao
65b94e92be releasetools: Add a common base class for unittests.
All the releasetools unittests extend the common base class of
test_utils.ReleaseToolsTestCase. Define tearDown() in the base class to
do the clean-up works.

Test: `pylint --rcfile=pylintrc test_*.py`
Test: `python -m unittest discover .`
Change-Id: I51775d964ef032dcdf3bb89c55e1a31371cde708
2018-10-11 23:01:09 -07:00
Tao Bao
986ee86854 releasetools: Move build_image.RunCommand into common.py.
build_image.RunCommand and common.Run are essentially doing the same
work, but with different interfaces and return values. This CL
consolidates them by moving build_image.RunCommand into common.py as
common.RunAndCheckOutput, with the former calling common.Run internally.

common.RunAndCheckOutput checks the exit code: it returns the output
string on success, or raises common.ExternalError otherwise.

Test: `m dist` with aosp_marlin-userdebug
Test: python -m unittest test_build_image
Test: python -m unittest test_common
Change-Id: I3cd0e9301c576572194618eba8dc00db06c3c35f
2018-10-10 18:13:24 -07:00
Tao Bao
c6bd70a5e6 releasetools: Raise on image building errors.
The image building functions in build_image.py have been returning
(success, result) or special values to indicate the validity of the
result. The same logic can be better expressed by raising exceptions
instead, because
 a) using a special value relies on caller to check for that magic
    value;
 b) exceptions can carry additional messages other than a boolean does,
    e.g. the output from the failing command;
 c) caller can have cleaner code flow without explicitly checking for
    the validity of the result.

This CL changes such functions to raise on errors. The majority of these
functions are internal to build_image.py only, except for BuildImage()
that has a few callers in add_img_to_target_files.py (which all die upon
error anyway).

Test: `m dist`
Test: python -m unittest test_build_image
Test: python -m unittest test_add_img_to_target_files
Test: python -m unittest test_validate_target_files
Test: Inject an error to the depended binaries (e.g. avbtool), and check
      that build_image.py exits with error messages.
Change-Id: Ibe4d51e267756bb1a00fa9238a213f9d55fd9b58
2018-10-03 11:31:44 -07:00
Bowgo Tsai
040410c104 Prevent AVB signing from using _RESERVED_SIZE
When AVB is enabled with PRODUCT_USE_DYNAMIC_PARTITION_SIZE, AVB
metadata (e.g., hash tree, fec metadata) will consume _RESERVED_SIZE,
resulting in smaller reserved size in file system (e.g., ext4).

Adding additional space for AVB signing and keep the _RESERVED_SIZE only
for file system. This is done by adding a function to binary search an
optimal partition size for a given image size (disk usage + _RESERVED_SIZE).

Bug: 112322265
Test: Build aosp_arm64-userdebug, calculate the running time of
      AVBCalcMinPartitionSize() is about 0.3-0.4 seconds.
Test: python -m unittest test_build_image
Change-Id: I8f0051b57701d6fbba6a9db3756dd194066c74b8
2018-10-03 10:08:45 +08:00
Tianjie Xu
573322279a Use the new executable for mkuserimg_mke2fs
This is in line with the change of mkuserimg_mke2fs.sh to python binary.

Bug: 112555072
Bug: 63866463
Test: unittests pass
Change-Id: I82c0be1e5bbc685edc15120da73aa43fdc9f2f05
2018-09-05 20:52:47 +00:00
Tom Cherry
d14b895665 Support a first stage ramdisk via TARGET_RAMDISK_OUT
Support a first stage ramdisk which will include the targets placed
into TARGET_RAMDISK_OUT.  This replaces the existing ramdisk on
existing devices.

All system images are now built to be mounted as the root dir.
Devices with a first stage ramdisk will switch root to the system
partition.

BOARD_BUILD_SYSTEM_ROOT_IMAGE remains and is used to specify if the
system partition is going to be directly used as rootfs without the
ramdisk.

Bug: 79173823
Bug: 79758715
Test: hikey boots, sailfish boots
Test: OTA walleye from P to master
Change-Id: Idbb2dccc6340b0235a4bef03e11e420a9ed154b6
2018-08-28 12:42:41 -07:00
Tao Bao
c2606eb548 releasetools: Fix an issue in image size computation.
When building a system image with system_root_image enabled, the size
computation should include files under both of in_dir (i.e. /system
files) and root (pointed by 'root_dir'). Because files from both
locations will end up into the built image. The files under root are
usually only a few MiBs, but should be accounted for especially in the
context of logical partitions (where the partition size will be
allocated based on the actual need). Note that we will still need some
"reserved space" (defined via BOARD_*_PARTITION_RESERVED_SIZE) to cover
the cost for filesystem and/or verity metadata.

This CL moves the combination of the two dirs up, before parsing and
computing other properties. This doesn't affect anything for a
successful image building path. It may however increase the time to
error out in certain error path, since it copies the files earlier now.

Test: python -m unittest test_build_image
Test: `make dist`
Test: Setup a target with PRODUCT_USE_LOGICAL_PARTITIONS == true and
      BOARD_SYSTEMIMAGE_PARTITION_RESERVED_SIZE == 20MiB. Build system
      image successfully.
Test: Setup a target with PRODUCT_USE_LOGICAL_PARTITIONS == true and
      BOARD_SYSTEMIMAGE_PARTITION_RESERVED_SIZE == 20MiB. Write a large
      file to root dir (PRODUCT_OUT_ROOT). The image building fails, but
      reporting a size that accounts for both of /system and root.
Change-Id: Idfb26b8e259626ba57ec3bd4f85d357c30e56163
2018-07-26 16:59:57 +00:00
Tao Bao
d8a953d7ab releasetools: Move assertions into CheckHeadroom().
Assert the input arguments inside CheckHeadroom() function, and add
matching tests. Also use common.MakeTemp{File,Dir} in tests.

Test: `m dist`
Test: python -m unittest test_build_image
Change-Id: Ib45f24651af9591a9cfdd72aac3558f6c9a80f44
2018-01-03 11:52:16 -08:00
Tao Bao
d4349f2106 build_image: Factor out CheckHeadroom() and add tests.
The test is mostly trivial, but it ensures the result parsing from
actual calls to mke2fs.

Test: python -m unittest test_build_image
Test: `m dist` on aosp_marlin-userdebug (w/ and w/o defining
      PRODUCT_SYSTEM_HEADROOM respectively).
Change-Id: I8b9964213950e76f6d7d5518414a1bab888b4706
2017-12-07 23:39:56 -08:00