Commit Graph

958 Commits

Author SHA1 Message Date
Tao Bao
f9023856ad releasetools: Add prefix when dumping fingerprints.
We used to dump "Source: <fingerprint>" in update logs. The "Source: "
prefix was unintentionally dropped out.

Test: Check the generated incremental BBOTA script.
Change-Id: I4de62333aa38e3fb09a76df0e769b62af48e0313
2016-12-14 11:53:38 -08:00
Patrick Tjin
a1900841bb build_image: add support for specifying number of inodes
Add support for specifying number of inodes when creating
system, vendor, oem partitions.  These are all read-only
and have no use for extra inodes.  Removing extra inodes
saves a lot of space.

Bug: 32246383
Change-Id: I13f1d4614b64a4abc752c42a1c65d3d151481c21
(cherry picked from commit b59eca3586)
2016-12-06 21:07:23 -08:00
The Android Automerger
ada51f3b8f Manually merge commit 'c13da91' into stage-aosp-master
* commit 'c13da91':
  Add support for creating ext4 images with mke2fs

BUG:33304034
2016-12-03 04:02:09 +00:00
Treehugger Robot
c13da91023 Merge "Add support for creating ext4 images with mke2fs" 2016-12-02 22:48:23 +00:00
Tao Bao
d07f9995b8 Merge "Build recovery-two-step.img for two-step OTAs."
am: 06e14e438f

Change-Id: I9c5d6e5f71da88bc0f662373335b024a55ca193e
2016-12-02 15:19:00 +00:00
Tao Bao
d42e97ebb4 Build recovery-two-step.img for two-step OTAs.
In two-step OTAs, we write recovery image to /boot as the first step so
that we can reboot from there and install a new recovery image to
/recovery. However, bootloader will show "Your device is corrupt"
message when booting /boot with the recovery image. Because the recovery
image encodes the path of "/recovery" as part of the signature metadata,
which fails the verified boot.

This CL generates a special "recovery-two-step.img" in addition to the
regular recovery.img. This image encodes "/boot" when being signed,
which will be flashed to /boot at stage 1/3 in a two-step OTA.

Here are the desired changes:

- 'IMAGES/recovery-two-step.img' exists in target_files.zip for non-A/B
targets (e.g. bullhead). The image should not exist for targets that
don't have a recovery partition (e.g. A/B devices like sailfish).

- <device>-img.zip should not contain 'recovery-two-step.img'.

- Nothing should change when building non-two-step OTAs. For two-step
OTAs, 'recovery-two-step.img' should be included in the OTA package;
'updater-script' should flash this image to /boot at stage 1/3.

- When building a two-step OTA with an input TF.zip that doesn't have
  IMAGES/recovery-two-step.img, it should use the existing
  IMAGES/recovery.img instead.

Bug: 32986477
Test: Tested the steps above on bullhead and sailfish.
Change-Id: I34e6c599bcf2011d4cd5c926999418b3975d6d0f
2016-12-01 17:47:59 -08:00
Tao Bao
2e3f1c5258 Merge "releasetools: Remove dead parameter in GetBootableImage()."
am: 8df4fea85f

Change-Id: I192af3927849d0d8f995f0002b8b8f2e191b794f
2016-12-01 17:09:07 +00:00
Tao Bao
2a4336bc04 releasetools: Remove dead parameter in GetBootableImage().
The 'system_img_path' parameter was introduced in commit
d995f4b04d, but became obsolete since
commit 2ce63edab7.

Test: m dist
Change-Id: Iffd496d929db5cc3dfc955a48bfc1b1317bd012f
2016-12-01 00:08:27 -08:00
Adrien Schildknecht
9a072cc0b0 Add support for creating ext4 images with mke2fs
We are investigating replacing make_ext4fs with the upstream tool mke2fs.
To mitigate the trouble that may arise if the new tool behave differently
compared to the old one, there will be a transition period.

Devices that want to use the new way of creating ext4 images can set the
variable "TARGET_USES_MKE2FS" to true in their BoardConfig.mk
By default, the build system will choose the old tool 'make_ext4fs'.

Test: m otapackage with TARGET_USES_MKE2FS={,false,true}
Change-Id: I282bcb9efe335a86c53986283090ca947d65c7f8
2016-11-18 17:06:29 -08:00
Tao Bao
dc04f93e39 Merge "releasetools: Use java_path in sign_target_files_apks.py."
am: 4a71d500c1

Change-Id: I61400574ac3f7d29906ca571f497554631ecd72d
2016-11-08 23:42:26 +00:00
Tao Bao
e95540e060 releasetools: Use java_path in sign_target_files_apks.py.
Prior to this CL, it was calling the hard-coded "java" although it was
accepting a "--java_path" option.

Also switch OPTIONS.java_args from string to list. Otherwise it won't
work when providing multiple args.

Bug: 32737832
Test: Specify "--java_path=" and "--java_args" when invoking
      sign_target_files_apks.py with "-v". Check the commands being
      called.
Change-Id: Id7ef98e778646d532027434de7fba9b7a104dbd0
2016-11-08 12:48:43 -08:00
Tao Bao
9242e1f750 Merge "releasetools: compare size between compressed and patch while build otapackage"
am: 49758959f2

Change-Id: I17b988878cc7ffdfbe490f404c9cd5ab996a8c5f
2016-11-05 00:13:55 +00:00
Tao Bao
49758959f2 Merge "releasetools: compare size between compressed and patch while build otapackage" 2016-11-05 00:09:35 +00:00
Tao Bao
e3554167e2 Merge "releasetools: Make BBOTA incremental generation repeatable."
am: b9bd78d110

Change-Id: I6402481bf300b52d36df1570cceeae6ec3ff4f8f
2016-11-04 21:29:15 +00:00
Tao Bao
3348228423 releasetools: Make BBOTA incremental generation repeatable.
set() doesn't keep elements according to the order of insertion. So
Transfers managed with set() in intermediate steps may not appear in the
same order across runs. This leads to slightly different output packages
when generating the same incremental OTA.

This CL fixes the issue by replacing set() with OrderedDict() in
blockimgdiff.GenerateDigraph() and blockimgdiff.FindVertexSequence().

It also adds a testcase that ensures blockimgdiff.GenerateDigraph()
preserves the insertion order for Transfer.goes_after set.

Bug: 32220816
Test: ota_from_target_files.py gives identical package when running
multiple times.

Change-Id: I56d551e5ca926993ab46896e33c80e0ce42e506a
2016-10-26 09:40:50 -07:00
Tao Bao
f4a88279ed Merge "releasetools: Support verity signer args."
am: 02d2d68aec

Change-Id: Ib1aed8ff11eeef2fd64a6520e92cfb30678dadc7
2016-10-24 18:53:33 +00:00
YOUNG HO CHA
ccc5c40523 releasetools: compare size between compressed and patch while build otapackage
Currently, whether contains patch or verbatim, compute with file size
and patch size.
But ota file must be compressed with zip, so it should be better with
compressed size than uncompressed.

Test: aosp_shamu-user build without proprietary blobs between MOB30P and NRD90S
$ du -k ota_shamu_old.zip ota_shamu_new.zip
217252	ota_shamu_old.zip
216520	ota_shamu_new.zip

Change-Id: If68cb1fbe2f7815067451915a0dcfe93ea5ba8d6
Signed-off-by: YOUNG HO CHA <ganadist@gmail.com>
2016-10-18 17:27:09 +09:00
Tao Bao
458104266f releasetools: Support verity signer args.
system/extras/verity/build_verity_metadata.py now accepts
"--signer_args" to specify verity signer args.

Also remove the duplicate "--verity_signer_args" in
add_img_to_target_files.py, as we already have that in common.py.

Bug: 31500665
Test: Building and signing work w/ and w/o --signer_args.

Change-Id: I02f59c50a1ebf15c5505e9fffd5b9bbbbaa785be
2016-10-17 16:25:22 -07:00
David Zeuthen
4e9c89ab87 resolve merge conflicts of 86180eb to nyc-mr1-dev-plus-aosp
Change-Id: Ia4bfb2571bac509cebe22f70ff43e6696f832423
2016-10-04 18:53:34 -04:00
David Zeuthen
70209ad9f0 Merge "Make room for AVB hashtree and metadata." 2016-10-04 22:32:27 +00:00
David Zeuthen
4014a9daa7 Make room for AVB hashtree and metadata.
While the system.img images currently built with AVB support verify
correctly, mounting the filesystem content fails. This is because
'avbtool add_hashtree_footer' used to claim some of the unused /
DONT_CARE space for stashing the verity tables and this resulting in the
mapped device ending up being smaller causing the mount failure.

Fix this by leaving enough room for AVB hashtree and metadata before
building the image. This is achieved by moving the AVB hashtree support
into build_image.py and using a just added '--calc_max_image_size'
option to 'avbtool add_hashtree_footer' to figure out how much space to
leave out.

This depends on https://android-review.googlesource.com/#/c/281821/

Bug: 31264226
Test: Mounting dm-verity set up from system.img now works.

Merged-In: I4c5de1004c1059f8c582e76b3b8517d427aa1a87

Change-Id: I945a5f1f6782791736cd319f216cfa6b448fb04d
2016-10-04 17:32:16 -04:00
Tao Bao
658af49662 Merge "releasetools: Clean up sign_target_files_apks.py." am: ad49deb5e6 am: 4299e48f0c
am: dfc210c9a4

Change-Id: Iee5cda2c10bc9c474140bf1f303261d8cf31d7b8
2016-10-01 00:24:10 +00:00
Tao Bao
9aa4b9b1b6 releasetools: Clean up sign_target_files_apks.py.
Remove the obsolete workaround for API 24. Also make it pylint clean.

Test: 1. Sign a target_files.zip and get identical results.
      2. `pylint --rcfile=pylintrc sign_target_files_apks.py` gives 10.00/10.

Change-Id: I21736f959c5182486fd8ccebea9bbc594edef9fb
2016-09-30 15:29:09 -07:00
Tao Bao
8145c70f2b Merge "releasetools: Fix the reference to OPTIONS.info_dict." am: 0a5b59e78b am: 8b6d59e110
am: b00dbd1b11

Change-Id: Ib31adf3e30f4b9077e14c577ffdc5791e89b746b
2016-09-30 17:59:31 +00:00
Tao Bao
b31b94e159 releasetools: Fix the reference to OPTIONS.info_dict.
sign_target_files_apks.py calls common.GetBootableImage() but without
calling 'OPTIONS = common.OPTIONS' first. In common.GetBootableImage(),
we should use the local info_dict parameter instead of OPTIONS.info_dict.

Test: sign_target_files_apks.py generates signed-TF.zip successfully.

Change-Id: Ia3d32b88691c26e5fb98feea709e3e3c3eb70fdb
2016-09-29 22:04:50 -07:00
David Zeuthen
27ac0e34fe resolve merge conflicts of c36ab42 to nyc-mr1-dev-plus-aosp
Change-Id: Iaf78fb6c723955fd92cc64194cb32bba73a9dafb
2016-09-29 14:25:03 -04:00
David Zeuthen
8ee6d119d1 Merge "Update for new Android Verified Boot (AVB)." 2016-09-29 17:48:40 +00:00
David Zeuthen
2ce63edab7 Update for new Android Verified Boot (AVB).
This updates the build system for the new Android Verified Boot
codebase. As this is based on Brillo Verified Boot, this change replaces
the existing BVB support.

Android Verified Boot is enabled by the BOARD_AVB_ENABLE variable

 BOARD_AVB_ENABLE := true

This will make the build system create vbmeta.img which will contain a
hash descriptor for boot.img, a hashtree descriptor for system.img, a
kernel-cmdline descriptor for setting up dm-verity for system.img and
append a hash-tree to system.img.

Additionally, the descriptors are left in boot.img and system.img so a
third party can create their own vbmeta.img file linking - using the
option --chain_partition - to these images. If this is not needed
footers can be erased using the 'avbtool erase_footer' command. It's
also harmless to just leave them in the images.

By default, the algorithm SHA256_RSA4096 is used with a test key from
the AVB source directory. This can be overriden by the
BOARD_AVB_ALGORITHM and BOARD_AVB_KEY_PATH variables to use e.g. a
4096-bit RSA key and SHA-512:

 BOARD_AVB_ALGORITHM := SHA512_RSA4096
 BOARD_AVB_KEY_PATH := /path/to/rsa_key_4096bits.pem

To prevent rollback attacks, the rollback index should be increased on a
regular basis. The rollback index can be set with the
BOARD_AVB_ROLLBACK_INDEX variable:

 BOARD_AVB_ROLLBACK_INDEX := 5

If this is not set, the rollback index defaults to 0.

The variable BOARD_AVB_MAKE_VBMETA_IMAGE_ARGS can be used to specify
additional options passed to 'avbtool make_vbmeta_image'. Typical
options to be used here include '--prop', '--prop_from_file', and
'--chain_partition'.

The variable BOARD_AVBTOOL_BOOT_ADD_HASH_FOOTER_ARGS can be used to
specify additional options passed to 'avbtool add_hash_footer' for
boot.img. Typical options to be used here include '--hash_algorithm' and
'--salt'.

The variable BOARD_AVBTOOL_SYSTEM_ADD_HASHTREE_FOOTER_ARGS can be used
to specify additional options passed to 'avbtool add_hashtree_footer'
for systems.img. Typical options to be used here include
'--hash_algorithm', '--salt', and '--block_size'.

BUG=31264226
TEST=Manually tested on edison-eng by inspecting {boot, system,
  vbmeta}.img in out/ directory as well as their counterparts in
  the IMAGES/ directory of edision-target_files-eng.zeuthen.zip

Merged-In: Ic9a61cfc65c148b12996e57f04da5432eef6b982

Change-Id: I97042655bca15e7eac899f12c5bada2f6184d307
2016-09-29 11:19:36 -04:00
Tao Bao
7f5f23160d Merge "Analyze unchanged blocks in odex files." am: 6e12b72966 am: 0493339a1f
am: 4fbdde5150

Change-Id: Ibbd1b92b81249b7f11de1ba166ff04ee7ba73b88
2016-09-28 17:18:15 +00:00
Tao Bao
08c8583aea Analyze unchanged blocks in odex files.
In BBOTA, we generate patches based on _all_ the blocks of a pair of
input files (src and tgt). For security incremental OTAs, one common
pattern is that only a few blocks are changed in odex files (e.g.
headers). We don't really need to stash/patch the unchanged blocks.

This CL analyzes the unchanged blocks in odex files and computes the
diff for the changed blocks only. It reduces the OTA install time by
about 25% to 40% in our experiments, by paying an increase of 5% to 30%
OTA generation time cost.

Bug: 31570716
Test: Generate an incremental and apply on device.

Change-Id: If842c1afeff6894a3d27eb60b7e8f65a179b7977
2016-09-23 12:15:23 -07:00
Tao Bao
f71719b635 Merge "Add a script that verifies OTA package signature." am: 3b0cb6e842 am: 3db2c7fedd
am: e3ca43013f

Change-Id: I04d5da95fe3299b2f8fb378cfa0b27e69333d1da
2016-09-19 17:42:36 +00:00
Tao Bao
9c63fb59bd Add a script that verifies OTA package signature.
Currently it supports verifying packages signed with RSA algorithms
(v1-v4 as in bootable/recovery/verifier.cpp). No support for ECDSA (v5)
signed packages yet.

$ ./build/tools/releasetools/check_ota_package_signature.py \
    bootable/recovery/tests/testdata/testkey_v1.x509.pem \
    bootable/recovery/tests/testdata/otasigned_v1.zip

Package: bootable/recovery/tests/testdata/otasigned_v1.zip
Certificate: bootable/recovery/tests/testdata/testkey_v1.x509.pem
Comment length: 1738
Signed data length: 2269
Use SHA-256: False
Digest: 115e688ec3b77743070b743453e2fc6ce8754484

VERIFIED

Bug: 31523193
Test: Used the tool to verify existing packages (like above).

Change-Id: I71d3569e858c729cb64825c5c7688ededc397aa8
2016-09-16 15:08:40 -07:00
Sami Tolvanen
3d207c8889 Merge "build_image: add padding to match partition size" am: ef5751c1ec am: 5875285ecf
am: e261a2dc20

Change-Id: Ib396abb9f6707c984b2888b8b5a265d8ebcf76be
2016-09-02 18:24:43 +00:00
Sami Tolvanen
433905f50e build_image: add padding to match partition size
For some partition sizes, we currently build an image that's 1-2
blocks smaller than the actual partition, which causes fs_mgr to
not find metadata. This change adds padding to FEC metadata that
correctly positions the metadata header at the end.

Bug: 28865197
Change-Id: Ie0e044715a9c5ae8ba395e7d2ff9fbd7cffc0b4c
2016-09-02 10:20:02 -07:00
Tianjie Xu
94a41630ef resolve merge conflicts of 8532aa4 to nyc-mr1-dev-plus-aosp
Change-Id: I104192d50552612d10e418791515a834266a8262
2016-08-24 16:28:23 -07:00
Treehugger Robot
a7adc06e68 Merge "Don't regenerate userdata.img when signing" 2016-08-24 21:10:08 +00:00
Tianjie Xu
b48589af57 Don't regenerate userdata.img when signing
The userdata.img and cache.img entries are not useful in signed builds;
because fastboot doesn't look at these two entries in the *img.zip when
flashing a device. And they aren't used elsewhere. Therefore, skip
building the image files for them when signing the target files with
sign_target_files_apks. Also, add an option "--is_signing" to avoid
adding these two images when we call add_img_to_target_files.

Change-Id: I39ba91a86d9a856d7d01771f6d1403dbf21f2011
Test: Run sign_target_files_apks on a target file and userdata/cache.img doesn't not generate.
Bug: 30642470
2016-08-24 12:50:35 -07:00
Tao Bao
ae3fdd2436 Merge "releasetools: Allow creating target_files with ZIP64." am: 794c69e603 am: cfab69eb32
am: 59c8c55611

Change-Id: Ibf5b7b4abbb4477fa1d23f15af443ad66772e906
2016-08-22 19:18:21 +00:00
Tao Bao
cfab69eb32 Merge "releasetools: Allow creating target_files with ZIP64."
am: 794c69e603

Change-Id: I94eac12de16cac4e4034d933f83fbfa61ca20591
2016-08-22 19:09:55 +00:00
Tao Bao
9c84e50f4c releasetools: Allow creating target_files with ZIP64.
add_img_to_target_files.py fails when the target_files.zip is over 4GiB
when adding IMAGES/ folder. Specify the flag to allow creating
target_files.zip with ZIP64 extension.

Other zip artifacts (-img.zip, -ota.zip etc) remain in non-ZIP64 format.
zip2zip is not affected, which still creates non-ZIP64 zips even when
copying from target_files in ZIP64.

Bug: 30961841
Test: "make dist" with large system image and check the artifacts.

Change-Id: I0568745f01ef8f0239081f783eac92288d4fdd84
2016-08-22 10:40:24 -07:00
Tianjie Xu
b4e4adc195 Merge "Replace META/care_map.txt when signing target files" am: 7b11cacbba am: bf5e3d168d
am: 4f4fb6ce45

Change-Id: If8dd9a34ad92a8d951332376273735573ea8a4c6
2016-08-12 18:13:06 +00:00
Tianjie Xu
bf5e3d168d Merge "Replace META/care_map.txt when signing target files"
am: 7b11cacbba

Change-Id: Ia8e888a0f330d00f6518e352356ca480819586db
2016-08-12 18:06:41 +00:00
Tianjie Xu
4f09900e67 Replace META/care_map.txt when signing target files
Do not copy the "META/care_map.txt" from the source zipfile when
signing the target files with sign_target_files_apks. Because we'll
generate a new care_map after rebuilding the system/vendor images;
and we'll write the new "META/care_map.txt" to the signed-target-file.

Change-Id: I6919cfdf8314a4084b5f612a9c89469f391486a4
Test: Run sign_target_files_apks locally, and the entry is updated.
Bug: 30812253
2016-08-11 18:04:27 -07:00
Tao Bao
6a0451f4e1 Merge "releasetools: Update the path to /default.prop for A/B devices." am: 6d64ff1bf7 am: 6eb5564025
am: cd6ce6182b

Change-Id: I53fffa12f748ebf97462cd0f5bc70d4478e027d5
2016-08-11 20:49:12 +00:00
Tao Bao
6eb5564025 Merge "releasetools: Update the path to /default.prop for A/B devices."
am: 6d64ff1bf7

Change-Id: I1deeff5d22454c1fae5b450dbf2e6a35ffbc6227
2016-08-11 20:42:34 +00:00
Tao Bao
28e2fa1726 releasetools: Update the path to /default.prop for A/B devices.
On A/B devices (i.e. system_root_image="true"), /default.prop is
packaged at ROOT/default.prop (as opposed to BOOT/RAMDISK/default.prop
for non-A/B devices). Update the path so that we handle properties like
ro.bootimage.build.fingerprint properly.

The one for recovery is not affected, which stays at
BOOT/RAMDISK/default.prop for A/B devices and gets updated correctly.

Bug: 30811237
Test: Verify the property in the generated signed-TF.zip.
Change-Id: Id201a042d7ea988a64f89c6d04f43326a9851e27
2016-08-11 11:28:51 -07:00
Alex Deymo
e8fbea342f Merge changes from topic 'ab_sideload' am: 3750ed3e17 am: 93ff77ad1a
am: 820bc82bed

Change-Id: I0dcac2e3cdbc3114a88d7ca49054a15bd9ccb292
2016-08-10 05:07:24 +00:00
Alex Deymo
93ff77ad1a Merge changes from topic 'ab_sideload'
am: 3750ed3e17

Change-Id: I4afbecc40068c7a87c7532b22a404319c24e2c05
2016-08-10 05:00:57 +00:00
Alex Deymo
b3e8ce6d1d Replace OTA sideload verification key when signing A/B devices.
The update-payload-key is used by update_engine_sideload from recovery
to verify an update payload.

Bug: 27178350
Change-Id: I7a0a307ae565e5e9cbf2c9b58fbcc055e87771ce
2016-08-04 16:06:12 -07:00
Tao Bao
9220a725bd Merge \\\"Handle multiple APKs from the same package in check_target_files_signatures.py.\\\" am: 653980bfe4 am: 88f7e6a867
am: c57f2e6f65

Change-Id: I8a634684509bbb91f6311a1e8ffb0e95174af5e5
2016-07-28 04:46:47 +00:00