Commit Graph

5374 Commits

Author SHA1 Message Date
Yifan Hong
92fb231105 build_image.py: remove extra new lines in error message
Test: m dist
Change-Id: I7908e4eb0a9bbcb187e1fa6d15f606537d69864d
2018-06-22 15:05:46 -07:00
Treehugger Robot
3967e0eb99 Merge changes from topic "board_partition_reserved_size"
* changes:
  build_image.py recognize BOARD_VENDORIMAGE_PARTITION_RESERVED_SIZE
  build_image.py recognize BOARD_SYSTEMIMAGE_PARTITION_RESERVED_SIZE
2018-06-22 20:10:47 +00:00
Yifan Hong
749062d379 build_image.py recognize BOARD_VENDORIMAGE_PARTITION_RESERVED_SIZE
Test: `make vendorimage`

Test: `make vendorimage` with the following:
    - install a large file to vendor image
    fails as expected (because _PARTITION_SIZE is exceeded)

Test: `make vendorimage` with the following:
    - set PRODUCT_USE_LOGICAL_PARTITIONS to true
    - set BOARD_VENDORIMAGE_PARTITION_RESERVED_SIZE
    fails as expected (BOARD_VENDORIMAGE_PARTITION_SIZE needs
    to be undefined)

Test: `make vendorimage` with the following:
    - install a large file to vendor image
    - set PRODUCT_USE_LOGICAL_PARTIIONS to true
    - add a small BOARD_VENDORIMAGE_PARTITION_RESERVED_SIZE
    - remove BOARD_VENDORIMAGE_PARTITION_SIZE
    build succeeds.

Bug: 79106666

Change-Id: Ica8fdce64e8f37d91e66e3d49c0c74fadd15a832
2018-06-22 10:25:54 -07:00
Yifan Hong
bbcba1e353 build_image.py recognize BOARD_SYSTEMIMAGE_PARTITION_RESERVED_SIZE
- Copy "use_logical_partitions" to *_image_info.txt before sending
  it to build_image.py, so that the script can use this variable.

- build_image.py emits an additional properties file to inform
  the build system about the system image size.

Test: `make systemimage`

Test: `make systemimage` with the following:
    - install a large file to system image
    fails as expected (because _PARTITION_SIZE is exceeded)

Test: `make systemimage` with the following:
    - set PRODUCT_USE_LOGICAL_PARTITIONS to true
    - set BOARD_SYSTEMIMAGE_PARTITION_RESERVED_SIZE
    fails as expected (BOARD_SYSTEMIMAGE_PARTITION_SIZE needs
    to be undefined)

Test: `make systemimage` with the following:
    - install a large file to system image
    - set PRODUCT_USE_LOGICAL_PARTIIONS to true
    - add a small BOARD_SYSTEMIMAGE_PARTITION_RESERVED_SIZE
    - remove BOARD_SYSTEMIMAGE_PARTITION_SIZE
    build succeeds.

Test: same for systemotherimage

Bug: 79106666

Change-Id: I574062882acd1ecd633ac38c5a8c5351b90a32d8
2018-06-22 10:25:54 -07:00
Tao Bao
93c2a01268 releasetools: Skip signing APKs with given prefixes.
We may pack prebuilts that end with ".apk" into target_files zip, via
PRODUCT_COPY_FILES. META/apkcerts.txt won't contain the cert info for
such files, and we want to keep them as is while signing, despite of the
".apk" extension.

This CL adds "--skip_apks_with_path_prefix" option to
sign_target_files_apks.py. APKs with matching prefixes will be copied
verbatim into the signed images. The prefix should match the entry names
in the target_files (e.g. "SYSTEM_OTHER/preloads/"). The option may be
repeated to specify multiple prefixes.

Note that although we may skip signing an APK file with "-e ApkName=".
This would skip *all* the APK files with the matching basename.
"--skip_apks_with_path_prefix" allows matching the exact prefix.

For example:
$ ./build/make/tools/releasetools/sign_target_files_apks.py     \
    --skip_apks_with_path_prefix SYSTEM_OTHER/preloads/         \
    --skip_apks_with_path_prefix PRODUCT/prebuilts/PrebuiltApp1 \
    --skip_apks_with_path_prefix VENDOR/app/PrebuiltApp2.apk    \
    target_files.zip                                            \
    signed-target_files.zip

Bug: 110201128
Test: Run the command above and check the logs.
Test: `python -m unittest test_sign_target_files_apks`
Change-Id: I7bd80b360917cef137cf1e7e8cfa796968831f47
2018-06-20 13:49:57 -07:00
Tao Bao
11f955c506 releasetools: Factor out the check for (compressed) APK file.
Test: Run sign_target_files.py to sign a target_files.zip.
Test: `python -m unittest test_sign_target_files_apks`
Change-Id: Ie795d1bce7bae6af427832283e3d10bfecad80c5
2018-06-19 14:50:23 -07:00
Tom Cherry
b6b2d49e23 Merge "Always create /vendor/etc/{passwd,group}" 2018-06-15 17:04:16 +00:00
Tom Cherry
53940fd2ba Always create /vendor/etc/{passwd,group}
Previously we only created these if the device set
TARGET_FS_CONFIG_GEN, however there are now other targets that want to
depend on these.  Instead of having those targets conditionally depend
on them, we always create them, defaulting to blank contents (by
reading /dev/null for TARGET_FS_CONFIG_GEN).

Test: builds succeed
Change-Id: Ie95286f5a800d891022eb66cd6fefcc967000c2e
2018-06-14 13:29:32 -07:00
Treehugger Robot
6eb060785b Merge "fs_config_generator: use vendor shell for vendor AID" 2018-06-14 03:28:32 +00:00
Tao Bao
50869f203a Merge "releasetools: Explicitly pass the SHA-1 of source boot image to PatchCheck." 2018-06-13 17:20:36 +00:00
Tao Bao
6d8ec5803b releasetools: Explicitly pass the SHA-1 of source boot image to PatchCheck.
Although the SHA-1 of the source boot image has been encoded in the
filename, the updater code currently doesn't extract that and would fail
to find a matching backup from /cache. This could lead to corrupt
install, if it's interrupted while patching the boot image.

Bug: 80630342
Test: Generate an non-A/B incremental package. Verify the script.
Change-Id: Ie2b1a90836c3ccb4758e1a2d9b69a88de94ae0c7
2018-06-12 15:04:08 -07:00
Andreas Gampe
b9dc23a4d8 Tools: Update warn.py for Errorprone 2.3.1
Note: this uses the default severity in Errorprone, modulo
forced-ERROR settings in Android.

Bug: 72971960
Test: manual
Change-Id: I6d8b855b88289b763b2a98e8685627bf1163989d
2018-06-08 10:24:35 -07:00
Andreas Gampe
2e987af407 Tools: Sort Errorprone warn.py entries by name
To improve updateability.

Bug: 72971960
Test: none
Change-Id: I7ab7cdfe53518b29d036f418fe66aef69b5e409a
2018-06-08 10:22:59 -07:00
Chih-Hung Hsieh
9be27764c0 Fix gpylint warnings.
* This file should be gpylint clean so it can be easily copied
  and pass presubmit checks in google repository.

Test: gpylint warn.py
Change-Id: I5391f19c7a0141bc211f5b3924f48b345bcb03ab
2018-06-07 10:51:47 -07:00
Wei Wang
77e329a6a6 fs_config_generator: use vendor shell for vendor AID
Bug: 109748241
Test: Build and check generated file
Change-Id: Ib6bca77aa906a1aa9ed41c51bddb2871ecb66e1a
2018-06-06 15:14:42 -07:00
Wei Wang
3b5d3b1525 Remove comment line in generated passwd file
The man page for the password file (man 5 passwd) does not mention any
provisions for comment line other than comment field

Bug: 109748241
Test: Build and check generated file
Change-Id: I2912d0844cf3f077184129d1a0121b50b09ef1e3
2018-06-05 15:34:58 -07:00
Treehugger Robot
991bbccf95 Merge "Remove references to common_time" 2018-06-05 10:33:13 +00:00
Chih-Hung Hsieh
9e9355dac6 Recognize more clang-tidy compiler warnings.
* Some are from clang compiler and Android makefile checker.

Test: run with build.log
Change-Id: I371f08f3ae956e9425515b2cd67c5af3d0425b89
2018-06-04 11:32:32 -07:00
Neil Fuller
ce483225bf Remove references to common_time
The code is unused and is being removed.

Bug: 80462439
Test: build / boot
Change-Id: I812bff4fdc061b060633949b7087ab9c951fa7e0
2018-06-04 15:38:21 +01:00
Tao Bao
d32936d657 releasetools: Skip checking files with non-monotonic ranges.
Bug: 79951743
Test: Run validate_target_files.py on target_files.zip that has
      non-monotonic ranges listed in IMAGES/system.map.
Change-Id: I28d3ca6972d361362589cd51e60731af9994a551
2018-05-17 19:45:16 -07:00
Chih-Hung Hsieh
c5c443c756 Do not use o=ALL_EMAILS parameter.
For security reason, this REST API parameter needs account modification
permission to get all email addresses. Now changed to ask only account
number to verify an email address.

Bug: 79863374
Test: test with existing OWNERS
Change-Id: Ic913b7ad96a69c35d1d91e5871f4c5636e73533d
2018-05-16 11:44:50 -07:00
Tom Cherry
e64694118a Merge "Require a vendor_ prefix for users/groups in /vendor/etc/{passwd,group}" 2018-05-15 22:36:34 +00:00
Tao Bao
3d840c6b98 Merge "releasetools: Use "ota-downgrade=yes" for --override_timestamp." 2018-05-15 16:58:11 +00:00
Jiyong Park
5e2056b906 Merge "Convert apicheck tool build to Soong." 2018-05-15 05:30:41 +00:00
Jiyong Park
1c671fd7dd Convert apicheck tool build to Soong.
Bug: b/78034256
Test: m clean && m checkapi
Change-Id: Iaced7335714138730349bbea8a993705f197ae77
2018-05-15 11:38:44 +09:00
Tom Cherry
2d197a1e19 Require a vendor_ prefix for users/groups in /vendor/etc/{passwd,group}
Bug: 79528966
Test: successful build with vendor_ prefixed users/groups
Test: expected build failure when not using vendor_ prefixed users/groups
Change-Id: If006c70178aa5bdcc9521a06ef8df2500f70bbb9
2018-05-14 13:16:50 -07:00
Tao Bao
138df4ef77 Merge "releasetools: Skip validating non-sparse images." 2018-05-14 17:37:53 +00:00
Tao Bao
63e2f49fdd releasetools: Skip validating non-sparse images.
Targets can define 'TARGET_USERIMAGES_SPARSE_EXT_DISABLED := true' to
generate non-sparse system images, but validate_target_files.py doesn't
work with such images. This CL adds a workaround to temporarily skip the
file consistency check for such images.

Bug: 79616357
Test: Run validate_target_files.py on a target_files.zip that's not
      using sparse image.
Test: Run validate_target_files.py on marlin target_files.zip (which
      uses sparse image).
Change-Id: I1f4066c5b3fec595b10cab10283d62c1c5a6c624
2018-05-11 23:38:46 -07:00
Daniel Colascione
334ece99dd Add zip hint generation support to signapk tool
Test: unzip -q -c myapp.apk.signed pinlist.meta | od --endian=big -w8 -tx4
Bug: 79259761
Bug: 65316207
Change-Id: I71c01ac24e93afe75f60697a9849e1dd35e1b49d
2018-05-10 14:39:20 -07:00
Tao Bao
faa8e0b7e9 releasetools: Use "ota-downgrade=yes" for --override_timestamp.
We used to write a fake post-timestamp into package metadata for
"--override_timestamp", which allows a package to be pushed based on
the fake timestamp. This CL stops doing that by using the designated
"ota-downgrade=yes" instead, but keeping the "--override_timestamp" flag
for backward compatibility.

Now both "--override_timestamp" and "--downgrade" allow pushing packages
with downgrade timestamps. The only differenece is that "--downgrade"
enforces a data wiping, because it's intended a real downgrade (based on
Android version or security patch level). "--override_timestamp" serves
the path of a legit "upgrade", but unfortunately with build timestamps
in reverse order (e.g. the two builds are cut from different branches).

With this CL, we write "post-timestamp" to all packages.

Bug: 77811049
Test: Generate an incremental pakcage with "--override_timestamp".
Test: Generate an incremental pakcage with "--downgrade".
Test: python -m unittest test_ota_from_target_files
Change-Id: I4ae4ce77cb600a06ffd9b15b6d4c730314aa0b79
2018-05-09 10:23:14 -07:00
Zhomart Mukhamejanov
603655f5b5 releasetools: Make _GetPropertyFilesString public
ota_from_target_files.py:

Rename PropertyFiles._GetPropertyFilesString to
PropertyFiles.GetPropertyFilesString.

Currently only PropertyFiles.Compute and
PropertyFiles.Finalize gives access to _GetPropertyFilesString.
But Compute force sets reserve_space=True,
and Finalize requires reserved_length.

_GetPropertyFilesString is useful method that can
be used outside releasetools.

One of the usage is in bootable/recovery/updater_sample/.

Test: python -m unittest test_ota_from_target_files
Change-Id: I2cc44ec46a0e68ba071531b003af8cdbfe90e588
Signed-off-by: Zhomart Mukhamejanov <zhomart@google.com>
2018-05-04 12:35:09 -07:00
Bryan Henry
7febcdf75f Revert "Support regenerating partition table with bpttool in sign_target_files_apks"
This reverts commit 2a40cc6996.

Reason for revert: Broke a bunch of builds, platform/system/tools/bpt must be missing from more manifests.

Change-Id: I4c367a1fec5c62c247544227e9d3b596227e31da
2018-04-27 19:02:51 +00:00
Bryan Henry
f130a239a1 Write combined bpt definition directly into META/partition-table.bpt
...instead of copying from IMAGES/partition-table.bpt during
sign_target_files_apks. This addresses a comment on go/oag/665565, but
is done as a separate change so that it can be cherry-picked later (or
not at all) to downstream branches to avoid breaking signing of existing
target-files zips.

Bug: 72837107
Test: Local sign_target_files_apks run of locally built target-files
Change-Id: Id14c859eefe075fc56b15869f2f21c570eb07e65
2018-04-26 12:43:27 -07:00
Bryan Henry
2a40cc6996 Support regenerating partition table with bpttool in sign_target_files_apks
For Android Things targets (or any other target which has
BOARD_BPT_INPUT_FILES defined), add_img_to_target_files will generate a
partition-table.img using bpttool. It also adds the final combined .bpt
definition file into target-files in IMAGES/partition-table.bpt.

When we're signing using sign_target_files_apks, add_img_to_target_files
needs to regenerate the partition table, but META/misc_info.txt still
contains the original list of bpt input files from the build that aren't
available. This change extracts the final bpt from the input
target-files, adds it to META/ in the output target-files, and then
updates the board_bpt_input_files property to point to it.

Bug: 72837107
Test: Local sign_target_files_apks run of locally built target-files
Change-Id: Id79125208f31c78b1ac2079172f9c91a9203849b
2018-04-26 12:42:59 -07:00
Bryan Henry
69d3feb23a Don't suppress stderr for 'avbtool make_vbmeta_image'
Output is useful for debugging.

Bug: 72837107
Test: Local sign_target_files_apks run of locally built target-files
Change-Id: I5c27fcc86fa3a51080e0502eb8f5f01a40b033c3
2018-04-26 12:42:40 -07:00
Tao Bao
30df8b4e2e releasetools: Group the option descriptions in ota_from_target_files.py.
This CL separates the options into three groups (excluding the global
options provided via common.py).
 - Non-A/B OTA specific options;
 - A/B OTA specific options;
 - Common options that apply to both.

It mostly reshuffles the lines, with minor change to "--verify" that
removes the obselete remounting behavior (which doesn't apply to
block-based OTA).

Hopefully this makes the expected behavior of some options less
confusing.

Test: `build/make/tools/releasetools/ota_from_target_files.py`
Change-Id: I194ea52c4f7d6a3c7f34531abbcf3fdc7b7f4fa8
2018-04-23 16:00:14 -07:00
Tao Bao
332a96b8b1 releasetools: Default journal size to 0 for system images.
This CL defaults the journal size to 0 for system images (i.e. system,
vendor, system_other, oem, product). We used to do this by explicitly
defining BOARD_{SYSTEM,VENDOR,OEM,PRODUCT}IMAGE_JOURNAL_SIZE to 0 in
device-specific BoardConfig. Targets can still specify a non-zero
journal size as needed.

With this CL, marlin/sailfish gets 32MB / 16MB free space back for
system and vendor respectively.

Bug: 75975085
Test: `m dist`. Check the journal size in the generated images.
Change-Id: Ib3185d07c49be2b4f0fac5fe17ec1a82093ba0c0
(cherry picked from commit 965542fe69)
2018-04-22 22:04:21 -07:00
Patrick Tjin
3f5f9934e9 build_image: default extfs reserved size to 0 on RO partitions.
Defaults the reserved blocks for root to 0% on read only partitions
(system, system_other, vendor, oem).  It also adds support for
explicitly specifying the extfs reserved percentage via
BOARD_{SYSTEM,VENDOR,OEM,PRODUCT}IMAGE_EXTFS_RSV_PCT.

This eventually translates down to the -m option for mkfs.

Removing the reserved space can save at least  5% from the default.

  dumpe2fs system:
    Reserved blocks uid:      0 (user root)
    Reserved blocks gid:      0 (group root)

Bug: 75975085
Test: Build, verify reserved space is changed accordingly
Change-Id: I212d82741908b636db0d658a1c4847bbaadfd5ba
(cherry picked from commit 5ff758799c)
2018-04-22 22:04:21 -07:00
Tao Bao
5277d1015f releasetools: Stop copying images from RADIO/ to IMAGES/.
We've added support in brillo_update_payload that allows additionally
looking for images under RADIO/ in the given target_files zips [1]. This
avoids having duplicate radio images in target_files zips.

Also adjust the unittest in test_ota_from_target_files.py to cover this
path.

As a result of this CL, the radio images will no longer appear in the
image archive (i.e. <target>-img.zip) as well - they are less useful
anyway because we have packed only the _updatable_ pieces that are part
of full bootloader/radio images.

Bug: 77218220
Test: `python -m unittest test_ota_from_target_files`
Test: `python -m unittest test_add_img_to_target_files`
Test: `m dist` produces the same full OTA package
Test: Build marlin-userdebug in internal branch. Check the image zip.
Change-Id: I05579480f0bb9ab90aaeecf75969ee29b6904ad6
2018-04-20 10:20:42 -07:00
Treehugger Robot
c904a89c93 Merge "Remove the support for BRILLO_VENDOR_PARTITIONS." 2018-04-18 21:32:35 +00:00
Tao Bao
36d7c5666d Remove the support for BRILLO_VENDOR_PARTITIONS.
It was initially introduced in commit
2e735ca34e, where it packs additional
vendor images into target_files zip in order to generate OTAs. We can
acheive the same goal with INSTALLED_RADIOIMAGE_TARGET, which is the way
being actively used across all targets, including IoT (the former
Brillo) targets.

Bug: 78201540
Test: `m dist` with aosp_marlin-userdebug
Test: Code search shows no active user of BRILLO_VENDOR_PARTITIONS.
Test: `python -m unittest test_add_img_to_target_files`
Change-Id: I8803d5377b5a39304a701cceafb243f9a228347d
2018-04-18 09:50:47 -07:00
Treehugger Robot
2fb914c56b Merge "Remove make/tools/droiddoc/test/stubs/Android.mk" 2018-04-16 23:30:56 +00:00
Nan Zhang
d196ee4019 Remove make/tools/droiddoc/test/stubs/Android.mk
This Android.mk hasn't been changed since the initial-contribution.

And the droiddoc_templatedir:tools/droiddoc/templates-google is gone now.

Test: manual
Bug: b/70351683
Change-Id: I6a364a96c02218cdc012c6350844d562b0e6c142
2018-04-12 17:00:25 -07:00
Dan Willemsen
c7f16e4fd4 Add USER to the docker environment
Apparently Jack needs this, and my previous testing didn't show it.

Test: build & run image; echo $USER
Change-Id: I66766b230f2f3e0762a49efc9bf2c6212b2b2c4d
2018-04-12 16:41:17 -07:00
Tom Cherry
946b5c5e7f Merge "Convert liboemaids to oemaids_headers as a headers only library" 2018-04-04 23:02:34 +00:00
Tom Cherry
7a95c15c15 Convert liboemaids to oemaids_headers as a headers only library
Bug: 77298768
Test: test that config.fs AIDs are visible through this library
Change-Id: Ifbf3276212ea6904533ac23021bfce29d9a3c5d9
2018-04-04 12:29:14 -07:00
Hridya Valsaraju
e74a38bc6d Allow addition of recovery DTBO to recovery image
Non-A/B devices need to include the DTBO image
within the recovery partition to be self-sufficient
and prevent OTA failures.

Test: Ran 'm dist' and verified that the DTBO image
was included in recovery.img using unpack_bootimg.
Also ran 'make' and verified that the DTBO image was
included in recovery.img using unpack_bootimg.
Also tested that aosp_angler-userdebug could
boot into recovery(including a fake dtbo.img
in recovery image).

Bug: 74763691

Change-Id: I38c9c395c95d21f4da42cfa646063bd4416f6bd8
2018-03-26 22:54:55 +00:00
Treehugger Robot
f18ba63095 Merge "releasetools: Fix an issue in GetMinSdkVersion." 2018-03-23 18:08:40 +00:00
Treehugger Robot
f3e39e5878 Merge "releasetools: Capture and dump stdout/stderr outputs on errors." 2018-03-23 01:05:27 +00:00
Tao Bao
f47bf0fecf releasetools: Fix an issue in GetMinSdkVersion.
The following is a buggy pattern that won't capture anything into err.
The issue is benign, since a failed run would be eventually captured by
a subsequent check.

  p = Run(["aapt", ...], stdout=subprocess.PIPE)
  output, err = p.communicate()
  if err:
    raise ...

This CL changes the error detection to be based on the return code from
aapt. It also adds some sanity test to ensure the call to aapt works.
The test app is built from AOSP com.android.cts.ctsshim (chosen mostly
because of its small size).

Test: python -m unittest test_common
Change-Id: I337f141bd0fc5f0801dfc628c601b88b7640789c
2018-03-22 10:27:04 -07:00