Commit Graph

6708 Commits

Author SHA1 Message Date
Logan Chien
0e53d887d6 Add prebuilt ELF binaries checker
This commit introduces a prebuilt ELF binaries checker.  The checker
will check:

1. Whether all DT_NEEDED shared libraries are specified in
   `shared_libs` (Android.bp) or `LOCAL_SHARED_LIBRARIES` (Android.mk).

2. Whether all undefined symbols in the prebuilt binary can be resolved
   to defined symbols exported by its dependencies.

This ensures that prebuilt binaries won't silently become ABI
incompatible.

To check the prebuilt binaries, all of the dependencies must be
specified in `shared_libs` (Android.bp) or `LOCAL_SHARED_LIBRARIES`
(Android.mk).

If your prebuilt binaries cannot be checked for some reason, you may add
the following property to Android.bp:

    check_elf_files: false,

Or, add the following setting to Android.mk:

    LOCAL_CHECK_ELF_FILES := false

Bug: 119084334
Test: CHECK_ELF_FILES=true make check-elf-files
Change-Id: I523d3083f22fd4053c096d26f61f8375800281c8
2019-01-21 21:35:01 +08:00
Yifan Hong
45433e44dd Really support removing / adding partitions.
- When removing a partition, BlockDifference() object
will have tgt=EmptyImage(). Fix the asserts accordingly.
Also, BlockDifference object now allow tgt=None case.

- When adding a partition, BlockDifference() object
will have src=None. Fix the asserts accordingly.

Also, add unit tests to DynamicPartitionsDifference.

Test: create incremental OTA
Test: test_common.DynamicPartitionsDifferenceTest
Bug: 111801737

Change-Id: I3a35378ecf93111b8f44545cff6ae9696b6b4851
2019-01-18 17:09:33 -08:00
Mark Salyzyn
2377996a50 Merge "build_image: Increase spare inodes to 12." am: aa58210dd0 am: fb37e0b6a2
am: 68fed3ce30

Change-Id: I4973b32d8799202eaee16075cab69c48dde4feca
2019-01-17 18:33:29 -08:00
Mark Salyzyn
68fed3ce30 Merge "build_image: Increase spare inodes to 12." am: aa58210dd0
am: fb37e0b6a2

Change-Id: If15f0120fd63197b510095492c51b11c48b0bcd1
2019-01-17 18:24:20 -08:00
Mark Salyzyn
60fa99d491 build_image: Increase spare inodes to 12.
Test: builds device with DAP
Change-Id: I816a7da54bbe23d4eb1140ac0d101c0fee175f0a
2019-01-17 11:30:12 -08:00
Tianjie Xu
42244aac19 Merge "blockimgdiff: selectively convert 'diff' commands to 'new' to reduce stash size" am: ffc87ac0a7 am: 4046b8f741
am: e8be9168fe

Change-Id: Iaf05286b871685bbaa6194c810b528198692a97d
2019-01-17 09:31:06 -08:00
Tianjie Xu
e8be9168fe Merge "blockimgdiff: selectively convert 'diff' commands to 'new' to reduce stash size" am: ffc87ac0a7
am: 4046b8f741

Change-Id: If6ad6cc16e5758f48ed79c3291e77c2321bd5a86
2019-01-17 09:22:55 -08:00
xunchang
b6105dcba3 blockimgdiff: selectively convert 'diff' commands to 'new' to reduce stash size
We cannot simultaneously stash more blocks than the size limit imposed by
the cache size. As a result, some 'diff' commands will be inevitably
converted to new. We used to do this conversion blindly when iterating
through the transfer list. This leads to an unintended large package.

In order to choose the right transfers to convert, we calculate the size
of the compressed data, and build a heuristic about the package size
increase to remove each stash blocks. After the process, the given
package size for the watch device further reduces from 186M->155M.

In some rare cases, the removed stashed blocks don't directly contribute
to the maximum simultaneously stashed size. For example,
stash A: 10 blocks
stash B: 5 blocks
free B: 5 blocks  <-- stash B has been freed before we reach max stashed blocks
stash C: 10 blocks

Converting these blocks lead to an uncertain result. On one hand, patches
are generally smaller than the new data; while on the other hand, the
regenerated graph may have fewer order violation and thus give some size
reduction. But these cases are rare and it seems an overkill to consider all
possible scenarios here.

Bug: 120561199
Test: build non-A/B incrementals and check the size
(p.s. it can be tested on all target files with customed cache threshold)
Change-Id: I599420a91b80f1a1d83d22ee1b336b699050cfb4
2019-01-16 12:59:13 -08:00
Yifan Hong
9780b8c1e4 Merge "Generate OTA for non-A/B devices with dynamic partitions" am: 885787f4d3 am: 4b0f34da5d
am: 7f7e2635cb

Change-Id: I5eb4228d4545f056772e1be5827a4df2ce49c2e4
2019-01-16 11:04:05 -08:00
Mark Salyzyn
33e3c26cfd Merge "build_image: right-sizing at least some spare inode" am: f2f1854ca4 am: f92eb52866
am: 34c0af7a11

Change-Id: Ia5322176bd51b8728dc1a724327616a243de8670
2019-01-16 11:01:37 -08:00
Yifan Hong
7f7e2635cb Merge "Generate OTA for non-A/B devices with dynamic partitions" am: 885787f4d3
am: 4b0f34da5d

Change-Id: I76ad575bf821f693b2879af5926ffd8abe73fc7e
2019-01-16 10:58:14 -08:00
Mark Salyzyn
34c0af7a11 Merge "build_image: right-sizing at least some spare inode" am: f2f1854ca4
am: f92eb52866

Change-Id: I80f5a8b6b53a043ff8e2b1ac0717dd19b43ea981
2019-01-16 10:56:25 -08:00
Yifan Hong
885787f4d3 Merge "Generate OTA for non-A/B devices with dynamic partitions" 2019-01-16 18:42:58 +00:00
Mark Salyzyn
c25b2bf458 build_image: right-sizing at least some spare inode
To address problems creating real tiny filesystem placeholders,
make sure extra spare inodes margin is greater than 0.
For initial estimate we add 4% of total we request at least 8.
For second pass estimate we add 0.2% of total we request at least 1.

We bumped up the margin for zie on the second pass to 0.3% as the
value was too close on one of the builds.

Test: build
Bug: 122328872
Change-Id: I41707bb6fcc8bbfbdda143a9ce62446cad9c1533
2019-01-16 09:54:31 -08:00
Yifan Hong
10c530d20a Generate OTA for non-A/B devices with dynamic partitions
Test: sideload full OTA on cuttlefish
Test: sideload incremental OTA on cuttlefish (that grows
      system, shrinks vendor, and move vendor to group foo)

Bug: 111801737

Change-Id: Ie8a267a90b4df9e9e0a2fbcc1b582ab2e353df52
2019-01-15 16:04:12 -08:00
Mark Salyzyn
588919f1df Merge changes I6457933a,I955aa0fa am: db3c8985be am: 1a34e1ed7e
am: 512ef0475a

Change-Id: I4a6d1a5abbfd002f6cbefc28c1fef1a6a1aaa4bd
2019-01-15 12:51:04 -08:00
Mark Salyzyn
512ef0475a Merge changes I6457933a,I955aa0fa am: db3c8985be
am: 1a34e1ed7e

Change-Id: I616c7b3642ea5a29cc23f3f3fa4959cc81960e79
2019-01-15 12:45:47 -08:00
Mark Salyzyn
db3c8985be Merge changes I6457933a,I955aa0fa
* changes:
  Revert "Relax inode usage estimation slack for DAP right sizing"
  Revert "Increase slack for inode estimation algorithm"
2019-01-15 20:33:50 +00:00
Tao Bao
acb85e8001 Merge "releasetools: OutputFile stages files under input dir." am: 3ab0ca72b5 am: 0d0458a74a
am: aeaabad880

Change-Id: Ic21f09f8132490c2f87d19668a5b87048e748f2e
2019-01-14 13:00:27 -08:00
Tao Bao
aeaabad880 Merge "releasetools: OutputFile stages files under input dir." am: 3ab0ca72b5
am: 0d0458a74a

Change-Id: Icba69dddf0b7a85d9ccd32179164b45be4034c8c
2019-01-14 12:09:54 -08:00
Mark Salyzyn
9f23b89273 Revert "Relax inode usage estimation slack for DAP right sizing"
This reverts commit bd17fab260.

Test: build
Bug: 122328872
Change-Id: I6457933a0dc51827bd86d9c55b02a65999612bcc
2019-01-14 09:44:40 -08:00
Hridya Valsaraju
9aa42f1721 Revert "Increase slack for inode estimation algorithm"
This reverts commit e8e7958c13.

Test: build
Bug: 122328872
Change-Id: I955aa0fa489ab62ec1643127d106925dcb9675ca
2019-01-14 09:44:26 -08:00
Tao Bao
93e7ebe779 releasetools: OutputFile stages files under input dir.
We used to create the intermediate output file as a tempfile, when
adding images to a given zip file. This CL changes OutputFile to write
intermediate files under the given input dir (i.e. OPTIONS.input_tmp
that holds the unzipped files), if the final destination is a zip
archive. This allows image building codes finding the generated images
at consistent locations. The intermediate files will be cleaned up as
part of OPTIONS.input_tmp.

Bug: 122608028
Test: `m dist`
Test: Delete OTA/super*.img from a target_files.zip. Run
      `add_img_to_target_files.py -a` that re-generates split super
      images.
Change-Id: I97903a59fcc0ca5e43bb9b07c3a0b25e9baa92f9
2019-01-13 23:49:25 -08:00
Tao Bao
3de7a10e2f Merge "releasetools: Skip copying split super images while signing." am: 774c0f92bf am: 2db2039202
am: f2cc65ba23

Change-Id: Ide56072bf73d8ada477aa5c03d30bf4a6d915e57
2019-01-11 16:25:18 -08:00
Tao Bao
ee3d514cf0 Merge "releasetools: Update OWNERS." am: dd95cf51fa am: f37455e064
am: 7e0da84aa7

Change-Id: Ib5256c1274255cf1545d13994a3e1795e1d8428c
2019-01-11 16:15:49 -08:00
Tao Bao
f2cc65ba23 Merge "releasetools: Skip copying split super images while signing." am: 774c0f92bf
am: 2db2039202

Change-Id: Ie73f72e7c7eb460156a160361c385aa6c881658e
2019-01-11 15:51:22 -08:00
Tao Bao
7e0da84aa7 Merge "releasetools: Update OWNERS." am: dd95cf51fa
am: f37455e064

Change-Id: Iaed31f353af02242c94ddc98a0d34deb963bb795
2019-01-11 15:48:47 -08:00
Tao Bao
774c0f92bf Merge "releasetools: Skip copying split super images while signing." 2019-01-11 23:18:49 +00:00
Tao Bao
33bf268e0b releasetools: Skip copying split super images while signing.
Bug: 122608028
Test: Run sign_target_files_apks.py on a target-files zip that has split
      super images (e.g. OTA/super_system.img).
Change-Id: Iaf7263790961a897ea3f339f5af6b18cf253b946
2019-01-11 12:39:31 -08:00
Tao Bao
4b2306b41c releasetools: Update OWNERS.
Test: N/A
Change-Id: I0c6ac3cd04cc6034787d4b8b90fbd61349c54655
2019-01-11 12:28:09 -08:00
Mark Salyzyn
7f38ceeb3f Merge changes Ic8ef968e,I6665bd6f am: e093534aba am: 2907f67255
am: 1708ff6094

Change-Id: If603ecc6567ba68dbbce38903a155bf673e17968
2019-01-11 10:06:00 -08:00
Mark Salyzyn
1708ff6094 Merge changes Ic8ef968e,I6665bd6f am: e093534aba
am: 2907f67255

Change-Id: Ib8b338c892acea4842d1f88df442de134f5a1d43
2019-01-11 09:45:53 -08:00
Mark Salyzyn
9f0b91de8b Merge "build_image: hard code set inode size to 256" am: 546550e095 am: a575edb5d9
am: 004b41040d

Change-Id: I6c1fd55a024362e71fbd2f84bf7d42e274eeb9da
2019-01-10 18:33:08 -08:00
Mark Salyzyn
004b41040d Merge "build_image: hard code set inode size to 256" am: 546550e095
am: a575edb5d9

Change-Id: I78681e0db634f2e459cb3e43e7cc174cd7bcc9e4
2019-01-10 18:23:08 -08:00
Mark Salyzyn
6541d0a80e build_image: support non-sparse right-sized images
Allow right-size to support images that are not sparse.
This is in support of cuttlefish.

Test: build
Bug: 122328872
Change-Id: Ic8ef968e750203dffde7044bc6dfc71c1e283158
2019-01-10 14:49:34 -08:00
Mark Salyzyn
60a716fe8c build_image: right-sizing add .2% margin for space and inodes
To address flakiness in image creation, add a .2% margin of safety
for both inodes and space for ext filesystems.

Test: build
Bug: 122328872
Change-Id: I6665bd6fe642291b825dba58dfd09cc55628230f
2019-01-10 14:49:24 -08:00
Mark Salyzyn
c777eaa779 build_image: hard code set inode size to 256
inode-size will jump around based on filesystem size, however
readonly Android system partitions have a specific pattern of
xattr associated with sepolicy and 256 is the most efficient at
absorbing the content.

Test: manual
Bug: 122328872
Change-Id: I06dd6a503067ab6477293b386d56a89dd86b0e83
2019-01-10 14:49:12 -08:00
Dan Willemsen
36beb0cb85 Merge "Add BUILD_USERNAME and BUILD_HOSTNAME" am: 1b8939b946 am: c2230d4684
am: 1a22c2a7e3

Change-Id: Ic8c8fba31e14c5accc9b4b401352e368e620550b
2019-01-10 13:52:32 -08:00
Dan Willemsen
1a22c2a7e3 Merge "Add BUILD_USERNAME and BUILD_HOSTNAME" am: 1b8939b946
am: c2230d4684

Change-Id: I0f4a7b51027d0696f2b6184eec0db3908a7db6a6
2019-01-10 13:47:26 -08:00
Treehugger Robot
1b8939b946 Merge "Add BUILD_USERNAME and BUILD_HOSTNAME" 2019-01-10 21:29:47 +00:00
Tao Bao
183e47ef1a Merge "Don't build retrofit OTA package if vendor.img is unavailable." am: 9f1eda65e6 am: 831fd23482
am: c3a4322c20

Change-Id: I69525f569e7c1c2753f1f7006ab599a3532a7eed
2019-01-07 17:18:23 -08:00
Tao Bao
c3a4322c20 Merge "Don't build retrofit OTA package if vendor.img is unavailable." am: 9f1eda65e6
am: 831fd23482

Change-Id: I3a45c14559c8579b580d8e9a53907a49897544f3
2019-01-07 17:07:56 -08:00
Tao Bao
519d18286e Don't build retrofit OTA package if vendor.img is unavailable.
We used to build retrofit full OTA package as long as
PRODUCT_RETROFIT_DYNAMIC_PARTITIONS is true. This doesn't work with AOSP
targets that have the flag set but without any available vendor image.

This CL detects such a condition and uses a separate flag to guard the
retrofit OTA building as well as the split super images generation.

Bug: 120852744
Test: `m -j dist` on blueline (w/ vendor images) and aosp_blueline (w/o
      vendor images).
Change-Id: I65726f24f8fc546be6802941a6a06590a3804c16
2019-01-07 12:22:48 -08:00
Dan Willemsen
7dd69f1b3a Merge "Remove more Android.mk files from build/make" am: ebe0b7a1c9 am: 2c99074cbe
am: b877b02724

Change-Id: I0540e99bfee84e2336ac454765f0295139711f38
2019-01-07 10:00:01 -08:00
Dan Willemsen
b877b02724 Merge "Remove more Android.mk files from build/make" am: ebe0b7a1c9
am: 2c99074cbe

Change-Id: I5af0dde7533eecb64dd68cf9003434b4eb5d8a40
2019-01-07 09:54:57 -08:00
Treehugger Robot
ebe0b7a1c9 Merge "Remove more Android.mk files from build/make" 2019-01-07 17:44:20 +00:00
Dan Willemsen
ef45381221 Remove more Android.mk files from build/make
soong_ui is passing the Android.mk file list to Kati, and it ignores
Android.mk files found via symlinks, so we don't need the workarounds
for build/make/target/board/Android.mk and build/target/board/Android.mk
both existing.

We've got dist support in Soong now, so we can remove the signapk
Android.mk

Bug: 122332221
Test: m dist out/dist/signapk.jar out/dist/libconscrypt_openjdk_jni.so
Change-Id: I2ac9016b04146340c5ac80f7090d00e186023574
2019-01-05 02:40:45 +00:00
Hridya Valsaraju
7dbacc000b Merge "Increase slack for inode estimation algorithm" am: dacde2259f am: e5af443000
am: be281179d4

Change-Id: Ibd8d354b327c07e9a4d17171430f78417bf4cb0d
2019-01-04 17:47:29 -08:00
Hridya Valsaraju
be281179d4 Merge "Increase slack for inode estimation algorithm" am: dacde2259f
am: e5af443000

Change-Id: Ic4f67c1155fa148a4b3f1cdeca8bf7d8edcf65dd
2019-01-04 17:36:38 -08:00
Treehugger Robot
dacde2259f Merge "Increase slack for inode estimation algorithm" 2019-01-05 01:22:43 +00:00