Commit Graph

22960 Commits

Author SHA1 Message Date
Ulya Trafimovich
77a78fca4e Refactor library path representation in dexpreopt.
This is a preliminary CL before fixing on-device paths to DEX jars.
It groups together the inormation about on-host build paths and
on-device install paths to library DEX jars.

This CL changes the structure of module dexpreopt.config files
generated by the build system. Aside of that, no functional changes.

Test: lunch aosp_cf_x86_phone-userdebug && m
Change-Id: I8e92829885f26522aa2b936e77e91ce91593b27c
2020-06-04 17:33:22 +01:00
Ulyana Trafimovich
4c5ed87faa Merge "Add product variable PRODUCT_BROKEN_SUBOPTIMAL_ORDER_OF_SYSTEM_SERVER_JARS." 2020-06-04 08:56:04 +00:00
Treehugger Robot
49efd2aad0 Merge "Re-enable link type checking" 2020-06-03 21:30:22 +00:00
Dan Willemsen
be15ded040 Re-enable link type checking
When I removed the actions that ran the python link type checker, I
forgot that the Make version hadn't been enabled yet. Enable the Make
version.

Bug: 143748700
Test: treehugger (particularly, build_test on the downstream branches)
Change-Id: I293b3be5ca7828c9d7148e13d88b0f0bba54d964
Merged-In: I293b3be5ca7828c9d7148e13d88b0f0bba54d964
Merged-In: I08090db44d47f809fbaa0c4ad373e46e6a3fbe2e
2020-06-03 21:09:32 +00:00
Treehugger Robot
8c6c2cd3de Merge "refactor kernel module build, add version support" 2020-06-03 18:10:52 +00:00
Treehugger Robot
aa62df07cf Merge changes Ia1392b7d,I199788d4,I9c8ac0e6
* changes:
  move BOARD_RECOVERY_KERNEL_MODULES_LOAD default
  move module list creation outside conditionals
  fix reference to BOARD_ODM_MODULES_LOAD
2020-06-03 18:10:44 +00:00
Ulya Trafimovich
40e55c20eb Add product variable PRODUCT_BROKEN_SUBOPTIMAL_ORDER_OF_SYSTEM_SERVER_JARS.
If true, incorrect order of system server jars does not cause a build
error. The order is non-optimal if some jar X precedes its dependency Y.
In that case dexpreopt will be unable to resolve any rerefences from X
to Y.

Test: lunch cf_x86_phone-userdebug && m nothing
Test: lunch aosp_car_arm && m nothing

Bug: 140451054

Change-Id: I70efd365fa9fbd9e40792d646f8471d009164924
Merged-In: I70efd365fa9fbd9e40792d646f8471d009164924
(cherry picked from commit 17715b48f0)
2020-06-03 17:38:26 +01:00
Steve Muckle
2335d76ada refactor kernel module build, add version support
Refactor the build support for kernel modules. This refactoring is
leveraged to add support for directories of kernel modules in each
partition. To add a directory of kernel modules, define
BOARD_KERNEL_MODULE_DIRS to include the directory name, for example:

BOARD_KERNEL_MODULE_DIRS := 5.4

The build will then check all the usual kernel module related macros
with an extension of _5.4, for example you can define

BOARD_VENDOR_KERNEL_MODULES_5.4 := <modules to copy>
BOARD_VENDOR_KERNEL_MODULES_ARCHIVE_5.4 = <path to archive file>
BOARD_VENDOR_KERNEL_MODULES_LOAD_5.4 := <modules for load list>

Bug: 157645635
Change-Id: I1687d0ec85c1dcf21350350623b4689aba3f6717
2020-06-02 15:17:26 -07:00
Steve Muckle
c88a8c88e4 move BOARD_RECOVERY_KERNEL_MODULES_LOAD default
Moving this makes each of the partition's kernel module Makefile
generation more similar, enabling a forthcoming cleanup/refactoring.

If recovery-is-boot and there are kernel modules defined in
BOARD_GENERIC_RAMDISK_KERNEL_MODULES, those kernel modules will
now be included in the default assignment to
BOARD_RECOVERY_KERNEL_MODULES_LOAD. If this is not desired then
BOARD_RECOVERY_KERNEL_MODULES_LOAD may simply be set to
BOARD_RECOVERY_KERNEL_MODULES (or a custom list of recovery kernel
modules to load).

Bug: 157645635
Change-Id: Ia1392b7d20d07cee75420edd95dcee1dc825516e
2020-06-02 15:17:15 -07:00
Steve Muckle
1013d789de move module list creation outside conditionals
Currently the two special-case calls to create just a module load list
(as opposed to copying a set of modules and creating a module load list)
are inside checks that the related board macro for the module set is not
empty.

While it would be incorrect to create a non-empty module load list with
an empty set of modules, this error would quickly surface at runtime.
More importantly, moving these out will permit refactoring this code to
clean it up.

Bug: 157645635
Change-Id: I199788d49072d4c59737b3ef2c75546852cf3e36
2020-06-02 15:17:04 -07:00
Steve Muckle
f291097221 fix reference to BOARD_ODM_MODULES_LOAD
This check is meant to initialize BOARD_ODM_KERNEL_MODULES_LOAD to
BOARD_ODM_KERNEL_MODULES if BOARD_ODM_KERNEL_MODULES_LOAD is empty.

Bug: 157645635
Change-Id: I9c8ac0e64bb002e926d15b446a63801d7cfa9911
2020-06-02 15:16:54 -07:00
Aurimas Liutikas
9c5b07966e Remove unused check-api rule
Change-Id: I0297b226d066e99ee4fc4c2b22e160eafc7f5a8a
Test: Treehugger?
2020-06-02 17:01:52 +00:00
Ulya Trafimovich
0b0cf4c411 Provide library path to android.test.base for dexpreopt.
It may be needed in class loader context if the target API version
is less than 30.

Test: lunch aosp_cf_x86_phone-userdebug && m
Change-Id: Ifa5ab18d1dc322184100207ad6eee4bdcf74deb0
2020-06-01 12:29:27 +01:00
Treehugger Robot
6be7b57ce7 Merge "remove spurious comma from module-load-list-copy-paths call" 2020-05-29 22:08:11 +00:00
Steve Muckle
06b48162f4 remove spurious comma from module-load-list-copy-paths call
The module-load-list-copy-paths fn takes five parameters.

Merged-In: I7d1736da1599e265793173a741053d8a09f1622f
Change-Id: I9286c9e4ca81caf5392bc804dead718bd028b193
2020-05-29 13:54:49 -07:00
Baligh Uddin
ab25b2dcfa Add apksigner to otatools.zip
BUG: 157735036
Test: Local build.
Change-Id: I95c7001ad520d6b4a415d21ffdef55152b0e82b2
2020-05-29 19:01:32 +00:00
J. Avila
bc4d5bdb87 Merge "Add build support for LZ4 ramdisks" 2020-05-29 16:28:47 +00:00
Jiyong Park
6dd5383c37 Merge changes from topics "remove_bootimage_sysprops", "sysprop"
* changes:
  Drop the dependency from vendor/build.prop to system/build.prop
  Remove ro.bootimage.* sysprops
  ro.build.expect.* sysprops are created in a separate rule
  Make use of ADDITIONAL_ODM_PROPERTIES
  Refactor post_process_props.py
  remove code for default.prop
  Make more use of ADDITIONAL_VENDOR_PROPERTIES
  ADDITIONAL_*_PROPERTIES are props determined by the build system
2020-05-29 01:01:52 +00:00
Yo Chiang
0dcb96c204 Merge "Check module existence with ALL_MODULES subvars" 2020-05-28 10:12:52 +00:00
Dan Willemsen
9767223dc5 Fix up recovery_text_res
First, make it safer for incremental builds. We used to just be
depending on the bootable/recovery/tools/recovery_l10n/res dir as a
dependency, but that would only trigger a rebuild if a direct child
file/directory was added or removed (so essentially, when a translation
was addded or removed). If a translation was updated, it wouldn't
re-trigger recovery_text_res.

So instead of depending on the directory, depend on the results from
running `find` to list all of the contents in that directory.

Next, move it out of PACKAGING. `m installclean`, which runs in between
incremental builds on our build servers, deletes the obj/PACKAGING
directory. So when recovery_text_res used that directory, we were having
to rebuild it on every incremental build, which isn't optimal when it
takes nearly 90s to build.

So with:
 $ lunch aosp_coral-eng
 $ m bootimage
 $ m installclean
 $ m bootimage

The second `m bootimage` went from executing 666 actions in 93s to
executing 658 actions in 6s.

Finally, remove the last uses of $(call include-path-for,recovery),
since they were all in this file (and they weren't using it for include
paths...). We'd like to remove all of these and switch them to header
libraries or other use cases, but in this case, it's just shorter to use
the real path.

Test: treehugger
Test: m bootimage;
      touch bootable/recovery/tools/recovery_l10n/res/values/strings.xml
      m bootimage
Test: m bootimage; m installclean; m bootimage

Change-Id: I005592e49443aab45ed039a2f0c63f7a69035565
2020-05-27 19:37:03 -07:00
J. Avila
3f6560e626 Add build support for LZ4 ramdisks
Ramdisks are currently compressed using gzip, which is generally
smaller, but slower to decompress than lz4. Provide an option for an
lz4 scheme to reduce the time taken to unpack initramfs during boot.

Bug: 156129966
Signed-off-by: J. Avila <elavila@google.com>
Change-Id: Iac9538e6ee6ec51e6b487de2101f53bb5d9c54c8
2020-05-27 23:44:33 +00:00
Treehugger Robot
a29edba6e8 Merge "Strip debug symbols from modules" 2020-05-27 20:46:16 +00:00
Jiyong Park
a9c3220368 Drop the dependency from vendor/build.prop to system/build.prop
The former doesn't use anything from the latter. The dependency is not
needed.

Bug: 117892318
Test: m
Change-Id: I61aeea9feb1ac4467af4f185f3d6930b03145afd
2020-05-27 12:52:44 +09:00
Jiyong Park
0f3ad31a81 Remove ro.bootimage.* sysprops
ro.bootimage.build.date.* are not used anywhere.
ro.bootimage.build.fingerprint was used in the recovery mode, which has
now been switched to ro.build.fingerprint.

Bug: 117892318
Test: m
Change-Id: I3776c4154f86ddb908b24c6899037c6a7df037df
2020-05-27 12:52:44 +09:00
Jiyong Park
3a2e95a63e ro.build.expect.* sysprops are created in a separate rule
This change is required to unify the build rules for */build.prop files;
The rules shouldn't have partition-specific actions.

Bug: 117892318
Test: m
Manually inspect ro.build.expect.[bootloader|baseband] props in
vendor/build.prop

Change-Id: I3b29b2728f779a9d7e1d71a5aa69b02ea940b3c6
2020-05-27 12:52:44 +09:00
Jiyong Park
ca9b02c1ce Make use of ADDITIONAL_ODM_PROPERTIES
Properties that were defined in the build rule for odm/build.prop are
now defined via ADDITIONAL_ODM_PROPERTIES.

Bug: 117892318
Test: m
Change-Id: I6220e217d0724b6e27127786e7a908e1e662d1a0
2020-05-27 12:52:44 +09:00
Jiyong Park
ae55638f75 Refactor post_process_props.py
The script is refactored to not parse lines everytime a prop is looked
up or updated.

Also it is now built using python_binary_host.

Bug: 117892318
Test: m

Change-Id: I1677ca5aa919c54b7b0740fdc52e1399009f9a12
2020-05-27 12:52:44 +09:00
Jiyong Park
f2b944ccea Make more use of ADDITIONAL_VENDOR_PROPERTIES
Properties that were defined in the build rule for vendor/build.prop are
now defined via ADDITIONAL_VENDOR_PROPERTIES.

Bug: 117892318
Test: m
Change-Id: Ib3dc9d001a8dcd0aa38f1acae384a96b67b5af8a
2020-05-27 12:52:43 +09:00
Jiyong Park
57f51f1693 ADDITIONAL_*_PROPERTIES are props determined by the build system
This change defines ADDITIONAL_<partition>_PROPERTIES to store properties
determined by the build system itself.

ADDITIONAL_SYSTEM_PROPERTIES was originally ADDITIONAL_BUILD_PROPERTIES.
ADDITIONAL_VENDOR_PROPERTIES was originally FINAL_VENDOR_DEFAULT_PROPERTIES.
ADDITIONAL_PRODUCT_PROPERTIES has been there and thus not changed.
ADDITIONAL_ODM_PROPERTIES is newly defined, although it is empty now.

In addition, ADDITIONAL_VENDOR_PROPERTIES are now defined in
core/main.mk along with other ADDITIONAL_*_PROPERTIES.

Bug: 117892318
Test: m

Change-Id: Ic41bba2e5695355436ac8df353866cfc044e1e0b
2020-05-27 12:52:43 +09:00
Treehugger Robot
aa0584c22d Merge "Add RBE_CXX_COMPARE variable to locally disable compare mode for C++" 2020-05-27 02:27:09 +00:00
J. Avila
a2ceec665d Strip debug symbols from modules
The size of the ramdisk has greatly increased, which can be a source of
boot time regressions. Strip the modules of debug symbols for the vendor
ramdisk and vendor partitions; vendor stripping can be skipped via BOARD
config.

Bug: 156395749
Signed-off-by: J. Avila <elavila@google.com>
Merged-In: I248d4e2097992f00975b344b5da41aa5d8016812
Change-Id: I248d4e2097992f00975b344b5da41aa5d8016812
2020-05-26 23:35:26 +00:00
Treehugger Robot
a74d425017 Merge "depmod: pair in a modules.load.recovery on BOARD_VENDOR_RAMDISK_RECOVERY_KERNEL_MODULES_LOAD" 2020-05-26 21:10:54 +00:00
Kousik Kumar
b0f055db13 Add RBE_CXX_COMPARE variable to locally disable compare mode for C++
When we do 64-32 migration, we want compare mode to be present only for
non-c++ actions as it would impact performance on 32-core machines
otherwise.

Bug: b/157414300
Test: Ran a build with "RBE_CXX_LINKS="true" RBE_CXX_LINKS_EXEC_STRATEGY="remote" RBE_CXX_COMPARE="false" RBE_compare="true" use_rbe mmma bionic"
and observed that local-executions happened only for link actions and
not for C++ actions.

Change-Id: Ia972c74d804085196a7b6ec755af63042c6b3909
2020-05-26 11:57:02 -07:00
Treehugger Robot
223c0f66d5 Merge "Generate product build props always" 2020-05-26 17:26:01 +00:00
Treehugger Robot
95fc5e76a6 Merge "Delete deps-license target." 2020-05-21 21:37:06 +00:00
Treehugger Robot
8a87dae43b Merge "Add options to sign the prebuilt custom images." 2020-05-21 16:34:13 +00:00
Bob Badour
2441181bdf Delete deps-license target.
If used, the results would be inaccurate in any case.

Deleted atest references: http://aosp/1315404 http://aosp/1315405

Test: treehugger
Test: atest -c --rebuild-module-info com.google.android.gts.updateengine.UpdateEngineHostTest

Change-Id: I6efbf1f5ff3a533b26d6bbedc14a6fcc1b200391
2020-05-21 07:11:55 -07:00
SzuWei Lin
1e6b3e78df Generate product build props always
GSI do not build product partition, and contains the product
contents under /system/product/. Original logic do not generate
the product build props on this case.

The patch fixes the logic for product build props generation to
align the other partitions.

Bug: 157191627
Test: build aosp_arm64-userdebug
Test: check /system/product/build.prop in out folder
Change-Id: Ia88c8efecb18db9c27baa466e492bfd898b581d3
2020-05-21 20:16:47 +08:00
Mark Salyzyn
4e20965769 depmod: pair in a modules.load.recovery on BOARD_VENDOR_RAMDISK_RECOVERY_KERNEL_MODULES_LOAD
If the BoardConfig defined BOARD_VENDOR_RAMDISK_MODULES_LOAD and
BOARD_VENDOR_RAMDISK_RECOVERY_KERNEL_MODULES_LOAD, generate both a
modules.load and modules.load.recovery respectively on the vendor
ramdisk.  BOARD_VENDOR_RAMDISK_MODULES must be the set, or super set,
of all the modules represented in both of *_LOAD BOARD variables.

Signed-off-by: Mark Salyzyn <salyzyn@google.com>
Bug: 151950334
Test: build, confirm, fastboot flashall, boot with split.
Change-Id: Ib2925aec6ea7a2a01a554c808d6697d8c85f9fc0
2020-05-20 14:44:01 -07:00
Treehugger Robot
0ead402c40 Merge "Do not fail if unzip found no files to extract" 2020-05-20 20:18:04 +00:00
Julien Desprez
f244a36850 Merge "Add tradefed-test-framework to the suites exported jar" 2020-05-20 16:46:22 +00:00
Yo Chiang
1db657a1ed Check module existence with ALL_MODULES subvars
Use ALL_MODULES.*.PATH to check module existence.
Use get-modules-for-2nd-arch to check the existence of second arch
variant.
Fix some typos *existant -> existent

Bug: 155869107
Test: lunch mainline_system_arm64; m
Test: lunch generic; m
Change-Id: Ic10fb34a11da0bfbfc528da7496fd929b1d77113
2020-05-20 16:02:07 +00:00
Sasha Smundak
262491b0a8 Do not fail if unzip found no files to extract
Fixes: 157081961
Test: manual
Change-Id: Ib84c9fa1ceb45542ebcee14e048ecf2c1783c0c0
Merged-In: Ib84c9fa1ceb45542ebcee14e048ecf2c1783c0c0
2020-05-20 04:58:01 +00:00
Yo Chiang
7c147f7e29 Merge "Resolve module bitness for PRODUCT_PACKAGES" 2020-05-20 04:55:22 +00:00
Hongguang Chen
f23364d280 Add options to sign the prebuilt custom images.
The custom images are any images owned by OEMs and SoCs, oem images
mounted on /oem is an example. The oem images can be used to customize
devices for different carriers, like wallpaper, ringtones, and
carrier-specific apks. OEMs can generate multiple oem images, like
oem.img, oem-carrier1.img and oem-carrier2.img and flash different oem
images for different carriers. The oem images are only one case, OEMs
and SoCs can add more custom images and mount them to custom partitions.

This change enables custom images to be vbmeta.img chained partitions.
The following configuration in BoardConfig.mk is an exmaple. It has two
custom partitions: oem and test. They will be signed by different keys.
And they will be chained by vbmeta.img. The custom images here are
prebuilts, which can be built by `make custom_images` separately.
BOARD_AVB_<CUSTOM_PARTITION>_IMAGE_LIST should include all custom images
to apply AVB signing. And to every custom partition, one image whose
name is partition name must be added in its
BOARD_AVB_<CUSTOM_PARTITION>_IMAGE_LIST.

BOARD_CUSTOMIMAGES_PARTITION_LIST := oem test

BOARD_AVB_OEM_KEY_PATH := external/avb/test/data/testkey_rsa4096.pem
BOARD_AVB_OEM_ALGORITHM := SHA256_RSA4096
BOARD_AVB_OEM_ADD_HASHTREE_FOOTER_ARGS :=
BOARD_AVB_OEM_ROLLBACK_INDEX_LOCATION := 1
BOARD_AVB_OEM_PARTITION_SIZE := 5242880
BOARD_AVB_OEM_IMAGE_LIST := \
    device/xxxx/yyyy/oem/oem.img \
    device/xxxx/yyyy/oem/oem1.img

BOARD_AVB_TEST_KEY_PATH := external/avb/test/data/testkey_rsa2048.pem
BOARD_AVB_TEST_ALGORITHM := SHA256_RSA2048
BOARD_AVB_TEST_ADD_HASHTREE_FOOTER_ARGS :=
BOARD_AVB_TEST_ROLLBACK_INDEX_LOCATION := 2
BOARD_AVB_TEST_PARTITION_SIZE := 10485760
BOARD_AVB_TEST_IMAGE_LIST := \
    device/xxxx/yyyy/test/test.img \
    device/xxxx/yyyy/test/test1.img

To resign the custom images in the target zip file, the
avb_extra_custom_image_key, avb_extra_custom_image_algorithms and
avb_extra_custom_image_extra_args options are added to the
sign_target_files_apks tool too. The following test cases list some
examples about how to use them.

BUG: 154171021
Test: 1) "atest --host releasetools_test releasetools_py3_test -c"
  2) Build images by 'make dist', sign and validate target files.
   a) Test on dist w/ chained vbmeta_system and ome custom images
   sign_target_files_apks -d certs \
     --avb_extra_custom_image_key oem=oem_rsa4096.pem \
     --avb_extra_custom_image_algorithm oem=SHA256_RSA4096 \
     xxx-target_xxx.zip signed.zip
   validate_target_files.py signed.zip
   Flash image and boot up.
   Verify the oem images and vbmeta images in OUT and target zips by
   avbtool.

   b) Test on dist w/ chained vbmeta_system and oem and test custom images
   sign_target_files_apks -d certs \
     --avb_extra_custom_image_key oem=oem_rsa4096.pem \
     --avb_extra_custom_image_algorithm oem=SHA256_RSA4096 \
     --avb_extra_custom_image_extra_args oem=--do_not_generate_fec \
     --avb_extra_custom_image_key test=test_rsa4096.pem \
     --avb_extra_custom_image_algorithm test=SHA256_RSA4096 \
     xxx-target_xxx.zip signed.zip
   validate_target_files.py signed.zip
   Verify the oem, test images and vbmeta images in OUT and target zips
   by avbtool.

   c) Test on dist w/o chained partition.
   sign_target_files_apks -d certs xxx-target_xxx.zip signed.zip
   validate_target_files.py signed.zip
   Flash image and boot up.
   Verify the vbmeta images in OUT and target zips by avbtool.

Change-Id: Ifccfee5e8909697eef6ccda0cc352fa16a9f6db6
2020-05-20 04:10:00 +00:00
Treehugger Robot
3dbbfde296 Merge changes I80341c2e,I0935f611,Ia805cb47
* changes:
  Deprecate ADDITIONAL_DEFAULT_PROPERTIES
  Move most of the sysprop generation rules to sysprop.mk
  Add PLATFORM_VERSION_LAST_STABLE
2020-05-20 03:49:26 +00:00
Bob Badour
fe1328755d Merge "Revert "Delete deps-license target."" 2020-05-20 02:16:06 +00:00
Bob Badour
233336e8a8 Revert "Delete deps-license target."
This reverts commit a553cc828e.

Reason for revert: atest has code that tries to build the target

Change-Id: I46902b3930a5a73aa3e5a15f29e0bb0246e59250
2020-05-20 02:14:04 +00:00
Yifan Hong
65afc07f9d Support forcefully generating non-AB packages.
Force generate a non-A/B update package when requested.

Bug: 154344887
Test: ota_from_target_files.py --force_non_ab ...
Test: apply it as well

Change-Id: I5e81eb161722e07ef50081b6a16685cbc9963ae2
(cherry picked from commit 7169f754cc)
Merged-In: I5e81eb161722e07ef50081b6a16685cbc9963ae2
2020-05-19 15:23:27 -07:00
Treehugger Robot
def74b4c2b Merge "support different boot partition sizes in aosp_arm64" 2020-05-19 21:30:07 +00:00