Commit Graph

58909 Commits

Author SHA1 Message Date
Jiyong Park
eb49b34b1e Organize product vars for adding properties
Properties for <partition>/build.prop are now added via
PRODUCT_<PARTITION>_PROPERTIES.

PRODUCT_SYSTEM_PROPERTIES is added. PRODUCT_SYSTEM_DEFAULT_PROPERTIES
will be deprecated.

PRODUCT_VENDOR_PROPERTIES is added. PRODUCT_PROPERTY_OVERRIDES and
PRODUCT_DEFAULT_PROPERTY_OVERRIDES will be deprecated in favor of the new
variable.

For other partitions (odm, system_ext, product), there is no change.

Exempt-From-Owner-Approval: cherry-pick from internal master

Bug: 117892318
Test: m
Merged-In: I98f3b5d0da661b3ad75260a23754e655103b7a44
(cherry picked from commit ad810b6dae)
Change-Id: I98f3b5d0da661b3ad75260a23754e655103b7a44
2020-06-11 15:35:23 +09:00
Elliott Hughes
9df45f6ec2 Merge "Remove TARGET_PREFER_32_BIT." 2020-06-10 18:07:06 +00:00
Treehugger Robot
8add8cb145 Merge "base_rules.mk optimizations" 2020-06-10 02:46:12 +00:00
Yo Chiang
988693fb00 Merge "Cleanup unused get-32-bit-modules" 2020-06-10 02:30:24 +00:00
Dan Willemsen
8447b8116a base_rules.mk optimizations
A few more misc improvements that I found while analyzing the
performance of base_rules.mk.

This brings an aosp-master/aosp_crosshatch-userdebug kati run from 33.3s
to 28.6s

Bug: 158488548
Test: build-aosp_crosshatch.ninja is the same before/after
Change-Id: If99c31cc7b5d7133d70eb644c6095f19060b71e5
2020-06-09 09:59:24 -07:00
Treehugger Robot
1d1bb3b94f Merge "More performance improvements" 2020-06-09 04:36:32 +00:00
Yo Chiang
1e55e74acc Cleanup unused get-32-bit-modules
These are no longer used by anyone:
* get-32-bit-modules
* get-32-bit-modules-if-we-can
* get-host-32-bit-modules
* get-host-32-bit-modules-if-we-can

Bug: 155869107
Test: TH
Change-Id: I441de7f94061efd40186b80d1a2ead11669a5753
2020-06-09 03:28:26 +00:00
Dan Willemsen
fd1e67b8ce More performance improvements
Instead of using `x := $(x) foo`, make sure `x` is initialized first,
then use `x += foo`.

For a aosp-master/aosp_crosshatch-userdebug build, this takes the build
from 49s to 33s (no significant difference in peak memory)

Bug: 158488548
Test: build-aosp_crosshatch.ninja is the same before/after
Change-Id: I41fb7611030dd3ba824f896a9c9eadbdd218f464
2020-06-08 19:10:23 -07:00
Treehugger Robot
83e9cfe291 Merge "Rewrite test data logic to make it faster" 2020-06-09 01:00:52 +00:00
Elliott Hughes
9b824d5ceb Remove TARGET_PREFER_32_BIT.
Bug: https://issuetracker.google.com/138812821
Test: builds
Change-Id: I2ff7793c8b1eb56f6be63375cc7b70aa44254443
2020-06-08 16:36:52 -07:00
Frank Feng
e4d2ccee92 Merge "Add test config template for sh_test." 2020-06-08 21:05:31 +00:00
Josh Gao
581e1f4f1f Merge "Rename adbd_system_binaries to adbd_system_api." 2020-06-08 21:00:19 +00:00
Dan Willemsen
c5ec8e95bc Rewrite test data logic to make it faster
It turns out we have a few tests with thousands of test data files,
which this logic does not handle very well. Just switching from `x :=
$(x) ...` to `x += ...` accomplishes the majority of these gains, but I
rewrote the logic to be less function-call+eval based to hopefully make
it easier to understand.

For an aosp-master/aosp_coral-eng build, this brings the time spent in
this logic from 7.6s to 0.3s (total kati runtime of 54s to 46s). Kati's
peak memory use (maxrss) also shrunk from 6.1GB to 3.8GB.

Bug: 158488548
Test: build-aosp_coral.ninja is the same before/after
Change-Id: Ia2ec8b29b5c1dd12fa7ea10b4f80fe802c692725
2020-06-08 11:42:51 -07:00
Treehugger Robot
470cfbe8d7 Merge changes I53c05800,Ic218283b
* changes:
  Leave a comment when removing a prop
  Rule for system/build.prop is also from macro
2020-06-08 01:42:20 +00:00
Jiyong Park
cc49c6b8cd Leave a comment when removing a prop
post_process_prop.py doesn't simply drop a line when deleting a prop.
Instead, it makes the line as comment and leave a comment to clearly
mark that the prop was force removed. This is to aid the debugging.

Bug: 117892318
Test: m

Change-Id: I53c05800ff71d431a56dc370bcfe8bfc95c03bfc
2020-06-08 10:09:12 +09:00
Jiyong Park
bb26c6f2bf Rule for system/build.prop is also from macro
Like other <partition>/build.prop files, the build rule for
system/build.prop is also created using the build-properties macro.

system/build.prop has a few number of auto-generate properties that the
macro doesn't create. Those properties are emitted to intermediate files
using manually crafted rules, the paths to the files are fed to the
macro, where their contents are copied to system/build.prop.

This change also refactors the code fragment that defines BUILD_*
variables that identifes the given build.

Bug: 117892318
Test: m

Change-Id: Ic218283b40fedeaddacb3fda93167edbec86094c
2020-06-08 10:09:12 +09:00
Jiyong Park
c844de4241 Merge changes I4d88733c,I61943476,Ibed9c96d
* changes:
  remove buildinfo_common.sh
  Partial cherry-pick of ag/10260813
  <partition>/build.prop rules are defined using macro
2020-06-08 00:58:48 +00:00
Treehugger Robot
208689cca8 Merge "fix spurious / on toplevel kernel module installations" 2020-06-06 01:53:24 +00:00
Xin Li
6e30a3fc86 Merge "Merge QQ3A.200605.002 into master" 2020-06-06 01:41:29 +00:00
Steve Muckle
950d723b9d fix spurious / on toplevel kernel module installations
When kernel modules are being put directly into lib/modules/ (as opposed
to lib/modules/<subdir>/), the empty part of the destination path causes
a target containing a redundant /. If the target is also added elsewhere
in the build without the redundant /, then rule merging will miss it,
and a build error will occur.

Bug: 158158635
Change-Id: I797addce760f4886166efa283302a53783581625
2020-06-05 12:10:56 -07:00
Ulyana Trafimovich
660edfdc4a Merge "Refactor library path representation in dexpreopt." 2020-06-05 10:46:01 +00:00
Treehugger Robot
a48079902e Merge "Do not print environment variables if ANDROID_QUIET_BUILD is set." 2020-06-05 06:37:12 +00:00
Yo Chiang
f9d1eee9b9 Merge changes I47d523e4,Ib2f4b548
* changes:
  Make required modules support :32 and :64 suffix
  Refactor *-select-bitness-of-required-modules
2020-06-05 05:12:23 +00:00
Treehugger Robot
173149f177 Merge "Strip debug symbols from modules (fixup)" 2020-06-05 03:51:09 +00:00
Jiyong Park
35a83d1a5a remove buildinfo_common.sh
The shell script was responsible for appending some system properties
that are common to <partition>/build.prop files. The values of the
system properties were given via environment variables whose names were
slightly different from their corresponding make var names.

This change removes the script and instead write the system properties
directly in Make, thus eliminating the need to keep track of the mapping
across sysprop names, environment variable names, and make var names.

Bug: 117892318
Test: m

Change-Id: I4d88733c1afa30ea0c0c4fcfc2b349f07f7fc090
2020-06-05 10:50:15 +09:00
Jiyong Park
9ebefc640d Partial cherry-pick of ag/10260813
This change is a partial cherry-pick of ag/10260813
(Iaeefb35137de01c6ea57115d8830dd9bc30eccba) to prevent downstream merge
conflict when buildinfo_common.sh is removed by the upcoming changes.

Bug: 117892318
Test: m

Change-Id: I6194347668fd57dfb29ce1bfda16b90088483591
Merged-In: Iaeefb35137de01c6ea57115d8830dd9bc30eccba
2020-06-05 10:49:35 +09:00
Jiyong Park
e28fa80e12 <partition>/build.prop rules are defined using macro
The macro `build-properties' is defined to generate build rules to
create <partition>/build.prop.

build.prop files in all partitions except for the system partition are
now build using the macro.

system partition will use the macro in the following changes.

Bug: 117892318
Test: m

Change-Id: Ibed9c96dac4366251ec59a0b1317aa92f501c9a3
2020-06-05 10:47:34 +09:00
Sasha Smundak
90d07bc72d Do not print environment variables if ANDROID_QUIET_BUILD is set.
Test: manual
Change-Id: I17c24beb9f0008e038a739a06027266b57e17e7a
2020-06-04 10:48:15 -07:00
frankfeng
7e32b34ed3 Add test config template for sh_test.
Verification of compatibility with existing sh_test targets is in progress.

Bug: 148805488
Bug: 151761373

Test: make
Test: unit tests
Test: verified test config auto-gen for bc-tests and toybox-tests
Change-Id: Id8ce5dd0723e8c95921e2e20bff72fc828d47d87
2020-06-04 10:35:53 -07:00
Ulya Trafimovich
77a78fca4e Refactor library path representation in dexpreopt.
This is a preliminary CL before fixing on-device paths to DEX jars.
It groups together the inormation about on-host build paths and
on-device install paths to library DEX jars.

This CL changes the structure of module dexpreopt.config files
generated by the build system. Aside of that, no functional changes.

Test: lunch aosp_cf_x86_phone-userdebug && m
Change-Id: I8e92829885f26522aa2b936e77e91ce91593b27c
2020-06-04 17:33:22 +01:00
Ulyana Trafimovich
4c5ed87faa Merge "Add product variable PRODUCT_BROKEN_SUBOPTIMAL_ORDER_OF_SYSTEM_SERVER_JARS." 2020-06-04 08:56:04 +00:00
Pierre Couillaud
d99da8fa17 Strip debug symbols from modules (fixup)
allow to also skip the stripping of the modules present in vendor
ramdisk when the BOARD configuration is set to skip the strip step.

there is also a shortcoming of this functionality: it currently assumes
the modules to strip are all created equal against the same arch in
particular, however this does not work for kernel modules which are
for mixed mode platforms, as example {kernel,android} == {aarch64,armv7}
because it will select the "android" mode strip which is unaware of the
kernel mode architecture.

Bug: 156395749
Bug: 155193290
Change-Id: I35ef51845224e94e3e253acdeb6bb14b990a401b
2020-06-04 00:17:58 +00:00
Treehugger Robot
49efd2aad0 Merge "Re-enable link type checking" 2020-06-03 21:30:22 +00:00
Dan Willemsen
be15ded040 Re-enable link type checking
When I removed the actions that ran the python link type checker, I
forgot that the Make version hadn't been enabled yet. Enable the Make
version.

Bug: 143748700
Test: treehugger (particularly, build_test on the downstream branches)
Change-Id: I293b3be5ca7828c9d7148e13d88b0f0bba54d964
Merged-In: I293b3be5ca7828c9d7148e13d88b0f0bba54d964
Merged-In: I08090db44d47f809fbaa0c4ad373e46e6a3fbe2e
2020-06-03 21:09:32 +00:00
Treehugger Robot
8c6c2cd3de Merge "refactor kernel module build, add version support" 2020-06-03 18:10:52 +00:00
Treehugger Robot
aa62df07cf Merge changes Ia1392b7d,I199788d4,I9c8ac0e6
* changes:
  move BOARD_RECOVERY_KERNEL_MODULES_LOAD default
  move module list creation outside conditionals
  fix reference to BOARD_ODM_MODULES_LOAD
2020-06-03 18:10:44 +00:00
Xin Li
e92524e4ce Merge QQ3A.200605.002 into master
Bug: 158095402
Merged-In: I216989b052070e9f84bf1fa0f9ecc4724db50418
Change-Id: I58f2e5b4c7039a22aaecb66eabc8288ff51cd97c
2020-06-03 10:44:41 -07:00
Ulya Trafimovich
40e55c20eb Add product variable PRODUCT_BROKEN_SUBOPTIMAL_ORDER_OF_SYSTEM_SERVER_JARS.
If true, incorrect order of system server jars does not cause a build
error. The order is non-optimal if some jar X precedes its dependency Y.
In that case dexpreopt will be unable to resolve any rerefences from X
to Y.

Test: lunch cf_x86_phone-userdebug && m nothing
Test: lunch aosp_car_arm && m nothing

Bug: 140451054

Change-Id: I70efd365fa9fbd9e40792d646f8471d009164924
Merged-In: I70efd365fa9fbd9e40792d646f8471d009164924
(cherry picked from commit 17715b48f0)
2020-06-03 17:38:26 +01:00
Yo Chiang
60da88da55 Make required modules support :32 and :64 suffix
Support specifying bitness of required modules with :32 and :64 suffix,
like in PRODUCT_PACKAGES.

Bug: 155869107
Bug: 129323707
Test: TH noop
Change-Id: I47d523e4e57cb8a99629fd241a16069c0be55903
2020-06-03 18:27:42 +08:00
Yo Chiang
1b8283358e Refactor *-select-bitness-of-required-modules
* Streamline required module bitness resolution

If a module is for cross host OS, the required modules are also for that
OS.
Otherwise if the requirer module is native and the required module is
shared library or native test, then the required module resolves to the
same bitness.
Otherwise the required module resolves to both variants, if they exist.

* Factor out the common logic and merge
  <target|host|host-cross>-select-bitness-of-required-modules

This lays the road for follow-up changes to add bitness suffix support.
This slightly reduces readibility. To make up for the loss, the
bitness-resolving logic is easier to maintain since we don't have
multiple copies of *-select-bitness-of-required-modules.

Bug: 155869107
Test: check the diff of product_target_FILES and product_host_FILES
Test: check the diff of ALL_MODULES.*.REQUIRED_FROM_(TARGET|HOST)
Change-Id: Ib2f4b548535e983d621dad71cb920101111140f2
2020-06-03 18:27:20 +08:00
Treehugger Robot
0208ce535d Merge "remove clatd.conf which is no longer required by clat daemon" am: 5638065faa
Original change: https://android-review.googlesource.com/c/platform/build/+/1172552

Change-Id: I216989b052070e9f84bf1fa0f9ecc4724db50418
2020-06-03 05:27:41 +00:00
Treehugger Robot
5638065faa Merge "remove clatd.conf which is no longer required by clat daemon" 2020-06-03 05:15:21 +00:00
Yo Chiang
eb4f43e6cc Merge "Revert^2 "Retire mainline.mk"" am: 2ca90c5e97
Original change: https://android-review.googlesource.com/c/platform/build/+/1322634

Change-Id: Ia1fe27c92188cf0dc07a1dcc44e819a2babd617c
2020-06-03 03:02:49 +00:00
Yo Chiang
2ca90c5e97 Merge "Revert^2 "Retire mainline.mk"" 2020-06-03 02:47:52 +00:00
Yo Chiang
82e4bc5d0c Revert^2 "Retire mainline.mk"
This reverts commit 4003bd9def.

Reason for revert: cleaned up mainline.mk users

Bug: 157548597
Fixes: 141443017
Change-Id: I5e38620ed1c90f7188901eb148c7e3aa0596a27d
2020-06-03 02:47:34 +00:00
Steve Muckle
2335d76ada refactor kernel module build, add version support
Refactor the build support for kernel modules. This refactoring is
leveraged to add support for directories of kernel modules in each
partition. To add a directory of kernel modules, define
BOARD_KERNEL_MODULE_DIRS to include the directory name, for example:

BOARD_KERNEL_MODULE_DIRS := 5.4

The build will then check all the usual kernel module related macros
with an extension of _5.4, for example you can define

BOARD_VENDOR_KERNEL_MODULES_5.4 := <modules to copy>
BOARD_VENDOR_KERNEL_MODULES_ARCHIVE_5.4 = <path to archive file>
BOARD_VENDOR_KERNEL_MODULES_LOAD_5.4 := <modules for load list>

Bug: 157645635
Change-Id: I1687d0ec85c1dcf21350350623b4689aba3f6717
2020-06-02 15:17:26 -07:00
Steve Muckle
c88a8c88e4 move BOARD_RECOVERY_KERNEL_MODULES_LOAD default
Moving this makes each of the partition's kernel module Makefile
generation more similar, enabling a forthcoming cleanup/refactoring.

If recovery-is-boot and there are kernel modules defined in
BOARD_GENERIC_RAMDISK_KERNEL_MODULES, those kernel modules will
now be included in the default assignment to
BOARD_RECOVERY_KERNEL_MODULES_LOAD. If this is not desired then
BOARD_RECOVERY_KERNEL_MODULES_LOAD may simply be set to
BOARD_RECOVERY_KERNEL_MODULES (or a custom list of recovery kernel
modules to load).

Bug: 157645635
Change-Id: Ia1392b7d20d07cee75420edd95dcee1dc825516e
2020-06-02 15:17:15 -07:00
Steve Muckle
1013d789de move module list creation outside conditionals
Currently the two special-case calls to create just a module load list
(as opposed to copying a set of modules and creating a module load list)
are inside checks that the related board macro for the module set is not
empty.

While it would be incorrect to create a non-empty module load list with
an empty set of modules, this error would quickly surface at runtime.
More importantly, moving these out will permit refactoring this code to
clean it up.

Bug: 157645635
Change-Id: I199788d49072d4c59737b3ef2c75546852cf3e36
2020-06-02 15:17:04 -07:00
Steve Muckle
f291097221 fix reference to BOARD_ODM_MODULES_LOAD
This check is meant to initialize BOARD_ODM_KERNEL_MODULES_LOAD to
BOARD_ODM_KERNEL_MODULES if BOARD_ODM_KERNEL_MODULES_LOAD is empty.

Bug: 157645635
Change-Id: I9c8ac0e64bb002e926d15b446a63801d7cfa9911
2020-06-02 15:16:54 -07:00
Treehugger Robot
6231dd8584 Merge "Remove unused check-api rule" am: fa9ea4b8b0
Original change: https://android-review.googlesource.com/c/platform/build/+/1322795

Change-Id: I5e1d6dd6629d51410a15620e22d0460d9c39c68b
2020-06-02 19:27:03 +00:00