Commit Graph

427 Commits

Author SHA1 Message Date
ShuJie Wang
76e93ba6cf build: clean up proguard_flag_files
we should clean up proguard_flag_files if LOCAL_PROGUARD_ENABLED
not defined in current module in case legacy proguard_flag_files
slips into current jack_all_deps dependencies.

Test: make -j8

Change-Id: I45618fc8e821ab7a8b964fa621824209e1e9894c
Signed-off-by: ShuJie Wang <wangshujie@xiaomi.com>
2017-11-09 15:26:28 +08:00
Søren Gjesse
c66d297e0a Use prebuild R8 for USE_R8=true
Instead of building R8 from external/r8 use the prebuild jar in
prebuilds/r8.

Test: m -j USE_R8=true

Change-Id: I9680ae430cdbc2b3beb910873c8eed27354f6af1
2017-11-07 16:21:50 +01:00
Tobias Thierer
89740664a5 Skip Proguard for EXPERIMENTAL_USE_OPENJDK9=true
Proguard doesn't support v53 class files. For now, this
CL simply replaces the Proguard step with a plain 'cp',
which is essentially a no-op.

To avoid apps exceeding the dex limit in this mode, this
CL also enables --multi-dex for those builds.

Test: Treehugger
Bug: 67673860
Change-Id: I17dc5d827360d606a0cac12e324f97e8dedbb630
2017-10-31 21:02:14 +00:00
Colin Cross
0239edea1d Merge changes I3bafbb2d,If30943ef
* changes:
  Desugar default methods if required
  Fix desugaring when LOCAL_SDK_VERSION is not set
2017-10-27 01:40:15 +00:00
Colin Cross
98caed6e7f Fix desugaring when LOCAL_SDK_VERSION is not set
When LOCAL_SDK_VERSION is not set, frameworks.jar is put
in the classpath for javac.  If it is also in the classpath
(as opposed to the bootclasspath) for desugar then desugar
tries to rewrite classes that implement interfaces with
default methods to contain calls to the generated companion
classes for those interfaces.  frameworks.jar will not
contain the companion classes, which causes proguard to
fail.

Move frameworks.jar to the bootclasspath, which matches more
closely with builds against the SDK stubs jars.

Also remove the static libraries from the classpath, they have
already been merged into the input jar.

Test: m checkbuild
Change-Id: If30943efcaea44b2db1b38a6c1d558113324757f
2017-10-26 13:09:07 -07:00
Søren Gjesse
2f05802a78 Merge "Define proguard dictionary output for R8" 2017-10-26 06:08:43 +00:00
Colin Cross
437bbed858 Merge changes I81796b45,Iaf585336
* changes:
  Desugar default methods if required
  Always raise proguard libraries
2017-10-25 21:44:18 +00:00
Søren Gjesse
4ab5b02e62 Define proguard dictionary output for R8
When running with R8 instead of Proguard + dx the proguard mapping is
produced together with the dex output.

Bug: 68249935
Test: m -j ahat-test
Change-Id: Ica5ef455eb3bd1eabe55c43e9e026f47cadace1c
2017-10-25 17:16:53 +02:00
Colin Cross
b5f25a257f Always raise proguard libraries
The platform build system doesn't have the concept of a "compile sdk
version", it uses the "min sdk version" throughout the compilation
process.  This means that no stage of the compilation process can
introduce references to APIs that do not exist in the selected
LOCAL_MIN_SDK_VERSION without causing problems.  Support library
worked around this by effectively making the "compile sdk version"
"current" when running proguard if support library was in the direct
dependency list.

Extend the support library hack everywhere, and always use "current"
or the full bootclasspath when running proguard.

Test: m checkbuild
Change-Id: Iaf5853361767f0ea3e9ad4ec908aadd72d639c79
2017-10-24 13:29:36 -07:00
Søren Gjesse
0c1fe64ce8 Added support for using R8 instead of Proguard + dx
With this change building with USE_R8=true will run R8 instead of Proguard + dx
for all targets that have LOCAL_PROGUARD_ENABLED set. R8 (through the
compatproguard wrapper) will use the same options as Proguard for now.

Additional Proguard configurations for R8 can be added using LOCAL_R8_FLAG_FILES
on individual targets. This makes it possible to make configuration updates
without touching the Proguard + dx configuration

Test: m -j
Test: m -j USE_D8=true USE_R8=true
Test: m -j USE_D8=true USE_R8=false
Test: m -j USE_D8=true
Change-Id: I78e8a0fd17cc5cefedccffe95b93c94a60a727f1
2017-10-23 08:58:30 +02:00
Colin Cross
f960257c14 Use system modules generated by soong when using javac -target 1.9
Soong has support for building system modules.  Use the directories
produced by Soong with --system to replace -bootclasspath arguments
when using javac -target 1.9.

Since soong can't generate current SDK stubs yet, and no existing
SDK stubs need -target, only use -target 1.9 for modules that are
not compiling against the SDK.  That means in practice the only
system modules that will be used for now is the default one,
core-system-modules.

Bug: 63986449
Test: m -j EXPERIMENTAL_USE_OPENJDK9=true makes some progress
Change-Id: I350ef50aedf36fdd72458c23d4fe8a2edf1a9a02
2017-10-16 17:55:24 -07:00
Nan Zhang
ad818dcbea Support java compilation sharding for target side.
Add "LOCAL_JAVAC_SHARD_SIZE" to represent the number of java
source path entries in each shard.

Sharding is not allowed when "LOCAL_JAVAC_SHARD_SIZE" and
"LOCAL_JAR_PROCESSORS" are both enabled.

Limitation:
1. 0 <= LOCAL_JAVAC_SHARD_SIZE <= 8192
1. 0 < NUM(java_sources) <= 8192
2. 0 <= NUM(shards) <= 100

Performance Compare:
<Unsharded Build------------------------------------------------>
1. Build from clean state
rm -r -f out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/
&& time m
out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/classes-full-debug.jar

real	1m2.720s user	5m26.604s sys	0m39.552s

2. Incremental build
m out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/classes-full-debug.jar &&
touch frameworks/base/core/java/android/net/http/HttpResponseCache.java && time m
out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/classes-full-debug.jar

real	0m37.586s user	5m47.804s sys	0m50.388s

<Sharded Build-------------------------------------------------->
1. Build from clean state
rm -r -f out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/
&& time m
out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/classes-full-debug.jar

Javac Shard Size: 50----real	1m10.163s user	25m59.008s sys	1m58.460s
Javac Shard Size: 100---real	1m2.115s user	21m3.600s sys	1m15.964s
Javac Shard Size: 150---real	0m59.520s user	18m10.544s sys	1m12.628s
Javac Shard Size: 200---real	0m56.894s user	15m39.244s sys	1m11.608s
Javac Shard Size: 250---real	0m55.991s user	14m38.716s sys	1m2.292s
Javac Shard Size: 300---real	0m55.114s user	13m6.568s sys	1m8.200s
Javac Shard Size: 350---real	0m53.144s user	12m7.740s sys	1m3.836s
Javac Shard Size: 400---real	0m54.929s user	12m9.324s sys	1m4.340s
Javac Shard Size: 450---real	1m30.194s user	25m31.468s sys	1m52.416s
Javac Shard Size: 500---real	0m53.976s user	10m35.500s sys	0m55.160s

2. Incremental build
m out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/classes-full-debug.jar &&
touch frameworks/base/core/java/android/net/http/HttpResponseCache.java && time m
out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/classes-full-debug.jar

Javac Shard Size: 50-----real	0m16.322s user	1m8.648s sys	0m31.700s
Javac Shard Size: 100----real	0m16.163s user	1m22.932s sys	0m29.440s
Javac Shard Size: 150----real	0m16.611s user	1m37.828s sys	0m21.168s
Javac Shard Size: 200----real	0m16.936s user	1m49.248s sys	0m28.636s
Javac Shard Size: 250----real	0m17.509s user	1m54.944s sys	0m32.768s
Javac Shard Size: 300----real	0m18.868s user	1m54.088s sys	0m28.824s
Javac Shard Size: 350----real	0m17.629s user	1m54.108s sys	0m31.056s
Javac Shard Size: 400----real	0m18.658s user	2m7.712s sys	0m30.636s
Javac Shard Size: 450----real	0m18.874s user	2m8.808s sys	0m33.540s
Javac Shard Size: 500----real	0m19.432s user	2m24.400s sys	0m30.368s

time m nothing:
real 0m5.799s user 0m7.236s sys 0m3.068s

Test: m clean && m -j checkbuild
Bug: b/67424047
Change-Id: Id0766d2b7de7c4546d29bbc7f8a0dd0e4b9ad45b
2017-10-12 13:23:06 -07:00
Treehugger Robot
b039cf792f Merge "Delete jack support" 2017-10-03 01:17:25 +00:00
Colin Cross
e673deb280 Delete jack support
Remove all support for running jack.

Bug: 65302138
Test: m -j checkbuild
Change-Id: I1ef87c88af3f2689f8b0dcf1a01c958b16159631
Merged-In: I1ef87c88af3f2689f8b0dcf1a01c958b16159631
(cherry picked from commit 5db5d31d73)
2017-10-02 11:15:19 -07:00
Dan Willemsen
4d99527dcd Move generated aidl/logtags java classes to a subfolder
So that they can't be generated into src/, which would be added to the
source list automatically.

Bug: 36698184
Test: m telephony-common without this change, pass
Test: m telephony-common without the cleanspec, fail with dup class
Test: m telephony-common with the cleanspec, pass
Change-Id: I7b2fc1847d52ff4a9ba8c69e76123fe152629ef8
2017-09-29 20:36:50 -07:00
Colin Cross
9deb331f20 Respect LOCAL_MIN_SDK_VERSION for dx
Follow Jack's behavior for now and pass LOCAL_MIN_SDK_VERSION to
dx.  Don't pass it to desugar for now, desugar with low
--min_sdk_version numbers enables features we haven't verified
yet, and existing apps won't be using the new language features
that require extra desugaring.

Test: examine classes.dex from an app with LOCAL_MIN_SDK_VERSION
Change-Id: Ic7d679b1b11fba98a33418a4f79d4762a91f1f9c
2017-09-28 21:40:35 -07:00
Treehugger Robot
b182ea7bcb Merge changes I06179fca,Ie48746c8,Ia47ebc95
* changes:
  Disable turbine for unbundled builds
  Allow disabling turbine
  Simplify the *-lib-* functions
2017-09-27 02:36:24 +00:00
Colin Cross
7dc9043546 Allow disabling turbine
m -j TURBINE_ENABLED=false will build without turbine.  This is
primarily useful for A/B comparisions of turbine builds, but will
also be used to temporarily disable turbine for unbundled builds
until we have turbine prebuilts.

Bug: 64308460
Test: m -j TURBINE_ENABLED=false java
Test: m -j java
Change-Id: Ie48746c8bfc60c361be9634cb1805ca1c09aa1fe
2017-09-26 16:23:00 -07:00
Treehugger Robot
96a782f1ea Merge "Only define proguard dictionary as output if obfuscating" 2017-09-26 21:45:28 +00:00
Dan Willemsen
bdd6a756d1 Only define proguard dictionary as output if obfuscating
If we're not obfuscating, proguard doesn't generate the mapping file,
and ninja will consider the edge dirty, since one of the output files
does not exist.

Also disable it for jack, since jack writes to a different file.

Bug: 66929996
Test: m cts; m cts
Test: m ahat-test with a clean build
Change-Id: Iff6707c44cbd86cd9328bbad8322398bc9d960fd
2017-09-26 13:01:49 -07:00
Colin Cross
b541aae17c Remove duplication of bootclasspath in LOCAL_JAVA_LIBRARIES
The bootclasspath libraries have direct dependencies now, so
remove them from LOCAL_JAVA_LIBRARIES so they don't also end
up in the classpath.

Test: m -j checkbuild
Test: m -j ANDROID_COMPILE_WITH_JACK=true java
Test: manually inspection of build-${TARGET_PRODUCT}.ninja
      only has expected changes
Test: no changes to system.img
Change-Id: Icbc8987468f237aa351708b33c3abe5ea3d1816e
2017-09-25 16:15:10 -07:00
Colin Cross
1d1e5ef61b Add dependencies on bootclasspath libraries
The libraries in the bootclasspath were getting their dependencies
because they were also in the default libraries list.  Make the
dependencies explicit in preparation for rearranging the default
libraries.

Test: m -j checkbuild
Change-Id: I3ea004714a31cd648ec2ef57ed3099bce70bc6b9
2017-09-25 16:14:42 -07:00
Miao Wang
28e78f4600 Correctly set the linker search path for libm and libc.
- ndk/r10 is removed, we need to use SOONG ndk stubs instead.
  - Use MIN_SUPPORTED_SDK_VERSION instead of hard coded version 9 for
    32 bit compilation.

Bug: 38342163
Test: mm
Test: make -j40 FORCE_BUILD_RS_COMPAT=true RSTest_Compat
Change-Id: I9549446cb7f8faee8842b06f096d035535dc7f3d
2017-09-22 14:57:02 -07:00
Richard Uhler
58e0b8c0a3 Merge "Proguard implicitly generates a proguard_dictionary" 2017-09-22 08:23:52 +00:00
Richard Uhler
b32775a9b5 Proguard implicitly generates a proguard_dictionary
Use .KATI_IMPLICIT_OUTPUTS to convey as such.

This is to support a use case for building platform/art/tools/ahat,
which depends on the generated proguard_dictionary for running tests.

Test: m
Test: m ahat-test with relevant changes from clean build.
Change-Id: Ic8f3b6819cfd4cd18620a2eaafd80d672fde8f5c
2017-09-21 10:53:09 +01:00
Nan Zhang
b88011768d Move private_manifest file insertion to jars merging phase.
By doing this, the change of private_manifest file won't trigger
Javac to re-run during re-build.

Bug: b/65455145
Test: m clean && m -j32
Change-Id: Ib90d56daf0be5f31b51e07b7714a933bc5cf4496
2017-09-20 14:56:48 -07:00
Nan Zhang
39bbfe217a Merge "Use merge_zips to merge jars during Java build in Make." 2017-09-20 19:48:27 +00:00
Nan Zhang
9bd540594d Use merge_zips to merge jars during Java build in Make.
1. Use merge_zips to merge jars without unzipping/rezipping static Java
libraries. And seperate merging process as a independent Make target so
that Javac can only depend on Turbine.

2. Remove duplicate vars: "full_java_lib_deps" in droiddoc.mk

3. Also use classes-header.jar for link_apk & link_instr

Bug: b/65455145
Test: m clean && m -j32

Change-Id: I7b0307664a6b86075c983284fce099bd3946318c
2017-09-19 18:49:33 -07:00
Treehugger Robot
68dad73d34 Merge "Move jacocoagent into the bootclasspath" 2017-09-18 19:57:53 +00:00
Colin Cross
69b7f3f0b1 Move jacocoagent into the bootclasspath
Move jacocoagent out of core-libart and into its own jar on the
bootclasspath.

Test: m -j EMMA_INSTRUMENT=true
Test: https://android-build.googleplex.com/builds/view-workplan?viewType=Table&workplanId=L19400000103297394&nodeType=Trybot
Change-Id: I9e6e3dc59dba8b93685d59b258d4ebc7835eab8b
2017-09-15 22:27:35 +00:00
Nan Zhang
870025f89b Move classes.jar generation right below JarJar phase.
By doing this, we don't need run desugar/Jacoco/proguard twice for
static java libraries(one for their self build, another for downstream
dependents).

Test: m clean && m java
Bug: b/65455145
Change-Id: I87aee34940937dbde33a977f55d1faf2c8054561
2017-09-14 14:44:40 -07:00
Nan Zhang
b3ec534bd1 Revert "Revert "Add Turbine as part of java build toolchain in Make.""
This reverts commit 33dcd0c5eb.

Reason for revert: <stub.jar.toc needs classes.jar.toc which is no longer existed after Turbine was introduced. So use classes.jar to represent stub.jar.toc>

Bug: b/64308460
Test: m clean && time m -j32 ANDROID_COMPILE_WITH_JACK=true
Change-Id: If618993f1e0c789d93cc15664212603f994fd460
2017-08-31 14:56:54 -07:00
Colin Cross
9a594e05af Remove GLOBAL_JAVAC_DEBUG_FLAGS
GLOBAL_JAVAC_DEBUG_FLAGS is no longer set, -g is already included
in the javac command line.  Jack is on its way out, just use -g
directly there for now.

Test: m -j checkbuild
Change-Id: Ib99c248bbb8b65fa2cd2b55449a502fc20deffa8
2017-08-31 14:48:31 -07:00
Nan Zhang
33dcd0c5eb Revert "Add Turbine as part of java build toolchain in Make."
This reverts commit 0739155c86.

Reason for revert: <sdk_google_aw_x86-sdk (linux) build 4310016 in git_master is broken>

Change-Id: I525d48557bd0425917e61a85f596663ccc63c887
2017-08-31 19:56:59 +00:00
Nan Zhang
0739155c86 Add Turbine as part of java build toolchain in Make.
1. Bundle static_java_header_libs during turbine-classes.jar if
necessary. This can be improved if we have the ability to merge jars on
the fly.

2. Copy sdk/stubs-classes.jar to sdk/stubs-classes-header.jar, and Use
.KATI_RESTAT to cut un-necessary targets rebuilt.

3. Copy prebuilt-classes.jar to prebuilt-classes-header.jar since
Turbine doesn't apply to prebuilt.

4. Run jarjar after Turbine compilation, otherwise downstream targets
won't find corresponding symbols.

5. Change classes.jar in -classpath used in Desugar to
classes-header.jar.

6. Change legacy-libs in Proguard from classes.jar to
classes-header.jar.

7. Add .KATI_RESTAT for turbine-classes.jar, and remove IJAR.

8.boot.art is re-generated during the incremental rebuild since
javalib.jar got re-generated. Then boot.art re-triggered lots of
downstream targets to be rebuilt. So we add .KATI_RESTAT for javalib.jar
to prevent downstream to be rebuilt. The ziptime & commit-change-to-toc
operation for javalib.jar(small) didn't bring too much overhead when we build
from clean state.

Next step: specify static_java_header_libs in -classpath, and merge jars
at the end.

touch frameworks/base/core/java/com/google/android/util/Procedure.java &&
time m
time: 1m58s (794 -> 49 targets)

m clean && time m java
time: 9m34s

Bug: b/64308460

Test: m clean && m checkbuild

Change-Id: Iefcc234405b9f461b6882c06bba15e21fa783d28
2017-08-30 16:01:39 -07:00
Colin Cross
dfc45ec337 Separate java source list generation into its own rule
Move the java source list generation step into its own rule.  This
has a couple of advantages.  It consolidates the source list
generation so that it only has to run once for javac, jack, and
jack-check.  It also massively reduces the length of the javac
command line, so that error messages are significantly shorter,
and allows easily rerunning the failing build command because
the file list is still on disk.

The primary disadvantage is that javac error messages no longer
include the list of files passed to javac, but the list is
available earlier in the build long when the file list was written,
and is still available on disk.

Test: m -j javac-check
Change-Id: I9730b352b33a060e08221b61c11c617d23320d67
2017-08-29 16:50:11 -07:00
Colin Cross
604a2423b6 Merge "Sort files in directories passed to jar"
am: 966ed15a1d

Change-Id: If01ba7d294e4308a28580b157b9efdb6c860bb35
2017-08-17 01:00:13 +00:00
Colin Cross
128800f2c8 Sort files in directories passed to jar
jar -C <dir> . produces a jar containing files in filesystem order,
which can vary between builds.  Manually find and sort the list of
files, and convert them into a list of -C <dir> <file> pairs.

Fixes: 64634025
Test: m -j checkbuild
Test: m -j out/target/product/sailfish/system/framework/ext.jar, check
      that entries are sorted
Test: m -j out/target/product/generic_arm64/system/framework/ext.jar on mac
Change-Id: I7dced6acbe621a60cd49daf17872941485602732
2017-08-16 13:37:00 -07:00
Colin Cross
4cea760106 Merge "Revert "Use dx.jar directly""
am: c7e3e962f8

Change-Id: Icd5402f7cc789224753537c070a91cbf6267ee66
2017-07-18 07:14:26 +00:00
Treehugger Robot
c7e3e962f8 Merge "Revert "Use dx.jar directly"" 2017-07-18 07:08:19 +00:00
Colin Cross
e44e75aa89 Revert "Use dx.jar directly"
This reverts commit e37a3c959a.

Reason for revert: breaks art test runner

Change-Id: Ice0cd3020ddec7f31eb789867a45f047a75a05d0
2017-07-18 05:22:59 +00:00
Colin Cross
4828fcad41 Merge "Use dx.jar directly"
am: c68434c971

Change-Id: Ief2c88d3ba0e93c7eb7bb9db0ac2b0a0b886bcea
2017-07-17 20:45:31 +00:00
Jeff Gaston
6264b662f3 Merge "Update to latest jacoco usage"
am: 483fae7430

Change-Id: Ibc94d5ba6db7690441825dd655b6cb26d989b78d
2017-07-17 20:44:10 +00:00
Colin Cross
c68434c971 Merge "Use dx.jar directly" 2017-07-17 20:40:13 +00:00
Jeff Gaston
483fae7430 Merge "Update to latest jacoco usage" 2017-07-17 20:36:39 +00:00
Colin Cross
06f8dec6e0 Merge "Fix LOCAL_JAR_PROCESSOR dependency"
am: 126e625fed

Change-Id: Ibe2786aa8853ea34513903728ae06fb5c5c39101
2017-07-14 23:10:57 +00:00
Jeff Gaston
aaae43cd24 Update to latest jacoco usage
Bug: 36792868
Test: EMMA_INSTRUMENT_STATIC=true m -j

Change-Id: I419b543283b52be9a72f5c6b10e4cbea68782174
2017-07-14 13:04:05 -07:00
Colin Cross
89b802b994 Fix LOCAL_JAR_PROCESSOR dependency
The LOCAL_JAR_PROCESSOR rule should depend on the path to the
processor jar and not the module name.

Test: m -j && m -j
Change-Id: I5ebd4cd3ad4429918edfd0a8cb5cb9a4e7e86ca7
2017-07-13 19:25:47 -07:00
Colin Cross
e37a3c959a Use dx.jar directly
Use $(JAVA) to invoke dx.jar instead of the dx wrapper script.
Also allow the dx command line or the dx jar to be specified on
the command line.  Allows better dependencies on dx.jar, and
doesn't rely on having java on the path.

Bug: 62956999
Test: m -j checkbuild
Change-Id: I5f265132690ad28b3139954fea44eb0c4a43393a
2017-07-07 13:34:33 -07:00
Przemyslaw Szczepaniak
8c485b6b0c Merge "Enable stripping of java debug information when required."
am: d659f806e5

Change-Id: I25facf64b176ae26db07d43cef92c522bf92e79a
2017-07-07 15:29:40 +00:00