Commit Graph

144 Commits

Author SHA1 Message Date
Nicolas Geoffray
02df14a67a Remove UseApexImage and GenerateApexImage.
These options are not used anymore.

Bug: 119800099
Exempt-From-Owner-Approval: Rebase after conflict
Test: m
Change-Id: I1057a6fc420b45e326ec7744b6df159f950ad853
2020-02-09 22:53:25 +00:00
Vladimir Marko
7ea8a3d9fa Use ART image for make-based preopt.
Test: taimen-userdebug boots when built with
      DEXPREOPT_USE_ART_IMAGE=true
Test: Check logcat for checksum verification failures.
      (Build ART with extra logging in OatFileAssistant.)
Test: Check that bootclasspath-checksums from some prebuilt
      oat files contain only one image checksum followed by
      dex file checksums with
          grep -az -A1 -E '^bootclasspath-checksums$' <oat-file> | \
          xargs -0 echo | gawk '{print $2}'
Bug: 119800099
Change-Id: I241b65d7323901906ffcde91ca9ba173e1755ad7
2020-02-07 14:42:09 +00:00
Ulya Trafimovich
bd98b55100 Use boot image extension in the JIT-zygote experiment.
Test: temporarily enable JIT-zygote configuration, build, boot the
  device and ensure that the JIT-zygote specific boot image
  apex-framework.art is mapped in the zygote address space:

  1. enable Jit zygote in the product device config (in my case
    device/google/muskie/aosp_walleye.mk):

    +# System server should not contain compiled code.
    +PRODUCT_SYSTEM_SERVER_COMPILER_FILTER := verify
    +
    +# Use the apex image for preopting.
    +DEXPREOPT_USE_APEX_IMAGE := true
    +
    +# Have the runtime pick up the apex image.
    +PRODUCT_PROPERTY_OVERRIDES += \
    +    dalvik.vm.boot-image=/apex/com.android.art/javalib/apex.art:/system/framework/apex-framework.art

  2. lunch aosp_walleye-userdebug \
    && m \
    && adb reboot bootloader \
    && fastboot flashall -w

  3. adb shell cat /proc/`adb shell ps | grep zygote64 | awk {'print $2'}`/maps | grep apex-framework.art
  6fe44000-7025c000 rw-p 00000000 fc:02 1179718                            /data/dalvik-cache/arm64/apex@com.android.art@javalib@apex-framework.art
  70571000-70696000 rw-p 0072d000 fc:02 1179718                            /data/dalvik-cache/arm64/apex@com.android.art@javalib@apex-framework.art
  75a339f000-75a33ac000 r--p 00852000 fc:02 1179718                        /data/dalvik-cache/arm64/apex@com.android.art@javalib@apex-framework.art

Change-Id: I6818fde087ec562057631003265bc1ec0e91688c
2020-01-10 15:29:42 +00:00
Martin Stjernholm
1e5581749d Remove generation of the dexpreopt tool paths from make.
They are now created from Soong, and therefore put in a separate config file
out/soong/dexpreopt_soong.config.

Test: m
Bug: 145934348
Change-Id: I27e710f09c37e64e8975b8e763d13434e5de71b3
2020-01-07 13:00:38 +00:00
Ulya Trafimovich
d1c9bfe87a Respect JIT-zygote config when generating boot image location.
Earlier CL Ida40dfae8c83bf7c2e737d5c7ea418e1197ad826 introduced
Soong-generated Make variable 'DEXPREOPT_IMAGE_LOCATIONS'. That CL was
erroneous in that it did not take JIT-zygote config into account and
generated identical location for "boot" and "apex" boot images.

This caused build breakages, because in case of JIT-zygote config the
two variables 'DexPreoptImages' and 'DexPreoptImageLocations' in the
module's dexpreopt.config were out of sync: 'DexPreoptImages' was
for the "apex" image, and 'DexPreoptImageLocations' was for the "boot"
image.

CL I9a91fc48e54d7d43abec2cb2b5a11e3581db380b introduced a workaround
for this problem: incorrect 'DexPreoptImageLocations' from the module
dexpreopt.config was ignored, and instead boot image location was
manually reconstructed from 'DexPreoptImages'. This workaround would
not work when we start using boot image extension and location will
become more complex.

This CL fixes the way 'DexPreoptImageLocations' is generated by
spliting the 'DEXPREOPT_IMAGE_LOCATIONS' variable in two variables
depending on the boot image flavour "boot" of "apex". This is
aligned with the way other similar variables are generated.

Test: aosp_walleye-userdebug boots.
Test: walleye_jitzygote-userdebug builds
  (on git_rvc-release branch with this CL cherry-picked).

Change-Id: I449c968909635dd8cc431323fccbc7fce440fea5
2020-01-06 15:58:52 +00:00
Ulya Trafimovich
3ad6f5ebbf Add boot image locations to the module dexpreopt config.
Multiple boot image locations should be passed to dex2oat after
splitting the boot image into primary ART image and the framework
boot image extension.

Test: m

Bug: b/119800099

Change-Id: Ide6dfed1d09a6c12aad6f1cc0d4af4fb2c63a6be
2019-12-03 10:27:26 +00:00
Ulyana Trafimovich
213a7d86d5 Revert^2 "Do not dexpreopt system server jars from updatable modules."
This reverts commit fa7cf280ca.

Reason for revert: Build failure is not reproducible.
Forrest build on the same build ID 6033773 and same target
cf_x86_phone-userdebug_coverage finished successfully.

Change-Id: I188623237dfba657fb48966e0a34a975f13d00d2
2019-11-27 12:29:42 +00:00
Ulyana Trafimovich
fa7cf280ca Revert "Do not dexpreopt system server jars from updatable modules."
This reverts commit 3fb6a949a6.

Reason for revert: broken build git_master/cf_x86_phone-userdebug_coverage (likely caused by these changes).

Exempt-From-Owner-Approval: revert.

Change-Id: Ib72fe0395fd084e774d25ee783b7380395f9d9bd
2019-11-26 13:06:13 +00:00
Ulya Trafimovich
3fb6a949a6 Do not dexpreopt system server jars from updatable modules.
Test: m

Test: The list of updatable system server jars is empty now,
  so I tested that the filer works with a manual experiment:
  - temporarily add wifi-service to PRODUCT_UPDATABLE_SYSTEM_SERVER_JARS
  - m nothing
  - fgrep -e 'wifi-service dexpreopt' $ANDROID_BUILD_TOP/out/build-$TARGET_PRODUCT.ninja
  - expect empty output (no dexpreopt command is generated)

Change-Id: Ia4206ccc263b535b6e39b54c6e8db28845ff2bf5
2019-11-25 16:36:52 +00:00
Nicolas Geoffray
2ca0e49f61 Remove support for stripping dex.
Stripping is incompatible with ART module updatability.

Bug: 65154345
Bug: 138851227

Test: build and observe no change in output (stripping is not used by
default).

Change-Id: Ic2d9738ef393814c3af3cad116071f7b5938aa84
2019-10-21 09:36:45 +01:00
Dan Willemsen
cb4c357661 Use dexpreopt image deps from Soong
Instead of just depending on the single image file, depend on the entire
image that gets loaded -- the .oat / .vdex files, and the modules that
are part of the image that have their own files.

This is necessary for RBE, where the rule only gets the files that it
depends upon.

Test: treehugger
Test: build a system image with RBE
Change-Id: I93a580b42cf4ef1824ee1397e551c065f083348a
2019-06-18 01:19:04 +00:00
Colin Cross
56199af6c2 Move verify_uses_libraries.sh out of dexpreopt
dexpreopt usually gets a dex jar instead of the final APK, which
means targetSdkVersion can't be parsed out of it.  Move the
shared library verification to a tool that operates on the final
AndroidManifest.xml instead.  verify_uses_libraries.sh is still
used to verify prebuilts where we don't have an AndroidManifest.xml
and must parse it out of the APK.

Test: m Gallery2
Bug: 132357300
Change-Id: I6ade74b6144c73aee094f5d5ff343067ca0a0e5a
2019-05-26 17:42:17 +00:00
Colin Cross
539543990e Don't add dependency on missing optional shared libraries
Filter out LOCAL_OPTIONAL_USES_LIBRARIES entries that are listed
in INTERNAL_PLATFORM_MISSING_USES_LIBRARIES before using them as
a dependency.

Bug: 132357300
Test: m Gallery2
Change-Id: I09bf331f7dc95d1aaa8e5176603bcd343dd93001
2019-05-25 22:02:36 -07:00
Colin Cross
51ba1df2f0 Use a copy of dexpreopt.config
Move the dexpreopt.config passed to Soong to out/soong/dexpreopt.config
and let it write a new copy in a rule and pass the path back to make
as DEX_PREOPT_CONFIG_FOR_MAKE.

Bug: 132379813
Test: m checkbuild
Change-Id: Iac29beac3af8864d36d1563ce42bed11b685548d
2019-05-09 22:16:13 -07:00
Nicolas Geoffray
19abe716f8 Add a flag to select the apex boot image for preopt.
Test: m && check that odex files are preopted with the apex image
Bug: 119800099
Change-Id: I5b64261cd765b511ad7e1ff513b85c0579023f6e
2019-03-08 14:41:14 +00:00
Nicolas Geoffray
269bc741a5 Support for install multiple boot images.
Bug: 119800099
Test: m && /system/framework/arm/apex*.art files exist
Change-Id: I78023f1086446c2c44ff6b282993537b053c0588
2019-02-22 15:33:23 +00:00
Colin Cross
36b5d1cd47 Update calls to dexpreopt_gen
dexpreopt_gen now requires a -out_dir argument, and does not need
DexPath, StripInputPath and StripOutputPath set.

Also use paths to Soong binaries for dex2oat, profman and aapt
so that they can be loaded as Path objects in Soong, and let
Soong handle the default for BootImageProfiles.

This reapplies I97b1865fd98d5108d9c63ff7a90a5591c7622526 with
fixes to disable preopt for Soong-only builds when the global
dexpreopt.config doesn't exist.

Test: m checkbuild
Change-Id: I5af9b749235823b7e1cfcfc55e121bfec413fb2f
2019-02-20 21:46:15 -08:00
Colin Cross
862ab81669 Revert "Update calls to dexpreopt_gen"
This reverts commit 1d8df4a97f.

Reason for revert: broke ndk build

Change-Id: I99ab62653fc15b985edd42301ec085ba68389f51
2019-02-21 05:03:11 +00:00
Colin Cross
1d8df4a97f Update calls to dexpreopt_gen
dexpreopt_gen now requires a -out_dir argument, and does not need
DexPath, StripInputPath and StripOutputPath set.

Also use paths to Soong binaries for dex2oat, profman and aapt
so that they can be loaded as Path objects in Soong, and let
Soong handle the default for BootImageProfiles.

Test: m checkbuild
Change-Id: I97b1865fd98d5108d9c63ff7a90a5591c7622526
2019-02-20 11:14:14 -08:00
Colin Cross
47e384c3d6 Remove dexpreopting of boot jars from Make
Dexpreopting of boot jars is now implemented in
build/soong/java/dexpreopt_bootjars.go.

Test: m checkbuild
Change-Id: I036510e7674b876976ce191c5b18bf0b7a15c6ee
2019-02-15 13:00:14 -08:00
Nicolas Geoffray
cd75414f1e Merge "Revert "Revert "Never strip and store dex files uncompressed when they are preopted on system.""" 2019-02-13 11:59:39 +00:00
Nicolas Geoffray
98f5e86660 Revert "Revert "Never strip and store dex files uncompressed when they are preopted on system.""
This reverts commit a81dad1bc4.

Test: build && atest android.text.cts.EmojiTest#testEmojiGlyphWebView on Cuttlefish
Change-Id: Ie3c43830fc9a994ed35883abb8ca71d97bd5c44d
2019-02-12 14:00:38 +00:00
Colin Cross
87378f1888 Move dexpreopt image selection out of dexpreopt_gen
Instead of passing both the module and global dexpreopt image
selection into dexpreopt_gen, determine the correct dexpreopt
image in Make.  This allows the Make dexpreopt rule to only
depend on the necessary boot images, preventing an order-only
dependency on the 32-bit boot image for a 64-bit only module.

Also stop using the boot image "location" as an input, only
track the real path, which will be converted to the "location"
that dex2oat expects when constructing the command line in
dexpreopt_gen.

Removes support for the unused LOCAL_DEX_PREOPT_IMAGE_LOCATION
variable.

Test: m checkbuild
Change-Id: I2e3e1d30fdfed92fb08e6cc1c9515edf8d99c7f0
2019-02-11 15:25:07 -08:00
Colin Cross
05ef66fc0b Merge "Revert "Never strip and store dex files uncompressed when they are preopted on system."" 2019-02-11 19:31:31 +00:00
Colin Cross
a81dad1bc4 Revert "Never strip and store dex files uncompressed when they are preopted on system."
This reverts commit 9cdc3beb3d.

Reason for revert: b/123436620

Change-Id: I45fa9236e37c3b7e2d44ede62a2701db2bc8a20d
2019-02-11 19:30:39 +00:00
Nicolas Geoffray
f07d63390c Merge "Never strip and store dex files uncompressed when they are preopted on system." 2019-02-11 10:31:12 +00:00
Nicolas Geoffray
9cdc3beb3d Never strip and store dex files uncompressed when they are preopted on system.
In order for the runtime module to always be able to compile apps,
make sure we keep a copy of the dex files optimally.

Gated by a product flag if a product doesn't include the module yet.

Test: build
Change-Id: Ia9bba7f6f12f3400078a43d39ba4134cf8818399
2019-02-08 22:45:48 +00:00
Colin Cross
fceb298ff7 Remove unused dexpreopt UseEmbeddedDex
dexpreopt doesn't need to know about UseEmbeddedDex, it has
already been taken into account for UncompressedDex.

Test: m checkbuild
Change-Id: I546bdaf604552792a2ac9b3557811da7418b37d1
2019-02-07 22:29:45 +00:00
Victor Hsieh
06769c7efb Rename preferCodeIntegrity to useEmbeddedDex
Test: build and run testing apps
Bug: 112037137
Change-Id: Ie8cd8e141df097095310ecc57b6460308210dea5
2019-01-29 13:03:22 -08:00
Sundong Ahn
a9f2ef5ed1 Change the impl lib name of java_sdk_library
The impl lib name of java_sdk_library is changed to {module_name}.jar
instead of {module_name}.impl.jar

Bug: 110404779
Test: m -j
Change-Id: I275cd40e3682640b96425e56698cea770888cbb1
Merged-In: I275cd40e3682640b96425e56698cea770888cbb1
(cherry picked from commit 0372f189ac)
2019-01-17 09:42:57 +09:00
Colin Cross
723a22feea Support LOCAL_DEX_PREOPT := nostripping
Support LOCAL_DEX_PREOPT := nostripping by passing it to
dexpreopt_gen.

Bug: 122610462
Test: m
Change-Id: I05f7009d38505f995cd16cc8dbb4ecd6ecc3d71c
2019-01-09 21:27:53 -08:00
Victor Hsieh
330d21d53b Rename to LOCAL_PREFER_CODE_INTEGRITY
During code review, the name change was suggested.

Test: local CTS passed
Bug: 112037137
Change-Id: I1782c532ad6dd36ee8aad7987e47a14f5c1895ee
2019-01-03 09:48:39 -08:00
Colin Cross
cff1cee3f1 Hide unzip warning when zip file is empty
unzip prints a warning and exits with code 1 when a zip file is empty.
Pipe the output of unzip through grep -v to hide the warning,
using PIPESTATUS to preserve the exit code of unzip, and then
filter out exit codes that are not 0 or 1.

Test: m
Change-Id: If2cf725fc3315d5996d5a7190288df46d84a59e2
2018-12-19 23:47:10 +00:00
Nicolas Geoffray
c04f3126f4 Fix build time preopt with shared library uses.
- Handle hidl libraries.
- Handle new shared library encoding.

Test: no mismatch preopt during boot.
bug: 117845483
bug: 111174995
Change-Id: Ie4bca653594c72df2eb2fdd827bc73cfcb76b1d7
2018-12-18 17:33:02 +00:00
Colin Cross
6db5b0ea9a Move dexpreopting to Soong
Move the dexpreopting logic into Soong.  Make modules will be
dexpreopted by executing the Soong logic in the standalone
dexpreopt_gen binary, which will generate scripts that will
perform dexpreopting for each module.  Export global configuration
as JSON to $OUT/dexpreopt.config, which will be used by
dexpreopt_gen and Soong, and per-module JSON configuration that
will be used by dexpreopt_gen.

This relands I59b20c931ee3e5a8d35eb30da4148691c5095502,
I39d580999947ee54cfefe875b57a028be5333bd7,
Ie7daa94e107d53eff075ca58dbe721bd9d7fc8c2 and
Ica006a007d112c232311435aaac0c0e476232b67, with a minor
update to match the changes made to dexpreopt_gen arguments
and a fix to correctly keep dexpreopt disabled on mac builds.

Bug: 119412419
Bug: 120273280
Test: no differences to dexpreopt outputs on aosp_sailfish system/,
      only expected changes to dexpreopt outputs on system_other
      (.vdex files for privileged Soong modules no longer incorrectly
      contain .dex contents).
Change-Id: I25163e91886cea6941afa25cdb529ed053278dcb
2018-12-17 13:46:17 -08:00
Colin Cross
b8901d7fb8 Revert "Move dexpreopting to Soong"
This reverts commit e736c58043.

Test: none
Bug: 119412419
2018-12-14 11:49:55 -08:00
Colin Cross
24258b1fb1 Revert "Create directories before unzipping dexpreopt zip"
This reverts commit 9eaeb56c24.

Test: none
Bug: 119412419
2018-12-14 11:49:55 -08:00
Colin Cross
10846f99fb Revert "Correctly create directories before unzipping dexpreopt zip"
This reverts commit 2bb1f518fd.

Test: none
Bug: 119412419
2018-12-14 11:49:55 -08:00
Colin Cross
2bb1f518fd Correctly create directories before unzipping dexpreopt zip
Ie7daa94e107d53eff075ca58dbe721bd9d7fc8c2 attempted to pre-create
the directories that unzip would unzip into, but incorrectly
created them in the top of the source tree instead of in
$(PRODUCT_OUT).  Fix the directory location and add a cleanspec
to clean up the incorrect directories in the source tree.

Bug: 119412419
Test: m correctly cleans up incorrect directories in source tree
Test: extract unzip command from out/verbose.log.gz and add bash -x
Change-Id: Ica006a007d112c232311435aaac0c0e476232b67
2018-12-13 23:32:06 -08:00
Colin Cross
9eaeb56c24 Create directories before unzipping dexpreopt zip
unzip sometimes fails with:
checkdir error: cannot create out/target/product/.../system/framework/oat
File exists

I think this happens when two unzips run in parallel, see that the
parent directory is missing, race to create it, and then one of them
treats the EEXIST when creating it as an error instead of continuing.
Work around this by creating the directories with mkdir -p before
running unzip.

Test: m installclean && m
Bug: 119412419
Change-Id: Ie7daa94e107d53eff075ca58dbe721bd9d7fc8c2
2018-12-13 18:32:57 -08:00
Colin Cross
e736c58043 Move dexpreopting to Soong
Move the dexpreopting logic into Soong.  Make modules will be
dexpreopted by executing the Soong logic in the standalone
dexpreopt_gen binary, which will generate scripts that will
perform dexpreopting for each module.  Export global configuration
as JSON to $OUT/dexpreopt.config, which will be used by
dexpreopt_gen and Soong, and per-module JSON configuration that
will be used by dexpreopt_gen.

Bug: 119412419
Bug: 120273280
Test: no differences to dexpreopt outputs on aosp_sailfish system/,
      only expected changes to dexpreopt outputs on system_other
      (.vdex files for privileged Soong modules no longer incorrectly
      contain .dex contents).
Change-Id: I59b20c931ee3e5a8d35eb30da4148691c5095502
2018-12-12 17:21:22 -08:00
Mathieu Chartier
cc4ca50cfb Change profman logging to only be for errors
Move profman logging to only be for errors. Avoids build spam.

Bug: 120907449
Test: make
Change-Id: I0707dc5c8d6516cbf95d263864bdddaf5a5990c9
2018-12-12 10:28:02 -08:00
Colin Cross
e0006c91ca Merge "Pass privileged dexpreopt module configuration to Soong" 2018-10-26 00:00:12 +00:00
Victor Hsieh
e53dd7b20b Allow target to specify LOCAL_PREFER_INTEGRITY
The option will:
 - Produce APK with uncompressed dex
 - Declare to prefer integrity in manifest

Test: unzip -vl, dex compression looks correct with the option provided
      or not.
Test: similarly, aapt dump correct attribute
Bug: None

Change-Id: I16b9a37255150c2ad84af84087dfabb536a3b07a
2018-10-24 11:40:07 -07:00
Colin Cross
3aa2f8f69b Pass privileged dexpreopt module configuration to Soong
Pass DONT_UNCOMPRESS_PRIV_APPS_DEXS, PRODUCT_LOADED_BY_PRIVILEGED_MODULES,
DEX_PREOPT_DEFAULT, and WITH_DEXPREOPT to Soong so it can determine when
to store uncompressed dex files in APKs and when to strip them.

Also convert DEXPREOPT.$(TARGET_PRODUCT).$(LOCAL_MODULE).CONFIG=disabled
to a DEXPREOPT_DISABLED_MODULES list, and export it to Soong.

Test: m checkbuild
Change-Id: I8652032d4b9f5d2e119e73acdf91b8148563eedd
2018-10-23 22:33:07 -07:00
Colin Cross
fc782ad949 Reduce the propagation of LOCAL_DEX_PREOPT := nostripping
nostripping causes confusing double negatives, allow nostripping
in LOCAL_DEX_PREOPT and DEFAULT_DEX_PREOPT, but convert to
LOCAL_STRIP_DEX outside of dex_preopt_odex_install.mk.

Test: m checkbuild
Change-Id: I996e9258ce20c394900d9fe937d638bc2ab8589d
2018-10-23 22:33:07 -07:00
Colin Cross
ff999725ac Merge changes I6a4060a7,I7743fbda,I88eb24f8
* changes:
  Unnest independent dexpreopt conditionals
  Indent dex_preopt_odex_install.mk
  Revert "Build: Do not treat org.apache.http.legacy.boot as boot classpath"
2018-10-10 22:35:49 +00:00
Colin Cross
c18757e34d Unnest independent dexpreopt conditionals
Most of the 6 nested conditionals are independent and can be
evaluated individually.  This has a slight change in behavior,
disabling preopt for a module in the product will now override
LOCAL_DEX_PREOPT, but that seems preferable.

Test: m
Change-Id: I6a4060a78df729c34bbf6978c84993859a947bf7
2018-10-10 10:12:03 -07:00
Po Hu
eaef4d8b9f Initialize variable to fix wrong odex installed
When "PRODUCT_DEX_PREOPT_GENERATE_DM_FILES := true" and
"PRODUCT_DEX_PREOPT_DEFAULT_COMPILER_FILTER := verify" are set,
boot jar may wrongly refer to previous module's variables such as
my_generate_dm, my_built_dm and my_installed_dm. And then cause
unexpected files installed along with this boot jar.

So initialize my_generate_dm to empty at first.

Bug: 117526952
Test: make out/target/product/generic/module-info.json
Test: cat out/target/product/generic/module-info.json | grep '"apache-xml":'
Change-Id: Iaa851f4fb530b3446de3dbcb1f54e545938830ce
2018-10-10 13:40:15 +08:00
Colin Cross
0a69bb6c90 Indent dex_preopt_odex_install.mk
Test: m checkbuild
Change-Id: I7743fbda98d1f1ea1d3e8242905239ff0115b6a9
2018-10-09 21:20:21 -07:00