Commit Graph

17171 Commits

Author SHA1 Message Date
Joseph Murphy
9fd9b92d51 Merge "Revert "Also package recursive jni_libs deps of android_apps as well as direct deps."" 2019-12-18 00:23:36 +00:00
Jaewoong Jung
425dd46e31 Declare manifest as input to test config fixer.
am: f192d55382

Change-Id: Ia10e74a3176d47b3324c44f52e6fbad9a6147bc8
2019-12-17 16:18:52 -08:00
Lorenzo Colitti
2973c1106f Revert "Also package recursive jni_libs deps of android_apps as well as direct deps."
This reverts commit 6f907ad3dd.

Reason for revert: Broke FrameworksNetSmokeTests

Bug: 146456945
Change-Id: Ibef7bb80c532e70cfcfb974f51a99ed25437a343
2019-12-18 00:15:07 +00:00
evitayan
13b0d51ffb Enable covergae for ike.jar which will be in mainline module IPsec(IKE)
Enable jacoco for ike.jar to set up code coverage. ike will be
shipped in mainline module com.android.ipsec

Bug: 146012420
Test: m EMMA_INSTRUMENT=true EMMA_INSTRUMENT_FRAMEWORK=true SKIP_BOOT_JARS_CHECK=true WITH_DEXPREOPT=false
Change-Id: I886551d980c0d9651b11c80051f17609bb3531d8
2019-12-17 15:23:38 -08:00
Colin Cross
fb0c16e95a Move imageMutator before archMutator
Run the imageMutator between osMutator and archMutator so that
different arch variants can be set for the different partitions.

Bug: 142286466
Test: m checkbuild
Change-Id: I65d05714b75aa462bf9816da60fdc2deda4de593
Merged-In: I65d05714b75aa462bf9816da60fdc2deda4de593
(cherry picked from commit 9c8f687584)
2019-12-17 14:45:11 -08:00
Jaewoong Jung
f192d55382 Declare manifest as input to test config fixer.
It needs to be an implicit input for it to work consistently.

Test: m GooglePermissionControllerUnitTests
Bug: 145011263
Change-Id: Ib71d768e750faa5b577652e79a18d05929bde52f
2019-12-17 14:22:19 -08:00
Peter Collingbourne
d6800751c7 Merge "Also package recursive jni_libs deps of android_apps as well as direct deps."
am: ff8cb1e69e

Change-Id: I22f9f842f7125a5ff86abedd70387456bd26210f
2019-12-17 12:55:13 -08:00
Peter Collingbourne
ff8cb1e69e Merge "Also package recursive jni_libs deps of android_apps as well as direct deps." 2019-12-17 20:51:18 +00:00
Jooyung Han
27151d93b6 Override package name for VNDK APEX
com.android.vndk.v<VER> is overridden with
com.google.android.vndk.v<VERE>.

Test: m com.android.vndk.current
    && aapt2 dump badging ../system/apex/com.android.vndk.current.apex
Change-Id: I623d7a144418d7198f0bbf09c5dc983ed1a631c6
Merged-In: I623d7a144418d7198f0bbf09c5dc983ed1a631c6
(cherry picked from commit 0ed1b1b169)
2019-12-17 15:22:21 +00:00
Jiyong Park
a92fbb3b72 Revert "Avoid duplicated classes for boot dex jars"
This reverts commit 2812df4edb.

This is no longer required as we have
I6463ebc59cf7fd861b812999d7a79c387bbb3335.

hostdex files are available even when the module doesn't have
"//apex_available:platform"

Bug: 128708192
Test: m
Change-Id: I0d1295683222bc9cdbb82630e61ab3421c717daf
2019-12-17 20:35:42 +09:00
Jooyung Han
f121a657c9 Fix apex to override certificate
The rules for apex certificate:
1. <unspecified>: use <default app cerficicate>
2. name: use <default app cerficiate dir>/<name>(.x509.pem|.pk8)
3. :module: use specified by <module>

Certificates can be overridden by PRODUCT_CERTIFICATE_OVERRIDES.

Currently, 1) and 2) aren't overridden by PRODUCT_CERTIFICATE_OVERRIDES,
which should be.

Bug: n/a
Test: m (apex_test.go amended)
Change-Id: Icbdf4979613ef10127ecc02f3debd6a798460532
2019-12-17 15:20:50 +09:00
Jooyung Han
671f1ce75e Fix apex to follow deps of modules with stubs.
In case that an apex module depends on a module with stubs directly
*and* indirectly, the build system should follow the deps graph further.

Note that WalkDeps() visits deps in DFS and it won't visit again visited
modules.

Bug: n/a
Test: m (apex_test.go amended)
Change-Id: I1a6f135dbda6d1eb641575a3fbbc9bbee0622076
2019-12-17 15:18:54 +09:00
Jiyong Park
880fd78f33 Merge "hostdex:true modules are available for platform"
am: 7cf4b5de6b

Change-Id: Ia8ae86ffef5e2e3015cdfc3832798b54e0379f18
2019-12-16 22:04:52 -08:00
Treehugger Robot
7cf4b5de6b Merge "hostdex:true modules are available for platform" 2019-12-17 06:02:05 +00:00
Jooyung Han
dc1c78fcd1 Merge "Support apps attribute in apex_defaults"
am: 1d1a920794

Change-Id: Ibfa92edacbb7c041d6d1ac01f5a0baf64b9562a3
2019-12-16 21:48:50 -08:00
Treehugger Robot
1d1a920794 Merge "Support apps attribute in apex_defaults" 2019-12-17 05:44:42 +00:00
Jiyong Park
a40bb59741 Merge "Avoid duplicated classes for boot dex jars"
am: 97e09d0dee

Change-Id: Iaf95a4cd63f57df1b5e6fab64c382308f3682e0d
2019-12-16 21:19:43 -08:00
Treehugger Robot
97e09d0dee Merge "Avoid duplicated classes for boot dex jars" 2019-12-17 05:14:25 +00:00
Peter Collingbourne
6f907ad3dd Also package recursive jni_libs deps of android_apps as well as direct deps.
Previously, android_app targets for which a.shouldEmbedJnis(ctx) = true
(e.g. CtsSelinuxTargetSdk25TestCases) would need to specify all of their
recursive library dependencies, including for example libc++ when depending
on the platform libc++. This means unnecessary churn when we add a new
dependency to libc++ (e.g. libunwind [1]). To avoid the churn and allow
jni_libs clauses to be simplified, make the build system search for the
recursive dependencies and automatically include them.

This change allows us to remove code that was previously adding NDK libc++
as a special case, as it is now covered by the generic code.

Also fix some improper quoting that was exposed as a result of this change
causing more files to be packaged than before.

[1] https://android-review.googlesource.com/q/topic:%22libunwind-so%22

Bug: 144430859
Change-Id: I3d6fbcce75bc108a982eb7483992a4b202056339
2019-12-16 15:20:09 -08:00
Mitch Phillips
630f2d4caf Merge "Add $ORIGIN to fuzzer-coverage sanitized .so's."
am: a1682631eb

Change-Id: I5748c74b623488b82c94c82ac0a14839f188dc26
2019-12-16 13:23:53 -08:00
Nick Desaulniers
7346c709e2 Merge changes from topic "separate_code"
am: 894cdee30a

Change-Id: I8e20f744c85a7439569fe5f0464c74e91d060c30
2019-12-16 13:20:22 -08:00
Nick Desaulniers
c2e2dce120 Revert "Revert "soong: upgrade Android platform to clang-r370808""
am: eb20744361

Change-Id: Ia0c47b43fcb53da68caf6496fb2a421c74581a02
2019-12-16 13:20:15 -08:00
Mitch Phillips
a1682631eb Merge "Add $ORIGIN to fuzzer-coverage sanitized .so's." 2019-12-16 21:17:03 +00:00
Nick Desaulniers
894cdee30a Merge changes from topic "separate_code"
* changes:
  soong: use -Wl,-z,separate-code w/ -Wl,--execute-only
  Revert "Revert "soong: upgrade Android platform to clang-r370808""
2019-12-16 21:13:28 +00:00
Colin Cross
2387d4f65e Merge "Use empty string for core image variant"
am: bd0624304e

Change-Id: I0ccff110f6661a47e303c1a84ae18d6adfcebaa9
2019-12-16 13:06:12 -08:00
Treehugger Robot
bd0624304e Merge "Use empty string for core image variant" 2019-12-16 20:54:32 +00:00
Nick Desaulniers
dcee1e5c54 soong: use -Wl,-z,separate-code w/ -Wl,--execute-only
The clang-r370808 upgrade contains a change to LLD allow PT_LOAD
segments to reside at non-multiples of the page size in the resulting
object file.  https://reviews.llvm.org/rL369344

While this helps reduce the alignment waste and resulting image size, it
has interesting implications for execute only memory (XOM): The runtime
loader will now load code or data from other segments into pages with
different protections than intended.

This would partially defeat execute only (XOM) text sections as the
segment could now overlap with previous and following sections. This
might allow for code or data from the preceding and following sections
(like .eh_frame, and .data.rel.ro) to be executable, and either ends of
.text to be readable.

When the runtime loader (linker[64]) `mmap`s segments from *.so files,
the file offset parameter (see `man 2 mmap`) MUST be a multiple of the
page size.  Since the updated LLD can now pack segments in a file (which
helps minimize resulting object file size) (previously, the segment
offsets were page aligned), this has interesting implications.

To appreciate the current bug, consider the following output from
`readelf` before this patch is applied, but after the toolchain upgrade:

```
$ readelf -lSW $OUT/symbols/apex/com.android.runtime/lib64/bionic/libc.so
...
  [Nr] Name              Type            Address          Off    Size   ES Flg Lk Inf Al
...
  [13] .eh_frame         PROGBITS        000000000002e7c0 02e7c0 013374 00   A  0   0  8
  [14] .text             PROGBITS        0000000000042b40 041b40 09ecb4 00  AX  0   0 64
  [15] .plt              PROGBITS        00000000000e1800 0e0800 001f30 00  AX  0   0 16
  [16] .data.rel.ro      PROGBITS        00000000000e4740 0e2740 005208 00  WA  0   0 32
...
  Type           Offset   VirtAddr           PhysAddr           FileSiz  MemSiz   Flg Align
  PHDR           0x000040 0x0000000000000040 0x0000000000000040 0x000230 0x000230 R   0x8
  LOAD           0x000000 0x0000000000000000 0x0000000000000000 0x041b34 0x041b34 R   0x1000
  LOAD           0x041b40 0x0000000000042b40 0x0000000000042b40 0x0a0bf0 0x0a0bf0   E 0x1000
  LOAD           0x0e2740 0x00000000000e4740 0x00000000000e4740 0x006720 0x006720 RW  0x1000
...
   01     .note.android.ident .note.gnu.build-id .dynsym .gnu.version .gnu.version_d .gnu.version_r .gnu.hash .dynstr .rela.dyn .rela.plt .rodata .eh_frame_hdr .eh_frame
   02     .text .plt
   03     .data.rel.ro .fini_array .init_array .dynamic .got .got.plt
...

The above output tells us:
1. .text will wind up in the third (02) segment.
2. The third segment will be (LOAD)'ed as (E)xecutable.
3. Because the file (Offset) of the first segment (0x41b40) is NOT a
   multiple of the page size, it cannot be passed as the `offset` to
   `mmap`. As such it will be rounded down to the first multiple of the
   page size, 0x41000.
4. The preceding section (.eh_frame) will be loaded in the preceding
   segment (01). It occupies file (Off)set range [(0x2e7c0):0x41b34].
   0x41b34 is not explicit in the output, instead you must use the
   formula:
     Off     + Size    == End
   ie.
     0x2e7c0 + 0x13374 == 0x41b34
   (This happens to match (FileSiz) of the second segment, which makes
   sense as .eh_frame is the final section in the second segment.)
5. mmap'ing file offset 0x41000 when loading the second segment will
   include 0x4c0 bytes (0x42000 - 0x41b40) from .text, now mapped as
   readable (oops). Suddenly code from .text is now readable (and thus
   scannable for gadgets for ROP chains).
6. mmap'ing file offset 0x41000 when loading the third segment will
   include 0xb34 bytes (0x41b34 - 0x41000) from .eh_frame, now mapped as
   executable (oops). Suddenly data from .eh_frame is now executable
   (and thus a potential gadget for ROP chains).
7. mmap'ing file offset 0xe2000 when loading the third segment will
   include 0x8CO bytes (0xe3000 - 0xe2740) from .data.rel.ro, now mapped
   as executable (oops). Suddenly data from .data.rel.ro is now
   executable (and thus a potential gadget for ROP chains).
8. mmap'ing file offset 0xe2000 when loading the fourth segment will
   include 0x730 bytes (0xe0800 + 0x1f30 - 0xe2000) from .plt, now
   mapped as readable (oops). Suddenly data from .plt is now readable
   (and thus scannable for gadgets for ROP chains).

All these oops' could be avoided if the linker placed .text+.plt at page
size aligned file offsets, which is what `-Wl,-z,separate-code` code
does.  After this patch, we have:

```
$ readelf -lSW $OUT/symbols/apex/com.android.runtime/lib64/bionic/libc.so
...
  Type           Offset   VirtAddr           PhysAddr           FileSiz  MemSiz   Flg Align
  PHDR           0x000040 0x0000000000000040 0x0000000000000040 0x000230 0x000230 R   0x8
  LOAD           0x000000 0x0000000000000000 0x0000000000000000 0x041b34 0x041b34 R   0x1000
  LOAD           0x042000 0x0000000000042000 0x0000000000042000 0x0a0be0 0x0a0be0   E 0x1000
  LOAD           0x0e3000 0x00000000000e3000 0x00000000000e3000 0x006720 0x006720 RW  0x1000
```

In the future, we could go back to tightly packing segments in the
binary if the runtime loader was improved to detect the previously
stated problem, and `memset` over the problematic ranges of the freshly
`mmap`ed pages (implying additional startup cost for reduced binary
size). This might save ~6 KB from each native binary, which adds up to
~17 MB for an AOSP image.

Also, prefer
-Wl,--execute-only
rather than
-Wl,-execute-only

Bug: 139945549
Bug: 146144180
Test: readelf -lSW $OUT/symbols/apex/com.android.runtime/lib64/bionic/libc.so
Change-Id: I64527e034ca3c71565ea52ed06f81f75d5216627
Reported-by: Ryan Prichard <rprichard@google.com>
Suggested-by: Fangrui Song <maskray@google.com>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
2019-12-16 09:55:37 -08:00
Jaewoong Jung
73a0b33360 Merge "Optionally overwrite package value in test config."
am: 9b1084f9b9

Change-Id: I7be612d317e79a00d430009ec8d2f352469c6cf9
2019-12-16 09:13:26 -08:00
Jaewoong Jung
9b1084f9b9 Merge "Optionally overwrite package value in test config." 2019-12-16 17:10:18 +00:00
Anton Hansson
ef76ca7f43 Fix duplicate sentence in visibility documentation
am: f56f74be4a

Change-Id: I7a67434bfdd1b9f67869044ea96c49743ad3abd6
2019-12-16 06:09:34 -08:00
Jooyung Han
f21c797f38 Support apps attribute in apex_defaults
Adding overridableProperties to apex_defaults as well.

Bug: 146217017
Test: m
Change-Id: I51182fec25ee589fea4006164c4dd69acc5ba67d
2019-12-16 13:36:34 +00:00
Anton Hansson
f56f74be4a Fix duplicate sentence in visibility documentation
This paragraph was duplicated.

Test: treehugger
Change-Id: I6057b96e310ac0ebf8f6f2a86f93efafd189579e
2019-12-16 11:39:17 +00:00
Jiyong Park
a838ea2fbc Merge "ignore apex_available for host variants"
am: 7829a9bf9e

Change-Id: I80067f1f149a812fd06bfbe10349fb0d0743ef29
2019-12-16 02:24:25 -08:00
Treehugger Robot
7829a9bf9e Merge "ignore apex_available for host variants" 2019-12-16 10:21:35 +00:00
Jiyong Park
c3aa31b02c Merge "java libs that isn't installable is gracefully rejected"
am: c829ff371e

Change-Id: I42430cc748b0109ecb683dc227c07b906c2a1f9d
2019-12-16 02:09:23 -08:00
Treehugger Robot
c829ff371e Merge "java libs that isn't installable is gracefully rejected" 2019-12-16 09:56:13 +00:00
Jiyong Park
479321dc34 java libs that isn't installable is gracefully rejected
... rather than causing soong panic.

Bug: 146197571
Test: m (apex_test amended)
Change-Id: I3679a4e3041527533c1c05d54f3aa514abd6095b
2019-12-16 15:59:54 +09:00
Jiyong Park
b02bb40f33 hostdex:true modules are available for platform
Java libraries with hostdex: true are available for the platform even if
it doesn't have "//apex_available:platform" in the apex_available
property. Note that the java libraries are still prevented from being
installed to the device.

Bug: 128708192
Test: m

Change-Id: I6463ebc59cf7fd861b812999d7a79c387bbb3335
2019-12-16 15:19:13 +09:00
Jiyong Park
2812df4edb Avoid duplicated classes for boot dex jars
When a boot classpath java library is directly or indirectly included in
APEXes, multiple variant of the library exist. When running the
hiddenapi tool, we need to eliminate the duplication, otherwise the tool
will complain.

Previously, we simply choose the platform variant of the java library
among the multiple variants. However, if the java library is marked not
available for the platform (i.e. "//apex_available:platform" is not in
the apex_available property), the platform variant does not exist and
thus it is not fed into the hiddenapi tool, which causes missing
references in the tool.

To solve the problem, the platform variant is selected only for the java
libs that are available for the platform. For those libs that are not
available for the platform, we choose one variant of it using a
heuristic; skip testing APEXes and choose com.android.art.release among
the com.android.art.* variants.

Bug: 128708192
Test: m

Change-Id: I33bf297eb3029696ae3504a011486210708fb2c2
2019-12-16 15:19:13 +09:00
Jiyong Park
505221f6e0 ignore apex_available for host variants
The main purpose of apex_available is to prevent a module from being
installed to the system partition (or being statically linked to
something that is installed to the system partition). It's not the goal
to prevent host tools from using the modules.

However, since both host and device variants are not mutated for
platform when the module is not available to platform, the host tools
that depends on those modules couldn't be built. To solve the problem,
the platform variation is NOT skipped for the host variants.

Bug: 128708192
Test: m

Change-Id: I1d662cd6d165581f344138d872329a15bfc43d17
2019-12-16 15:19:13 +09:00
Jiyong Park
2bd71e4da1 fix: LOCAL_PATH for modules included in APEX is wrong
am: 1833ceff09

Change-Id: I9ff5f1726806e35ae841c4026646ada7e0570c2b
2019-12-15 20:20:07 -08:00
Jiyong Park
1833ceff09 fix: LOCAL_PATH for modules included in APEX is wrong
This change fixes a bug that LOCAL_PATH for modules included in an APEX
is set to the path of the APEX bundle, not to the path of the embedded
module. For example, LOCAL_PATH of libconscrypt included in
com.android.adbd was set to /system/core/adb instead of
/external/boringssl. This caused a problem that NOTICE file in
/external/boringssl is not tagged to libconscrypt, but the NOTICE file
for adbd is.

Fixing the problem by recording the module directories of the included
modules and emitting it in LOCAL_PATH.

Bug: 145347092
Test: Settings -> About Phone -> Legal Information -> Third-party
license. The license for /apex/com.android.adbd/lib64/libconscrypt.so is
OpenSSL.

Change-Id: I76f1830d5a10af63fa74dcc2a42730ffabb8c4ed
2019-12-16 10:53:44 +09:00
Elliott Hughes
86aad618f3 Merge "Switch the mac back to toybox xargs."
am: 0f41daf682

Change-Id: I1d39656361560d650b946207470a5532b402787f
2019-12-15 08:38:03 -08:00
Treehugger Robot
0f41daf682 Merge "Switch the mac back to toybox xargs." 2019-12-15 16:35:05 +00:00
Dan Willemsen
d476fc7dbe Merge "Remove obsolete copy headers"
am: 950bf327b7

Change-Id: Ibcccd4de27bdb8e2425632872a40c02e21a55e00
2019-12-14 16:08:10 -08:00
Dan Willemsen
950bf327b7 Merge "Remove obsolete copy headers" 2019-12-15 00:04:15 +00:00
Paul Duffin
d32b891ff3 Merge "Decouple addition of new sdk member types from sdk code"
am: 0558ecce51

Change-Id: Ic1da6424bffada70a0e69bb6605495f45a2f71b2
2019-12-14 07:00:58 -08:00
Treehugger Robot
0558ecce51 Merge "Decouple addition of new sdk member types from sdk code" 2019-12-14 14:57:42 +00:00
Colin Cross
7113d20774 Use empty string for core image variant
Use the empty string for the core image variant so that modules
added to imageMutator do not change their build directory.

Bug: 142286466
Test: m checkbuild
Change-Id: Ida4534d9a4d6176236aaa480fed359ce27acfaa1
Merged-In: Ida4534d9a4d6176236aaa480fed359ce27acfaa1
(cherry picked from commit 72d685ee7f45e5393be44ae4159edf083ac918de)
2019-12-13 20:44:36 -08:00
Dan Willemsen
7842826f49 Remove obsolete copy headers
When LOCAL_COPY_HEADER lines are removed, we just removed the copy rule,
we didn't actually remove the old header, so C/C++ files could still
include the old headers during incremental builds.

While we do consider LOCAL_COPY_HEADERS as near-obsolet, and it'll
disappear as we move everything over to Soong, this could produce some
unfortunate incremental build results while people are attempting to
remove them.

Fix this by ensuring that only the files currently listed in
LOCAL_COPY_HEADERS exist in TARGET_OUT_HEADERS after we run Kati.

Test: Remove a LOCAL_COPY_HEADERS entry; see the header removed
Change-Id: I817305703a6996d50490d552623d7df019b608c9
2019-12-13 19:38:23 -08:00