The clang-r370808 upgrade contains a change to LLD allow PT_LOAD
segments to reside at non-multiples of the page size in the resulting
object file. https://reviews.llvm.org/rL369344
While this helps reduce the alignment waste and resulting image size, it
has interesting implications for execute only memory (XOM): The runtime
loader will now load code or data from other segments into pages with
different protections than intended.
This would partially defeat execute only (XOM) text sections as the
segment could now overlap with previous and following sections. This
might allow for code or data from the preceding and following sections
(like .eh_frame, and .data.rel.ro) to be executable, and either ends of
.text to be readable.
When the runtime loader (linker[64]) `mmap`s segments from *.so files,
the file offset parameter (see `man 2 mmap`) MUST be a multiple of the
page size. Since the updated LLD can now pack segments in a file (which
helps minimize resulting object file size) (previously, the segment
offsets were page aligned), this has interesting implications.
To appreciate the current bug, consider the following output from
`readelf` before this patch is applied, but after the toolchain upgrade:
```
$ readelf -lSW $OUT/symbols/apex/com.android.runtime/lib64/bionic/libc.so
...
[Nr] Name Type Address Off Size ES Flg Lk Inf Al
...
[13] .eh_frame PROGBITS 000000000002e7c0 02e7c0 013374 00 A 0 0 8
[14] .text PROGBITS 0000000000042b40 041b40 09ecb4 00 AX 0 0 64
[15] .plt PROGBITS 00000000000e1800 0e0800 001f30 00 AX 0 0 16
[16] .data.rel.ro PROGBITS 00000000000e4740 0e2740 005208 00 WA 0 0 32
...
Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align
PHDR 0x000040 0x0000000000000040 0x0000000000000040 0x000230 0x000230 R 0x8
LOAD 0x000000 0x0000000000000000 0x0000000000000000 0x041b34 0x041b34 R 0x1000
LOAD 0x041b40 0x0000000000042b40 0x0000000000042b40 0x0a0bf0 0x0a0bf0 E 0x1000
LOAD 0x0e2740 0x00000000000e4740 0x00000000000e4740 0x006720 0x006720 RW 0x1000
...
01 .note.android.ident .note.gnu.build-id .dynsym .gnu.version .gnu.version_d .gnu.version_r .gnu.hash .dynstr .rela.dyn .rela.plt .rodata .eh_frame_hdr .eh_frame
02 .text .plt
03 .data.rel.ro .fini_array .init_array .dynamic .got .got.plt
...
The above output tells us:
1. .text will wind up in the third (02) segment.
2. The third segment will be (LOAD)'ed as (E)xecutable.
3. Because the file (Offset) of the first segment (0x41b40) is NOT a
multiple of the page size, it cannot be passed as the `offset` to
`mmap`. As such it will be rounded down to the first multiple of the
page size, 0x41000.
4. The preceding section (.eh_frame) will be loaded in the preceding
segment (01). It occupies file (Off)set range [(0x2e7c0):0x41b34].
0x41b34 is not explicit in the output, instead you must use the
formula:
Off + Size == End
ie.
0x2e7c0 + 0x13374 == 0x41b34
(This happens to match (FileSiz) of the second segment, which makes
sense as .eh_frame is the final section in the second segment.)
5. mmap'ing file offset 0x41000 when loading the second segment will
include 0x4c0 bytes (0x42000 - 0x41b40) from .text, now mapped as
readable (oops). Suddenly code from .text is now readable (and thus
scannable for gadgets for ROP chains).
6. mmap'ing file offset 0x41000 when loading the third segment will
include 0xb34 bytes (0x41b34 - 0x41000) from .eh_frame, now mapped as
executable (oops). Suddenly data from .eh_frame is now executable
(and thus a potential gadget for ROP chains).
7. mmap'ing file offset 0xe2000 when loading the third segment will
include 0x8CO bytes (0xe3000 - 0xe2740) from .data.rel.ro, now mapped
as executable (oops). Suddenly data from .data.rel.ro is now
executable (and thus a potential gadget for ROP chains).
8. mmap'ing file offset 0xe2000 when loading the fourth segment will
include 0x730 bytes (0xe0800 + 0x1f30 - 0xe2000) from .plt, now
mapped as readable (oops). Suddenly data from .plt is now readable
(and thus scannable for gadgets for ROP chains).
All these oops' could be avoided if the linker placed .text+.plt at page
size aligned file offsets, which is what `-Wl,-z,separate-code` code
does. After this patch, we have:
```
$ readelf -lSW $OUT/symbols/apex/com.android.runtime/lib64/bionic/libc.so
...
Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align
PHDR 0x000040 0x0000000000000040 0x0000000000000040 0x000230 0x000230 R 0x8
LOAD 0x000000 0x0000000000000000 0x0000000000000000 0x041b34 0x041b34 R 0x1000
LOAD 0x042000 0x0000000000042000 0x0000000000042000 0x0a0be0 0x0a0be0 E 0x1000
LOAD 0x0e3000 0x00000000000e3000 0x00000000000e3000 0x006720 0x006720 RW 0x1000
```
In the future, we could go back to tightly packing segments in the
binary if the runtime loader was improved to detect the previously
stated problem, and `memset` over the problematic ranges of the freshly
`mmap`ed pages (implying additional startup cost for reduced binary
size). This might save ~6 KB from each native binary, which adds up to
~17 MB for an AOSP image.
Also, prefer
-Wl,--execute-only
rather than
-Wl,-execute-only
Bug: 139945549
Bug: 146144180
Test: readelf -lSW $OUT/symbols/apex/com.android.runtime/lib64/bionic/libc.so
Change-Id: I64527e034ca3c71565ea52ed06f81f75d5216627
Reported-by: Ryan Prichard <rprichard@google.com>
Suggested-by: Fangrui Song <maskray@google.com>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
Use the empty string for the core image variant so that modules
added to imageMutator do not change their build directory.
Bug: 142286466
Test: m checkbuild
Change-Id: Ida4534d9a4d6176236aaa480fed359ce27acfaa1
Merged-In: Ida4534d9a4d6176236aaa480fed359ce27acfaa1
(cherry picked from commit 72d685ee7f45e5393be44ae4159edf083ac918de)
This reverts commit 862eb4648a.
Re-upgrades the compiler to clang-r370808, after first dealing with
regressions in ndk_translate and execute only pages.
Bug: 139945549
Bug: 145807809
Bug: 145827049
Bug: 145825270
Test: atest \
CtsSelinuxTargetSdk27TestCases:android.security.SELinuxTargetSdkTest#testNoExecuteOnly
Test: m ndk_translation_host_unit_tests && \
./out/host/linux-x86/nativetest/ndk_translation_host_unit_tests/ndk_translation_host_unit_tests
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
Change-Id: I714b582faffa8c92384818a7b12338d621968548
Previously, adding a new SdkMemberType would require adding a new
sdkMemberListProperty instance to the sdkMemberListProperties as well
as adding a new property into the sdkProperties struct. They are
potential sources of conflict and couple the sdk code with all the
packages that add members to it. This change switched to a
registration model that allows each package to register its sdk
member types decoupling them from the sdk code.
Adds an SdkPropertyName() method to SdkMemberType that specifies the
name of the property to use in the sdk/sdk_snapshot. Also provides
an SdkMemberTypeBase struct to be used by providers of SdkMemberType
implementations.
SdkMemberType instances are registered using the
RegisterSdkMemberType() func which sorts the registered instances
by their SdkPropertyName() to ensure the behavior is consistent and
not affected by order of registration.
When creating a new sdk module a dynamicSdkMemberTypes instance is
created that contains the following:
* A properties struct is created dynamically that contains a field for
each registered SdkMemberType, corresponding to that type's
SdkPropertyName().
* A list of sdkMemberListProperty instances is also created, one for
each registered SdkMemberType.
The dynamicSdkMemberTypes instance is cached using a key that uniquely
identifies the set of registered types just in case new types are
registered after one has been created, e.g. by tests.
Bug: 142918168
Test: m checkbuild
Change-Id: I4bf2bf56a2a49025aa41454048bc1e8ccc6baca2
The exported include dirs includes both source and generated
directories (e.g. containing aidl generated headers). The latter are
always arch specific so if they are present they make all the include
directories arch specific.
This change separates the source and generated include dirs so that
the source include dirs (which are probably not arch specific) can be
optimized separately from the arch specific generated include dirs.
The FilterPathList() func was refactored to extract the more general
FilterPathListPredicate() func.
A number of tests needed to be updated to reflect the more optimal
snapshot creation.
Bug: 142918168
Test: m checkbuild
Change-Id: Id1a23d35a45b250ae2168834f9c2a65c86a5fd77
Previously, code that attempted to optimize the generated .bp rules
treated the properties structure as a single entity. So, a single arch
specific value would cause all properties to be treated as arch
specific. Also, that code was specific to one structure type.
This generalizes the optimization to work with any properties structure
which will be helpful for other multi-variant module types. It also
treats each property separately.
The hasArchSpecificFlags field has been removed from nativeLibInfo and
a commonProperties field has been added instead into which the common
values will be found. File path creation that conditionally prefixed a
path with archType has been replaced with general code that relies on
archType being "" for common properties and filepath.Join(..) ignoring
empty string components.
The common and arch variant properties are always processed. The first
within the context of the .bp module's property set and the latter
within an arch specific property set. There are always some properties
that are arch specific, e.g. outputFile, so there is no need to worry
about an empty arch property set being created.
The archSpecificNativeLibInfo type was renamed nativeLibInfoProperties
as it may not be arch specific.
The printExportedDirCopyCommandsForNativeLibs variable was renamed to
addExportedDirCopyCommandsForNativeLibs as it no longer does any
printing.
Bug: 142918168
Test: m checkbuild
Change-Id: Iad45913299c37fd76fe03ed0ca68bdc68ed76431
Parameterized the cc.librarySdkMemberType to allow it to support
both static and shared libraries. Created two instances, one for shared
and one for static libraries. A follow up change will add support for
libraries that can be both.
Added *librarySdkMemberType to nativeMemberInfo as information from
there is needed when generating the snapshot.
Made organizeVariants() func a method of *librarySdkMemberType so that
it can initialize the new field. Moved it to be with all the other
methods of that type.
Added host and device tests for the new module type.
Bug: 142918168
Test: m nothing
Change-Id: I00b1e8424b9d81f7d15edc4883971d10668ec2cc
AndroidMkEntries now returns multiple AndroidMkEntires so that a module
can emit multiple Make modules if needed.
Bug: 128708192
Test: m
Change-Id: I56b6f76d22943b80329951c5acb80a1b932441ad
Build fuzzer-sanitized libraries with an $ORIGIN DT_RUNPATH. Android's linker
uses DT_RUNPATH, not DT_RPATH. When we deploy cc_fuzz targets and their
libraries to /data/fuzz/<arch>/lib, any transient shared library gets the
DT_RUNPATH from the shared library above it, and not the executable, meaning
that the lookup falls back to the system. Adding the $ORIGIN to the DT_RUNPATH
here means that transient shared libraries can be found colocated with their
parents.
This may have some interesting consequences if:
1. Your fuzz target depends on a shared library which has `sanitize.fuzzer:
false` (as the DT_RUNPATH won't have `$ORIGIN`, and so you may get missing
libraries).
2. A `SANITIZE_TARGET=fuzzer` platform has a shared object in two
different directories (like system vs. ndk) and is depending on the linker
implementation details to resolve this in some manner.
I don't believe either of these instances should reasonably happen in
practise.
Fixes: 145973404
Fixes: 145988908
Test: m example_fuzzer
Change-Id: I94cbf628fc1ce15c43283d72bdabd9817de1fef8
This is needed for a follow up change that makes sure that the
prebuilt modules have the same visibility as the source modules.
Bug: 143678475
Test: m conscrypt-module-sdk
Change-Id: I9461c8c094ab19ee9ececb5e5fd50565789f2fa2
Because aidl_interface depends on some ndk modules, their factories are
exposed so that aidl_test.go can use them.
Bug: n/a
Test: m
Change-Id: I98d282cc77310d5896a7abaf3936456a14d56ccf
For VNDK snapshot and SDK snapshot, deps files have been used to capture
generated headers. But exported deps might contain intermediate phony
files instead of actual header files, which are for optimization of
ninja. To correctly capture all headers, exported generated header files
are gathered separately.
Bug: 65377115
Test: m nothing
Change-Id: Ia03fa69186490a818578190e3c0bfb0261d1fd6e
Move the ImageMutator to be registered just after the archMutator
in preparation for moving it between osMutator and archMutator.
Requries updating variants in a few tests that now run the
ImageMutator.
Bug: 142286466
Test: no change to build.ninja
Test: all soong tests
Change-Id: Ia9d2a7bc0e225bedec3c9a83ea04f471a931bf47
Extracts the type specific functionality into the SdkMemberType
interface which has to be implemented by each module type that can
be added as a member of the sdk. It provides functionality to add
the required dependencies for the module type, check to see if a
resolved module is the correct instance and build the snapshot.
The latter was previously part of SdkAware but was moved because
it has to be able to process multiple SdkAware variants so delegating
it to a single instance did not make sense.
The custom code for handling each member type specific property,
e.g. java_libs, has been replaced with common code that processes
a list of sdkMemberListProperty struct which associates the
property (name and getter) with the SdkMemberType and a special
DependencyTag which is passed to the SdkMemberType when it has to add
dependencies.
The DependencyTag contains a reference to the appropriate
sdkMemberListProperty which allows the resolved dependencies to be
grouped by type.
Previously, the dependency collection methods would ignore a module if
it was an unsupported type because they did not have a way of
determining which property it was initially listed in. That meant it
was possible to add say a droidstubs module to the java_libs property
(and because they had the same variants) it would work as if it was
added to the stubs_sources property. Or alternatively, a module of an
unsupported type could be added to any property and it would just be
ignored.
However, the DependencyTag provides information about which property
a resolved module was referenced in and so it can detect when the
resolved module is of the wrong type and report an error. That check
identified a bug in one of the tests where the sdk referenced a
java_import module (which is not allowed in an sdk) instead of a
java_library module (which is allowed). That test was fixed as part
of this.
A list of sdkMemberListProperty structs defines the member properties
supported by the sdk and are processed in order to ensure consistent
behaviour.
The resolved dependencies are grouped by type and each group is then
processed in defined order. Within each type dependencies are grouped
by name and encapsulated behind an SdkMember interface which includes
the name and the list of variants.
The Droidstubs and java.Library types can only support one variant and
will fail if given more.
The processing for the native_shared_libs property has been moved into
the cc/library.go file so the sdk package code should now have no type
specific information in it apart from what is if the list of
sdkMemberListProperty structs.
Bug: 143678475
Test: m conscrypt-module-sdk
Change-Id: I10203594d33dbf53441f655aff124f9ab3538d87
"data" field specifies data dependencies that will be installed in
fuzzer's output directory.
"data" behaves similar to "corpus", except "data" preserves directory
structure, e.g.
data: ["foo/bar.txt"]
is installed into
$OUT/data/fuzz/<arch>/<target>/data/foo/bar.txt
Test: build a fuzzer with data depenency, check data is installed
correctly
Change-Id: Ia1255026278435181b6d93f91f8f9ad39c96d07f
cc_fuzz ignored soong namespaces, and built all cc_fuzz modules into the
architecture package. This caused naming conflicts where two modules had
the same name but were in different namespaces.
Now, cc_fuzz will only build modules that are in the same namespace as
the product definition - resolving the duplicate definitions error.
Bug: 145240569
Test: m haiku
Change-Id: I0a0acd139426301586ac5e3aba8ce5527cb7a1b5
Exclude vendor/ and hardware/ from the manually written binder interface
whitelist.
Bug: 136279235
Test: interfaces in excluded directories are not checked against whitelist.
Change-Id: I0d640e23489b37d0c0787d5fca6bcdab10034109
The use of pragma pack(1) affects the packed-ness of every struct
definition that follows it in the translation unit, even in following
header files. This can lead to multiple translation units disagreeing
on whether a struct is packed.
Enable the warnings (as errors) that help spot this problem.
Bug: http://b/73971777
Test: treehugger
Change-Id: Id3096f35dff5dc8723d30d1305f1ed7ae8190a32
This commit enables ABI checks for shared libs exported by APEX and
explicitly enabled ABI checks.
Bug: 145608479
Test: m com.android.resolv
Change-Id: I3b58178b0258df35fcc848e84642152516f6774f
Previously, it was defaultable only for cc_library. Now, it is
defaultable for all module types.
Bug: 128708192
Test: m
Change-Id: If248f593da6f4b724bf889ccd7f2e077d48069b5
-Wc99-designators warns for the use of nested and array initializers in
C++20 code. Many internal projects have pushed back against this
warning.
Disables:
- -Wimplicit-int-float-conversion
- -Wpointer-compare
- -Wxor-used-as-pow
- -Wfinal-dtor-non-final-class
- -Wreorder-init-list
For projects under external/, vendor/, etc, but only if they use
Android.bp (not Android.mk) and -Wreorder-init-list may be re-enabled by
-Wall if set locally.
-Wno-incomplete-setjmp-declaration is needed for NDK stubs that get
generated (functions without full definitions of their parameters).
Sets
- -Wno-error=implicit-int-float-conversion
- -Wno-error=reorder-init-list
for all projects in order to unblock landing the compiler upgrade due to
internal projects not being able to disable or fix these before the
upgrade.
Bug: 139945549
Test: build (aosp_crosshatch, crosshatch, aosp_x86_64-eng,
walleye), boot internal devices, bionic + RS atests, kernel builds.
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
Change-Id: I18e3c0eb657fd59824526f36c6dd57bdcf0526ba
TARGET_VNDK_USE_CORE_VARIANT devices explicitly track libs which are
differentiated between system/vendor. b/145157349 is filed to try to
catch these errors during normal builds (or in presubmit).
aidl libs need to be differentiated because they explicitly set
different system/vendor stabilities.
Fixes: 145154788
Test: fixes build
Change-Id: I682ceaa2dd23f0f80ba060ae262d766b1367187e
Avoid having to pass ModuleFactoryAdaptor to every call to
RegisterModuleType in a test by wrapping RegisterModuleType.
Test: all soong tests
Change-Id: If8847d16487de0479cc3020b728256922b3cadba
This is a workaround to improve hwasan reports involving stack
variables.
Bug: 112437883
Test: SANITIZE_TARGET=hwaddress, build and boot
Change-Id: I21ea2fdd2d1f04e2bb9b9847785c00a951030c74
