Commit Graph

17171 Commits

Author SHA1 Message Date
Anton Hansson
d57bd3c84f Revert "Move odexes of non-system apps into system_other"
This reverts commit 12b8d427c7.

Bug: 142537657
Test: No
Change-Id: Iefd882ec11c060877671f5bf0f2429ba4d811f8b
Merged-In: Iefd882ec11c060877671f5bf0f2429ba4d811f8b
2019-12-06 13:42:25 +00:00
Ulya Trafimovich
163664a548 Pass correct --image argument to oatdump.
This was broken after CL:
  Ida40dfae8c83bf7c2e737d5c7ea418e1197ad826
..which split boot image into primary and extension.
The argument to oatdump was not updated properly.

The old form before the split was:

  --image=out/soong/generic_x86/dex_bootjars/system/framework/boot.art

The new form after the split is:

  --image=out/soong/generic_x86/dex_artjars/apex/com.android.art/javalib/boot.art:out/soong/generic_x86/dex_bootjars/system/framework/boot-framework.art

Test: lunch aosp_x86-userdebug && m dump-oat

Test: Manually inspect build command to make sure correct image
  location is used:

  $ fgrep -e 'oatdump' $ANDROID_BUILD_TOP/out/soong/build.ninja | egrep -e '--image[^ ]*' -o
  --image=out/soong/generic_x86/dex_artjars/apex/com.android.art/javalib/boot.art:out/soong/generic_x86/dex_bootjars/system/framework/boot-framework.art

Change-Id: I54a5146c404aefb6eaeadbe3e9363db935ecad86
2019-12-06 13:42:21 +00:00
Paul Duffin
a6e737b078 Organize sdk member properties
Grouping in alphabetical order by package and then by name within the
package should minimize conflicts when making changes.

Bug: 143678475
Test: m conscrypt-module-sdk
Change-Id: Ia7dbcd41ce8b8dd8675a90b1b6868fcaeaf72ee4
2019-12-06 12:16:59 +00:00
Paul Duffin
a80fdec00d Extract the cc and java sdk related tests out into their own file
Makes it easier for multiple people to work on different areas without
conflicts.

Bug: 143678475
Test: m conscrypt-module-sdk
Change-Id: I93140450c0b049946a9d0432225bccff82f5a873
2019-12-06 12:16:59 +00:00
Paul Duffin
c3c5d5e351 Improve testing of sdk snapshot generation
Adds TestHelper to provide general test helper functionality for use by
any test.

Adds testSdkResult, composed with TestHelper that encapsulates the
result of processing and sdk {..} definition and provides specialized
support for testing the build rules.

Dedups the analysis of the sdk build rules, and improves it to extract
more information, and in different forms. That is represented by the
snapshotBuildInfo struct.

Adds a CheckSnapshot() method which checks the snapshot for a specified
sdk version. It takes a list of functions that can each perform a check
on a specific facet of the supplied snapshotBuildInfo.

Methods are provided for tests to use to check the following facets:
* Generated Android.bp contents.
* Copy rules
* Merge zip inputs

This approach makes it possible for each test to customize what is
being checked without either duplicating functionality, causing a
proliferation of specialized forms of the CheckSnapshot method for
different types of tests or adding arguments for every possible check
that any test would need which would lead to lots of churn to existing
tests when new arguments are added.

The main testing improvement is for CheckSnapshot() to actually try and
load the Android.bp that is generated. In order to do that it was
necessary to create a mock filesystem populated with information from
the build rules, i.e. the destination files from every Cp command as
well as the destination directory from every repackage zip command.

That helps detect a number of sources of errors:
* Failing to copy a file/directory that is mentioned in the generated
  Android.bp file.
* Invalid properties.
* Invalid format of the .bp file.
* Integrity issues within the .bp file.

Bug: 143678475
Test: m conscrypt-module-sdk
Change-Id: I4d3fe18f86698186d18e7e8b32d2e319183f7f0c
2019-12-06 12:16:59 +00:00
Paul Duffin
82d90438be Separate sdk testing infrastructure from sdk tests
Bug: 143678475
Test: m conscrypt-module-sdk
Change-Id: Ib00870ddefc4c1dd9e42ca594e6ebe8e24c42e05
2019-12-06 12:16:59 +00:00
Paul Duffin
1387957727 Parameterize the sdk member processing
Extracts the type specific functionality into the SdkMemberType
interface which has to be implemented by each module type that can
be added as a member of the sdk. It provides functionality to add
the required dependencies for the module type, check to see if a
resolved module is the correct instance and build the snapshot.

The latter was previously part of SdkAware but was moved because
it has to be able to process multiple SdkAware variants so delegating
it to a single instance did not make sense.

The custom code for handling each member type specific property,
e.g. java_libs, has been replaced with common code that processes
a list of sdkMemberListProperty struct which associates the
property (name and getter) with the SdkMemberType and a special
DependencyTag which is passed to the SdkMemberType when it has to add
dependencies.

The DependencyTag contains a reference to the appropriate
sdkMemberListProperty which allows the resolved dependencies to be
grouped by type.

Previously, the dependency collection methods would ignore a module if
it was an unsupported type because they did not have a way of
determining which property it was initially listed in. That meant it
was possible to add say a droidstubs module to the java_libs property
(and because they had the same variants) it would work as if it was
added to the stubs_sources property. Or alternatively, a module of an
unsupported type could be added to any property and it would just be
ignored.

However, the DependencyTag provides information about which property
a resolved module was referenced in and so it can detect when the
resolved module is of the wrong type and report an error. That check
identified a bug in one of the tests where the sdk referenced a
java_import module (which is not allowed in an sdk) instead of a
java_library module (which is allowed). That test was fixed as part
of this.

A list of sdkMemberListProperty structs defines the member properties
supported by the sdk and are processed in order to ensure consistent
behaviour.

The resolved dependencies are grouped by type and each group is then
processed in defined order. Within each type dependencies are grouped
by name and encapsulated behind an SdkMember interface which includes
the name and the list of variants.

The Droidstubs and java.Library types can only support one variant and
will fail if given more.

The processing for the native_shared_libs property has been moved into
the cc/library.go file so the sdk package code should now have no type
specific information in it apart from what is if the list of
sdkMemberListProperty structs.

Bug: 143678475
Test: m conscrypt-module-sdk
Change-Id: I10203594d33dbf53441f655aff124f9ab3538d87
2019-12-06 12:16:59 +00:00
Paul Duffin
375058f67d Use static build rules in snapshot generation
It is easier to extract information out of static build rules than it
is out of custom build rules built using the builder as the former
provides access to the in/out and args separate from the rule whereas
the latter only provides access to in/out.

Also, cleans up some warnings that appear in Intellij.

There is a lot of duplication in the testing code. That will be
resolved in a follow up change.

Bug: 143678475
Test: m conscrypt-module-sdk
Change-Id: I973bc0c90b0affd84487f1b222dd3e6c22c07ec0
2019-12-06 12:16:53 +00:00
Jiyong Park
f8bde14a30 Merge "apex_available is defaultable"
am: cea93afe76

Change-Id: Iff09b7c9fc51747883a70b88117cdc8e623ebe52
2019-12-05 20:31:04 -08:00
Treehugger Robot
cea93afe76 Merge "apex_available is defaultable" 2019-12-06 04:23:37 +00:00
Dan Willemsen
9609ad9c21 Add sharding support to multiproduct_kati
This is so that we can split the build_test build on large branches into
multiple builds, each testing a subset of the products.

Test: m blueprint_tools  (runs the go unit tests)
Test: build/soong/build_test.bash --only-config
      vs
Test: build/soong/build_test.bash --only-config --shard-count=2
Test: build/soong/build_test.bash --only-config --shard-count=2 --shard=2

Change-Id: I40ccc1aa477bc0ffa74ff564d155068509be18f0
2019-12-05 19:54:40 -08:00
Ivan Lozano
cc9c86c35e Merge "Exclude dirs from binder interface whitelisting"
am: 1fdb0b55a5

Change-Id: I4cb503b3aa804f8ffd9e0ca0deeafc96abb1504d
2019-12-05 18:29:28 -08:00
Ivan Lozano
1fdb0b55a5 Merge "Exclude dirs from binder interface whitelisting" 2019-12-06 02:25:10 +00:00
Jiyong Park
80b1dbe558 Merge "Fix: vendor and odm are restricted to use public platform properties"
am: 337cd8e493

Change-Id: I04004cae3d6ee11eb4cd359a395a50b6493a6575
2019-12-05 18:04:54 -08:00
Treehugger Robot
337cd8e493 Merge "Fix: vendor and odm are restricted to use public platform properties" 2019-12-06 02:00:11 +00:00
Colin Cross
617d508800 Merge "Add CUJ tests"
am: 6cfd3f0319

Change-Id: I887243b42cc946189e3b000643522d8db1280552
2019-12-05 17:22:07 -08:00
Treehugger Robot
6cfd3f0319 Merge "Add CUJ tests" 2019-12-06 01:17:04 +00:00
Jooyung Han
3ab2c3e68c Install flattened/unflattend apexes for GSI
GSI targets are supposed to have both 'flattened' and 'unflattened'
APEXes. By adding 'flattened' APEX as REQUIRED moduled for 'unflattened'
APEX, both will be installed togetther.

This is done by a new variable PRODUCT_INSTALL_EXTRA_FLATTENED_APEXES,
which is set only for GSI targets.

Bug: 137802149
Test: lunch aosp_arm64-userdebug && m
  resulting apex images under /system/apex
  and flattened apexes under /system/system_ext/apex

Change-Id: I336e2674e427b358542e0045b2a49dfa3d84095b
2019-12-06 09:52:19 +09:00
Calin Juravle
6db8f6c37f Merge "Export art dexpreopt image zip file"
am: af7c2fcb71

Change-Id: I17b85446b1da661a30e7ba8acdfc390b283e2ce7
2019-12-05 15:03:44 -08:00
Tri Vo
ad172d88b6 cc_fuzz: add "data" field
"data" field specifies data dependencies that will be installed in
fuzzer's output directory.

"data" behaves similar to "corpus", except "data" preserves directory
structure, e.g.

    data: ["foo/bar.txt"]

is installed into

    $OUT/data/fuzz/<arch>/<target>/data/foo/bar.txt

Test: build a fuzzer with data depenency, check data is installed
correctly
Change-Id: Ia1255026278435181b6d93f91f8f9ad39c96d07f
2019-12-05 14:45:51 -08:00
Calin Juravle
af7c2fcb71 Merge "Export art dexpreopt image zip file" 2019-12-05 22:11:28 +00:00
Mitch Phillips
8e83eb03c7 Merge "Fix cc_fuzz ignoring soong namespaces."
am: 5eb62bc2fe

Change-Id: I36c2f67b2e28bfb2f299d7417651188b9cefffff
2019-12-05 13:46:19 -08:00
Treehugger Robot
5eb62bc2fe Merge "Fix cc_fuzz ignoring soong namespaces." 2019-12-05 21:41:41 +00:00
Nick Desaulniers
e350a6bf35 Merge "soong: upgrade Android platform to clang-r370808"
am: ac0336d81f

Change-Id: I2c537d2672db4a3aa850da0760be0a571b2cd727
2019-12-05 12:21:49 -08:00
Nick Desaulniers
ac0336d81f Merge "soong: upgrade Android platform to clang-r370808" 2019-12-05 20:12:08 +00:00
Jaewoong Jung
e5cd4e10a0 Optionally overwrite package value in test config.
If package_name is set for an android_test module, package names in its
AndroidTest.xml config need to be updated too.

Test: test_config_fixer_test.py
Fixes: 145011263
Change-Id: I52f17ef1d1902364b9bcfec4f6e246bd655dbf09
2019-12-05 11:17:09 -08:00
Colin Cross
d0be2109cf Add CUJ tests
Add a cuj_tests binary that runs through a sequence of critical
user journey builds and collects metrics.

Bug: 145688720
Test: cuj_tests
Change-Id: I6bb1a3fc4a458fd8cea7b7c924f565d0cf3e6df3
2019-12-05 11:11:37 -08:00
Mitch Phillips
6a9bf214f9 Fix cc_fuzz ignoring soong namespaces.
cc_fuzz ignored soong namespaces, and built all cc_fuzz modules into the
architecture package. This caused naming conflicts where two modules had
the same name but were in different namespaces.

Now, cc_fuzz will only build modules that are in the same namespace as
the product definition - resolving the duplicate definitions error.

Bug: 145240569
Test: m haiku
Change-Id: I0a0acd139426301586ac5e3aba8ce5527cb7a1b5
2019-12-05 09:38:56 -08:00
Dan Willemsen
0bf7108a59 Merge "Test par file argument handling"
am: bed70f837b

Change-Id: Id46092dfbfb68ade7848feca7198c0d1e2f14244
2019-12-05 08:54:04 -08:00
Treehugger Robot
bed70f837b Merge "Test par file argument handling" 2019-12-05 16:23:56 +00:00
Colin Cross
42808fb2fc Merge changes Id261b5a3,Ice6f34d0
am: dc0e8fd131

Change-Id: I5bb5bb93d44182088b7eedd8a96e70de8c23bd9b
2019-12-05 01:21:22 -08:00
Treehugger Robot
dc0e8fd131 Merge changes Id261b5a3,Ice6f34d0
* changes:
  Fix errorProtoLog error messages
  Write BUILD_DATETIME_FILE after SetupOutDir
2019-12-05 09:10:13 +00:00
Søren Gjesse
e40d85575d Merge "Remove passing -forceprocessing flag to R8"
am: 5f9e5244c5

Change-Id: Iaa03b8a0cf786195c18b02338ad17329e61674eb
2019-12-05 01:07:43 -08:00
Treehugger Robot
5f9e5244c5 Merge "Remove passing -forceprocessing flag to R8" 2019-12-05 08:56:38 +00:00
Jiyong Park
b326ac2b65 Fix: vendor and odm are restricted to use public platform properties
There was a mistake in the condition.

Bug: N/A
Test: N/A

Change-Id: I07f79fdcc3c946a4b9e957e5a42b6694b5eeb251
2019-12-05 13:35:35 +09:00
Jiyong Park
317645e84c Overridden APEX doesn't contribute to the file contexts
This change fixes the problem that when an apex module is overridden by
another override_apex, the <apex_name>-file_contexts are duplicated when
creating the system-level file-contexts.

Fixing this by not emitting the file_context info for the overridden
apex.

In doing so, OverridableModule interface was extended to have
GetOverriddenBy() method which can be used to test whether a module is
an overridden one or not.

Bug: 144338929
Test: m (apex_test amended)
Test: add "override_apex {name:"com.googlge.android.tzdata",
Change-Id: I5e9401c32899bb9987c90cba4185f571dc1a87f0
base:"com.android.tzdata"}" and the build is successful
2019-12-05 13:25:39 +09:00
Colin Cross
31bf00dac3 Export art dexpreopt image zip file
Test: m  out/target/product/generic_arm64/boot.zip
Bug: 144491400
Change-Id: I84342703ac614e072b0cde8c6f5597dc8cffbc65
2019-12-04 16:50:52 -08:00
Jiyong Park
5f5e277514 Merge "Remove implicit outputs for the flattened APEX Make module"
am: 7b34ebf447

Change-Id: Ib30fd58cd1dffc1a0e5d6441e9a2f4e9dfea7873
2019-12-04 16:24:55 -08:00
Treehugger Robot
7b34ebf447 Merge "Remove implicit outputs for the flattened APEX Make module" 2019-12-05 00:21:29 +00:00
Ivan Lozano
d094d4006f Exclude dirs from binder interface whitelisting
Exclude vendor/ and hardware/ from the manually written binder interface
whitelist.

Bug: 136279235
Test: interfaces in excluded directories are not checked against whitelist.
Change-Id: I0d640e23489b37d0c0787d5fca6bcdab10034109
2019-12-04 09:10:21 -08:00
Elliott Hughes
2cdbdf1093 Enable -Wpragma-pack warnings as errors.
The use of pragma pack(1) affects the packed-ness of every struct
definition that follows it in the translation unit, even in following
header files. This can lead to multiple translation units disagreeing
on whether a struct is packed.

Enable the warnings (as errors) that help spot this problem.

Bug: http://b/73971777
Test: treehugger
Change-Id: Id3096f35dff5dc8723d30d1305f1ed7ae8190a32
2019-12-04 08:48:45 -08:00
Logan Chien
e274fc9892 Run ABI checks for shared libs exported by APEX
This commit enables ABI checks for shared libs exported by APEX and
explicitly enabled ABI checks.

Bug: 145608479
Test: m com.android.resolv
Change-Id: I3b58178b0258df35fcc848e84642152516f6774f
2019-12-04 15:26:37 +00:00
Paul Duffin
2944053af4 Disable TestHostSnapshot test for non-linux build
am: 46fc58ee8d

Change-Id: Iff0c275c6cb5894235d5acbeed4b587edbb6f2c0
2019-12-04 06:20:35 -08:00
Paul Duffin
46fc58ee8d Disable TestHostSnapshot test for non-linux build
Building sdk snapshots on anything other than linux glibc is not
supported at the moment.

Bug: 145598135
Test: m nothing
Change-Id: I34155252545e9f37c5361ed94a27a87024fcece2
2019-12-04 10:15:40 +00:00
Dan Willemsen
67d4258786 Test par file argument handling
This had been broken with my python3 launcher.

Test: build/soong/python/tests/runtest.sh
Change-Id: I0524e5a80244bf1080cb28237b23ef0d023e03dc
2019-12-03 23:54:04 -08:00
Jiyong Park
1753e58df7 Remove implicit outputs for the flattened APEX Make module
When flattened, the Make module for an APEX is a phony package for the
files in it. There is no output (either implcit or explicit) there.

Bug: 144338929
Test: m
Test: check the generated Android-<target>.mk file
Change-Id: I86df3b69f402a7e2d94bafa39f2aad3312d8c28b
2019-12-04 13:38:56 +09:00
Sundong Ahn
0d7dff48f5 Add system_ext_specific
The system_ext partition was created. So if java_sdk_library module is
installed to system_ext partition, .xml and .jar install path must be
changed to system_ext.

Bug: 143440787
Test: add system_ext_specific to java_sdk_module && make -j && check
system_ext parition

Change-Id: Ie0d0df426d4aa96ac89eb4215e7376eea3f03f54
2019-12-04 04:00:21 +00:00
Xin Li
3dc38c94a4 Merge "Merge Android10 QPR1 into AOSP master"
am: e06c84e103

Change-Id: I214380c0e1d2b34badb8bb57fe90d2f8b286fc91
2019-12-03 16:07:37 -08:00
Treehugger Robot
e06c84e103 Merge "Merge Android10 QPR1 into AOSP master" 2019-12-03 23:15:05 +00:00
Chih-Hung Hsieh
204cad9008 Merge "Generate tradefed config for rust device tests."
am: dc9369ef7f

Change-Id: I6948b70be74135d81dfaa310927c12848db16fdf
2019-12-03 13:28:35 -08:00