Commit Graph

5488 Commits

Author SHA1 Message Date
Jim Tang
c44ba2a6f9 Dump deps for native and java to json files.
Abandon the use of SOONG_COLLECT_CC_DEPS and SOONG_COLLECT_JAVA_DEPS,
soong can generate module_bp_cc_deps.json and module_bp_java_deps.json
without passing any variables.

BUG: 204397180

Test: rm out/soong/module_bp*json
"m nothing" should see both json files were generated.

Change-Id: If30ad5dbabb6709f834134cbc9a513a9bdc9d9de
2021-11-05 12:06:15 +08:00
Paul Duffin
10fb76f467 Rename core-current-stubs-system-modules to be more consistent
Renames to core-public-stubs-system-modules so that it is of the format
core-<sdk-kind>-stubs-system-modules.

Bug: 204189791
Test: m nothing
Change-Id: Iac565c940c2ef92be9cc64c0c6b8102a26afe0dd
2021-11-03 16:58:29 +00:00
Paul Duffin
8380ee66a5 Merge "Use module-lib system modules when building from prebuilts" 2021-11-03 15:32:42 +00:00
Mohammad Islam
569f032a9e Merge "Add exemption for test_framework-apexd and test_service-apexd jars" 2021-11-02 08:51:01 +00:00
Colin Cross
1caea35278 Merge "Export Soong install rules to Make" 2021-11-01 21:31:56 +00:00
Paul Duffin
004547facd Use module-lib system modules when building from prebuilts
When building from source the build uses the java system modules for
the public or module APIs as needed. However, previously when building
from prebuilts it would always use the public API. That difference lead
to build failures when building from prebuilts.

This change makes the selection of java system modules when building
from prebuilts consistent with the selection when building from
sources.

As API levels 30 and 31 (which are the only previous releases to
provide system modules) did not provide separate java system modules
for the module-lib API those levels always use the public APIs.

Bug: 204189791
Test: - before applying these change
      m TARGET_BUILD_APPS=framework-connectivity
      - build fails with compilation error due to missing module APIs
      m sdk dist
      cp out/dist/system-modules/module-lib/core-for-system-modules.jar prebuilts/sdk/current/module-lib/core-for-system-modules.jar
      - apply these changes
      m TARGET_BUILD_APPS=framework-connectivity
      - build passes as expected
Change-Id: Id113ff014e7892b1009fbcaad89b1ae23a7c3b79
2021-11-01 16:40:28 +00:00
Paul Duffin
b077bcc6bc Create prebuilt system modules for all API scopes
Previously, system modules were only created for the public API scope.
This change creates them for any API scope as long as its directory has
a core-for-system-modules.jar.

It does that by hooking into the existing logic for creating a
java_import for all jars in the API directories and creating a
java_system_modules for every core-for-system-modules.jar file. That
avoids the need for extra path globs.

Test: m droid
      m sdk_public_current_system_modules
      - works as expected.
      m sdk_module-lib_current_system_modules
      - fails with missing target as expected.
      touch prebuilts/sdk/current/module-lib/core-for-system-modules.jar
      m sdk_module-lib_current_system_modules
      - fails with invalid jar file as expected.
Bug: 204189791
Change-Id: I27a264941009e03439d5d847dab14a7b4f6f119f
2021-11-01 16:37:49 +00:00
Paul Duffin
1cad3a53db Make prebuilt_api test environment realistic
Previously, the fixture preparer for prebuilt_apis would add a
core-for-system-modules.jar file in every API directory even though
currently they only exist in the public API directories.

This change makes the test environment more realistic by only creating
them for the public API. Rather than hard code that into the test code
(which would duplicate the hard coding in the decodeSdkDep func) this
extracts a function that is used by both. That ensures that any changes
to that func will be reflected in both the test and runtime behavior.

Bug: 204189791
Test: m nothing
Change-Id: I346ac9c0dcf407c61de16b6027663a05821bcf62
2021-11-01 16:36:23 +00:00
Paul Duffin
6d448b7a0a Add test for prebuilt_apis creation of system modules
Previously, there were no tests for this (outside uses in other tests).
This change adds a test that uses the fixture preparers that create a
prebuilt_api in order to make prebuilt APIs available. That ensures
that both the prebuilt_api is working as expected and the preparer
creates a realistic test environment.

Bug: 204189791
Test: m nothing
Change-Id: I57352aa00f7b268e5286be92f177764dd63ba7e8
2021-11-01 16:36:02 +00:00
Paul Duffin
bce9f08d02 Dist API specific core-for-system-modules
Previously, the dist only contained a core-for-system-modules.jar for
the public API. This change adds API specific directories containing a
core-for-system-modules.jar file for each of the following APIs:
* public
* module-lib

Bug: 204189791
Test: rm -fr out/dist
      m sdk dist
      find out/dist -name core-for-system-modules.jar
      - outputted the following:
      out/dist/core-for-system-modules.jar
      out/dist/system-modules/module-lib/core-for-system-modules.jar
      out/dist/system-modules/public/core-for-system-modules.jar
Change-Id: Id1845926e2085f70d58e9fc22e9c11cb3d62b919
2021-11-01 14:01:39 +00:00
Joe Onorato
22e973c539 Merge "Add support for java streaming proto" 2021-11-01 00:14:59 +00:00
Colin Cross
6301c3cffa Export Soong install rules to Make
Previously Soong's install rules have been disabled when embedded
in Make (ctx.Config().KatiEnabled() == true).  The primary blocker
for moving installation into Soong has been the `required` proeprty,
which is too vague to be easily handled in Soong.  Keeping
installation in Make has resulted in two host bin directories,
the Make-owned directory (e.g. out/host/linux-x86/bin), and the
Soong-owned directory (e.g. out/soong/host/linux-x86/bin).  The
lack of knowledge in Soong about the final, Make-owned installation
location makes it hard to support NOTICE files entirely in Soong.

This patch begins to solve this problem by supporting the creation of
the installation rules into Soong, but rather than writing the rules
to the ninja file it writes them to a Makefile and lets Kati convert
them to ninja.  This allows Kati to inject extra dependencies to
handle the `required` property.

Converting all modules to create their installation rules in Soong
would be too complex, so only modules that return true from
InstallBypassMake will use the Soong installation rules.  This
is currently only set for robolectric tests.

Bug: 204136549
Test: m checkbuild
Change-Id: I28af9fa7fadece8ea1f98f5efd140c823751cae7
2021-10-29 16:34:55 -07:00
Paul Duffin
e8412c4e7a Merge changes I88ee9709,If2ea3fde,I657fbcde,I585a8861
* changes:
  Use toModule for all "standard" API surfaces
  Run TestClasspath test cases with Always_use_prebuilt_sdks=true/false
  Refactor TestClasspath to simplify follow up change
  Add tests for prebuilts of all API surfaces
2021-10-29 19:41:33 +00:00
Jiakai Zhang
65947f6fc4 Merge "Add a build-time check for dexpreopting system server jars." 2021-10-29 18:00:51 +00:00
Paul Duffin
e9758b0415 Use toModule for all "standard" API surfaces
For the purposes of this change a standard API surface is one of the
following:
* public
* system
* test
* module-lib
* system-server

Test: m droid
Bug: 204189791
Change-Id: I88ee9709430ca455dd6c7d1523ae22f8c22b0b7e
2021-10-29 17:57:12 +01:00
Paul Duffin
132c3e61c0 Run TestClasspath test cases with Always_use_prebuilt_sdks=true/false
Previously, the TestClasspath test cases were only run with the default
setting of Always_use_prebuilt_sdks=false. That meant that some of the
code under test that depended on the setting of that variable was not
tested properly.

This change runs the test cases m with Always_use_prebuilt_sdks=true as
well. Those test cases whose behavior depends on the setting of that
variable are split into two separate test cases, each of which only
runs with the appropriate setting of that variable. All other test
cases are run for both settings of the variable.

That revealed a slight issue with the test setup (a missing
prebuilts/sdk/public/core/android.jar file) which broke the
core_current test when run with Always_use_prebuilt_sdks=true which has
also been fixed.

Bug: 204189791
Test: m nothing
Change-Id: If2ea3fde40c7573262e93691af0b5a57e4d54469
2021-10-29 17:57:12 +01:00
Paul Duffin
0b3b36a402 Refactor TestClasspath to simplify follow up change
Extracts classpathTestCase and extracts the logic for creating the test
into a separate testClasspathTestCases func.

Bug: 204189791
Test: m nothing
Change-Id: I657fbcde23a18f7f5651c174fbf17a9b7b7f1a9a
2021-10-29 17:57:12 +01:00
Paul Duffin
12e311d958 Add tests for prebuilts of all API surfaces
Previously, the tests only covered checking the sdk dependencies added
when using a prebuilt current (public) and system SDKs, i.e. with
sdk_version set to "current_30" or "system_30". This change adds tests
to cover other APIs, e.g. "test_30", "module_30" and
"system_server_30". It adds tests for "module" and "system-server" up
to API level 32 as following changes will give them different behavior
as API levels less than or equal to 31 do not have a separate
core-for-system-modules.jar file for the module-lib API.

Bug: 204189791
Test: m nothing
Change-Id: I585a88615439a24abf75250114a86113b5f5da57
2021-10-29 17:57:12 +01:00
Jiakai Zhang
0a0a2fbea9 Add a build-time check for dexpreopting system server jars.
This change adds a Soong module that stores the paths to the compilation
artifacts of system server jars in a variable, which will then be
consumed by Make to check if the artifacts are installed. When the check
fails, it means that dexpreopting is not working for some system server
jars and needs to be fixed.

Bug: 201371822
Test: m nothing
Test: manual -
  1. Add "service-permission" to DEXPREOPT_DISABLED_MODULES (https://cs.android.com/android/platform/superproject/+/master:build/make/core/product_config.mk?q=DEXPREOPT_DISABLED_MODULES)
  2. m nothing
  3. See the error:
    Missing compilation artifacts. Dexpreopting is not working for some system server jars
    Offending entries:
    system/framework/oat/x86_64/apex@com.android.permission@javalib@service-permission.jar@classes.odex
    system/framework/oat/x86_64/apex@com.android.permission@javalib@service-permission.jar@classes.vdex
Change-Id: I4816f19668f1dae180a34adafdbfa448c97aa0db
2021-10-29 11:55:20 +00:00
Mohammad Islam
c874504988 Merge "Make car apex's library optional in PRODUCT_APEX_SYSTEM_SERVER_JARS" 2021-10-29 09:40:49 +00:00
Joe Onorato
83fdc94102 Add support for java streaming proto
Test: m services with ag/15937571 applied
Bug: 199427116
Change-Id: Icf262efa015900f14d9f4cabe3b8af7ec872a668
2021-10-28 15:08:34 -07:00
Samiul Islam
7b385c5677 Add exemption for test_framework-apexd and test_service-apexd jars
For testing purpose, we need to add java libraries to bootclasspath and
systemserverclasspath that are not present in PRODUCT_APEX_BOOT_JARS and
PRODUCT_APEX_SYSTEM_SERVER_JARS.

Currently, we can't unconditionally add all the contents to the config
for test APEXs. Until a formal way is introduced, hardcoding the
artifact in the build system like how it's done for
test_framework-sdkextensions.

Bug: 187444679
Test: atest ApexTestCases
Test: atest StagedInstallInternalTest
Change-Id: I1f7b01a57bc9c46163e025de2ab00e671a42b202
2021-10-28 06:12:48 +01:00
Keun young Park
d64ab233e4 Make car apex's library optional in PRODUCT_APEX_SYSTEM_SERVER_JARS
- car-frameworks-service-module is added from com.android.car.framework
  apex which only exisits in car products.

Bug: 203233647
Bug: 203201931
Test: build, bootup
Change-Id: I5f13b0505c1ac4fd2fd7b6559e080c8a6fb526e0
Merged-In: I5f13b0505c1ac4fd2fd7b6559e080c8a6fb526e0
(cherry picked from commit 6e3ba9ac3b)
2021-10-28 06:12:26 +01:00
Paul Duffin
9804da0544 Dedup use of "prebuilt_"
Bug: 177893770
Test: m nothing
Change-Id: I664ee4d9c3aafda3de0a3f8985ee23f25db38680
2021-10-26 10:43:23 +01:00
Colin Cross
d26943459d Merge changes Ifa4594ea,Ia756f309
* changes:
  Reduce commands per shard for java protos
  Up per-product memory usage to 30 GB
2021-10-18 22:03:43 +00:00
Colin Cross
93fa5ef99d Reduce commands per shard for java protos
`m platformprotos` with a long OUT_DIR can trigger E2BIG errors in
posixspawn, reduce the commands per shard.

Test: OUT_DIR=$PWD/really-long-out-directory m platformprotos
Change-Id: Ifa4594eac7ba0e1091df0ef75e4dfc3ff366939b
2021-10-18 10:19:03 -07:00
Jiakai Zhang
ae9ba1c93a Merge "Extract the logic of setting Uncompress_dex to a common place." 2021-10-15 03:30:47 +00:00
Dan Willemsen
9fe1410213 Add android_sdk_repo_host to build platform-tools&build-tools
The Android SDK has been built with Make up until now, monolithically,
then split up into several sdk-repo zip files for different purposes.
The Mac and Windows SDKs really only need to contain the platform-tools
and build-tools pieces, but due to this monolithic sdk zip, we had to
build the whole SDK first.

This adds an `android_sdk_repo_host` module that can build these
platform-tools and build-tools zips.

Bug: 187222815
Change-Id: I55809e1d7447dd65e22461f921b2b8abb6d5f822
2021-10-13 01:25:02 -07:00
Steven Moreland
371dc53d18 Merge "Allow defaults in java_genrule modules" 2021-10-11 17:47:22 +00:00
Anton Hansson
6fcddda43c Merge "Update reference to merged services txt" 2021-10-11 15:57:02 +00:00
Jiakai Zhang
22450f25da Extract the logic of setting Uncompress_dex to a common place.
Bug: 194150908
Test: m nothing
Change-Id: Ia33965067586cfd48183e6d17f5d17626eef3c5a
2021-10-11 03:05:20 +00:00
Colin Cross
99c3ed94b8 Allow defaults in java_genrule modules
Call InitDefaultableModule in java_genrule factories to allow
the defaults property to be used.

Bug: 202547639
Test: m checkbuild
Change-Id: I3a5960304af3e9babb30d036d63016087000d773
2021-10-08 17:52:32 -07:00
Anton Hansson
f2dffd9c04 Update reference to merged services txt
The module is changing name. Update the reference.

Bug: 169103987
Test: m
Change-Id: I1e3f05b6d9c0262adcde6881ea79df9338bb363a
2021-10-08 16:15:10 +01:00
Xin Li
800112b97f Merge "Merge Android 12" 2021-10-07 23:50:21 +00:00
Pedro Loureiro
2937eb5bb4 Merge "Add module-lib to droidstubs' supported sdk types" 2021-10-07 10:17:03 +00:00
Paul Duffin
dc3951c407 Merge "Allow implementation details to be excluded from signature patterns file" am: 6546573ec9
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1788094

Change-Id: I28f0ddcfe3746a8bf37471f9f1c9b9771fab690a
2021-10-06 12:35:12 +00:00
Paul Duffin
bf027e4e58 Retry: Separate hidden API flags needed in sdk snapshots for S and T am: 191be3a186
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1844294

Change-Id: Id8977b8160f175b736214428cae94ea27f95675b
2021-10-05 18:04:10 +00:00
Pedro Loureiro
cc203505b9 Add module-lib to droidstubs' supported sdk types
If the android.jar is not found in the module-lib folder, it falls back
to searching it in the system folder and then falls back on the public
folder.

This only brings the functionality, we are not yet using it.

Test: m nothing
Bug: 202023154
Change-Id: I99c9004740c8e51faf53ebc3ba2444e8164215f3
2021-10-05 15:40:46 +00:00
Paul Duffin
1e18e98b14 Allow implementation details to be excluded from signature patterns file
Previously, the signature-patterns.csv file (which was output in an sdk
snapshot) included implementation details, e.g. the names of classes
that are not part of any API, including the hidden API. This change
allows a bootclasspath_fragment module owner to customize the generated
signature patterns file to remove as many implementation details as
possible from the sdk snapshot. That means that implementation only
changes would not require that sdk snapshots be updated in sync with
the corresponding APEX.

Module owners can remove implementation in the following ways:
1) Limit the number of packages that are treated as split packages by
   explicitly specifying the split packages in the split_packages
   property. A split package is one whose classes are provided by
   separate bootclasspath_fragment modules and so the signature
   patterns has to include every class in that package provided by a
   specific bootclasspath_fragment module, including implementation
   classes, instead of just listing the package.

   The default is to treat all packages as being split, so if no
   split_packages is specified then it defaults to ["*"] which matches
   all packages. Assuming that no package was split unless specifically
   stated would require that all the modules that had split packages
   explicitly list them before this change could be submitted as
   without them this change would break the build.

   Once all existing modules with split packages have been updated to
   include them then the default may be changed.

2) Adding package prefixes for any hierarchy of packages that are
   owned solely by that bootclasspath_fragment. This removes the need
   to list the packages in that hierarchy, including implementation
   specific packages.

Bug: 194063708
Test: atest --host verify_overlaps_test signature_patterns_test
      m out/soong/hiddenapi/hiddenapi-flags.csv
      - manually change files to cause difference in flags to check
        that it detects the differences.
Change-Id: If2f90cfc41b6fff9fa4ac8b9d4973110938b9392
2021-10-05 15:47:05 +01:00
Paul Duffin
191be3a186 Retry: Separate hidden API flags needed in sdk snapshots for S and T
Previous change that was reverted: https://r.android.com/1835222
An additional test was added that revealed a bug in the previous change
which has been fixed here.

Previously, the behavior of the stub_flags and all_flags properties
was different between S and T. In S they contained paths for the
complete set of stub flags and all the encoded flags. However, in T
they contained filtered sets of flags which if used in S would prevent
build checks from detecting possible inconsistencies. Also, a new
signature_patterns property was added in T that is not supported in S.

This change creates separate properties/files for T and reverts the
behavior of the properties/files that were added in S back to how they
behaved in S. The new properties are called filtered_stub_flags and
filtered_flags.

The S and T properties are tagged with the appropriate
supported_build_releases tag to ensure that they are only output when
specifically targeted.

Bug: 197842263
Test: m nothing
Change-Id: I1ce0a3d6623dabf73e32af1a7457b9b444fc3b7c
2021-10-05 15:27:42 +01:00
Treehugger Robot
3e745e63f3 Merge "Bp2Build common properties auto-handling" am: d905146590
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1815376

Change-Id: I38f1063302ea90583a51e625076e6232b4364ab7
2021-10-04 19:20:28 +00:00
Treehugger Robot
d905146590 Merge "Bp2Build common properties auto-handling" 2021-10-04 19:02:07 +00:00
Ulya Trafimovich
ef92591c59 Update variable name in error messages. am: e5b2b49a73
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1844308

Change-Id: Icb57f58d9efdf8793162254eefe8e3150d6d9cf5
2021-10-04 16:18:58 +00:00
Ulya Trafimovich
e5b2b49a73 Update variable name in error messages.
Variable PRODUCT_UPDATABLE_SYSTEM_SERVER_JARS was renamed to
PRODUCT_APEX_SYSTEM_SERVER_JARS in https://r.android.com/1779472.

Bug: 191127295
Test: m nothing
Change-Id: I0c8ae839cab443c824760b40d30b4374982010f9
2021-10-04 15:43:39 +01:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
447f6c99c9 Bp2Build common properties auto-handling
Introduce `commonAttributes` & `fillCommonBp2BuildModuleAttrs used in
CreateBazelTargetModule

Adapt `bp2BuildInfo` to use `commonAttrs` instead of `Name`.
And thus also all downstream users of `CreateBazelTargetModule`.

As initial user, the Soong `required` property will be
translated to Bazel's `data`.

Bug: 198146582, 196091467
Test: build_converstion_test.go:TestCommonBp2BuildModuleAttrs
Test: go test
Test: mixed_{libc,droid}.sh
Change-Id: Ib500e40f7e2cb48c459f1ebe3188962fc41ec124
2021-10-04 14:43:04 +00:00
Paul Duffin
6c68db45d3 Remove unnecessary additionalCheckedModules field am: 0b58fdb058
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1842815

Change-Id: Ic6e55395138cfed7257529b2c5165bb4907460e1
2021-10-04 11:11:47 +00:00
Paul Duffin
0b58fdb058 Remove unnecessary additionalCheckedModules field
This change removes the field and the code that uses it which is
effectively dead code as it only does something when the list is not
empty and a previous change removed the only code that added a Path to
this list.

Bug: 201753898
Test: m nothing
Change-Id: I67c74ad2a3b115fd3b20b3b416f45b92c749749c
2021-10-04 10:42:56 +01:00
Salmax Chang
12d6b80da2 Merge changes from topic "bb_201948713" am: 8bb9c6a1d9
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1844137

Change-Id: Ia042b5b24141be4ebfa17ab15adf34c3af632635
2021-10-04 08:13:19 +00:00
Salmax Chang
8bb9c6a1d9 Merge changes from topic "bb_201948713"
* changes:
  Revert "Support generating sdk snapshot for specific build release"
  Revert "Separate hidden API flags needed in sdk snapshots for S and T"
2021-10-04 07:53:19 +00:00
Salmax Chang
8194d61ae0 Revert "Separate hidden API flags needed in sdk snapshots for S and T"
This reverts commit ba68c93630.

Reason for revert: suspect to cause build break in b/201948713

Change-Id: Ide3693ac1f8e24bd2f7c12857361885cb6b787ef
2021-10-04 05:28:58 +00:00