Commit Graph

45591 Commits

Author SHA1 Message Date
Jingwen Chen
9af49a49f8 Fix //conditions:default excludes computation for LabelListAttribute.
Previously, excludes computation for the default condition in label list
selects was clobbering pre-existing values. This CL fixes it by
performing a union of the new values with existing ones instead.

Test: properties_test.go
Bug: 198556411
Change-Id: Id11f4fb14e359201304afde0d8ba856851d41395
2021-11-03 01:56:42 +00:00
Jingwen Chen
55bc820d66 bp2build: split Bazel conversion context into smaller ones,
and change TopDownMutatorContext signatures to use Bazel conversion context.

This minimizes the context interfaces/functions actually needed to
convert a module, and makes such interfaces easier to mock/test.

Test: CI
Change-Id: Id573d97023d59e06ef70e1f54437024d3f7aadbd
2021-11-02 06:51:27 +00:00
Jingwen Chen
80b6b64db5 Add an error check in bazelPackage for malformed labels.
Bug: 204281595
Test: CI
Change-Id: Ia94a0b5c8dd3a0294475e9bc816a19ae03f7342d
2021-11-02 06:24:31 +00:00
Dan Willemsen
8528f4ec5e Add Darwin+Arm64 toolchain support
This just sets up the toolchain and allows Darwin+Arm64 to be specified
as a HostCross target. These variants will not be exported to Make, or
be installed on a Soong-only build. A future CL will add support for
universal binaries using these variants.

This config is a bit stranger than the regular 64/32 multilib, as it's
two primary 64-bit configs. And on a Darwin/X86 machine, the Arm64
versions are HostCross (doesn't work on the current machines), while a
Darwin/Arm64 machine, either version works (if Rosetta is installed).

Bug: 203607969
Change-Id: Iacaed77d267773672da027cd74917e33fb1c1e94
2021-11-01 15:07:37 -07:00
Colin Cross
1caea35278 Merge "Export Soong install rules to Make" 2021-11-01 21:31:56 +00:00
Treehugger Robot
2bbd2c55c5 Merge "Create prebuilt system modules for all API scopes" 2021-11-01 19:16:03 +00:00
Treehugger Robot
57be76d9ac Merge changes I346ac9c0,I57352aa0
* changes:
  Make prebuilt_api test environment realistic
  Add test for prebuilt_apis creation of system modules
2021-11-01 18:11:06 +00:00
Treehugger Robot
0888700fe1 Merge "Make manifest_check errors colorful." 2021-11-01 17:11:52 +00:00
Paul Duffin
b077bcc6bc Create prebuilt system modules for all API scopes
Previously, system modules were only created for the public API scope.
This change creates them for any API scope as long as its directory has
a core-for-system-modules.jar.

It does that by hooking into the existing logic for creating a
java_import for all jars in the API directories and creating a
java_system_modules for every core-for-system-modules.jar file. That
avoids the need for extra path globs.

Test: m droid
      m sdk_public_current_system_modules
      - works as expected.
      m sdk_module-lib_current_system_modules
      - fails with missing target as expected.
      touch prebuilts/sdk/current/module-lib/core-for-system-modules.jar
      m sdk_module-lib_current_system_modules
      - fails with invalid jar file as expected.
Bug: 204189791
Change-Id: I27a264941009e03439d5d847dab14a7b4f6f119f
2021-11-01 16:37:49 +00:00
Paul Duffin
1cad3a53db Make prebuilt_api test environment realistic
Previously, the fixture preparer for prebuilt_apis would add a
core-for-system-modules.jar file in every API directory even though
currently they only exist in the public API directories.

This change makes the test environment more realistic by only creating
them for the public API. Rather than hard code that into the test code
(which would duplicate the hard coding in the decodeSdkDep func) this
extracts a function that is used by both. That ensures that any changes
to that func will be reflected in both the test and runtime behavior.

Bug: 204189791
Test: m nothing
Change-Id: I346ac9c0dcf407c61de16b6027663a05821bcf62
2021-11-01 16:36:23 +00:00
Paul Duffin
6d448b7a0a Add test for prebuilt_apis creation of system modules
Previously, there were no tests for this (outside uses in other tests).
This change adds a test that uses the fixture preparers that create a
prebuilt_api in order to make prebuilt APIs available. That ensures
that both the prebuilt_api is working as expected and the preparer
creates a realistic test environment.

Bug: 204189791
Test: m nothing
Change-Id: I57352aa00f7b268e5286be92f177764dd63ba7e8
2021-11-01 16:36:02 +00:00
Ulya Trafimovich
b4c19f8a31 Make manifest_check errors colorful.
Insert ANSI escape codes in the error text. Colors are red, blue, green
and some text is in bold --- these are all bright enough on either dark
or light background.

Also, add a link to the online documentation.

Bug: 132357300
Test: manually mangle one of Android.bp files to get a manifest_check
  error and observe that it is colorful.
Change-Id: I2af2aa0415d0eb0eabc88dc5504198e11bfb91b6
2021-11-01 14:39:46 +00:00
Paul Duffin
bce9f08d02 Dist API specific core-for-system-modules
Previously, the dist only contained a core-for-system-modules.jar for
the public API. This change adds API specific directories containing a
core-for-system-modules.jar file for each of the following APIs:
* public
* module-lib

Bug: 204189791
Test: rm -fr out/dist
      m sdk dist
      find out/dist -name core-for-system-modules.jar
      - outputted the following:
      out/dist/core-for-system-modules.jar
      out/dist/system-modules/module-lib/core-for-system-modules.jar
      out/dist/system-modules/public/core-for-system-modules.jar
Change-Id: Id1845926e2085f70d58e9fc22e9c11cb3d62b919
2021-11-01 14:01:39 +00:00
Joe Onorato
22e973c539 Merge "Add support for java streaming proto" 2021-11-01 00:14:59 +00:00
Colin Cross
91eaa6316a Merge changes I04f2f558,Ie16a6264,I83a3be9f,Icb3eeef3
* changes:
  Rename amod variable in AndroidMkEntries.fillInEntries to base
  Make HostToolPath, HostJNIToolPath and sboxPathForToolRel use pathForInstall
  Add PathForGoBinary
  Remove PathForOutput from InstallPathToOnDevicePath
2021-10-30 19:19:53 +00:00
Colin Cross
e7fc5a25c5 Merge "Use --skip-config --soong-only instead of --skip-make" 2021-10-30 15:56:22 +00:00
Colin Cross
4985224198 Merge "Make installable arch-variant and static vs shared variant for cc modules" 2021-10-30 04:56:22 +00:00
Chih-hung Hsieh
7b2239f648 Merge "Add -{checkbuild,tidy} targets only for modules in mm" 2021-10-30 03:57:06 +00:00
Chih-Hung Hsieh
a3d135b1af Add -{checkbuild,tidy} targets only for modules in mm
Test: lunch aosp_coral-userdebug; WITH_TIDY=1
      make 7-libgptutils.coral-checkbuild 7-libgptutils.coral-tidy
Test: only coral libgptutils in out/build-aosp_coral.ninja
Test: lunch aosp_arm64-eng; make has no *libgptutils* tagets
Test: no libgptutils in out/build-aosp_arm64.ninja
Test: lunch aosp_cf_arm64_phone-userdebug; make libskia-checkbuild
Test: lunch aosp_arm64-eng; WITH_TIDY=1 make tidy-soong

Bug: 202763874
Change-Id: I2d5a3fdc627d62e43c3f10e61b338a03c10440be
2021-10-29 18:02:27 -07:00
Treehugger Robot
cae893ae40 Merge "apex.custom_sign_tool property" 2021-10-29 23:39:26 +00:00
Colin Cross
6301c3cffa Export Soong install rules to Make
Previously Soong's install rules have been disabled when embedded
in Make (ctx.Config().KatiEnabled() == true).  The primary blocker
for moving installation into Soong has been the `required` proeprty,
which is too vague to be easily handled in Soong.  Keeping
installation in Make has resulted in two host bin directories,
the Make-owned directory (e.g. out/host/linux-x86/bin), and the
Soong-owned directory (e.g. out/soong/host/linux-x86/bin).  The
lack of knowledge in Soong about the final, Make-owned installation
location makes it hard to support NOTICE files entirely in Soong.

This patch begins to solve this problem by supporting the creation of
the installation rules into Soong, but rather than writing the rules
to the ninja file it writes them to a Makefile and lets Kati convert
them to ninja.  This allows Kati to inject extra dependencies to
handle the `required` property.

Converting all modules to create their installation rules in Soong
would be too complex, so only modules that return true from
InstallBypassMake will use the Soong installation rules.  This
is currently only set for robolectric tests.

Bug: 204136549
Test: m checkbuild
Change-Id: I28af9fa7fadece8ea1f98f5efd140c823751cae7
2021-10-29 16:34:55 -07:00
Colin Cross
f1f763a981 Rename amod variable in AndroidMkEntries.fillInEntries to base
AndroidMkEntries.fillInEntries calls its android.ModuleBase amod,
despite also handling an android.Module.  Rename amod to base to
match other locations, and add a new amod for the android.Module.
This will simplify the next patch that needs to access the
android.Module.

Bug: 204136549
Test: m checkbuild
Change-Id: I04f2f558959def22e8b3f5b8c534b8d655b06a4e
2021-10-29 16:34:55 -07:00
Colin Cross
34d60c9281 Use --skip-config --soong-only instead of --skip-make
--skip-make is deprecated because of its odd behavior around the
out/.soong.kati_enabled file.  Replace it with --skip-config --soong-only
instead, which will unambigiously skip running kati.

Bug: 204136549
Test: build/soong/tests/run_integration_tests.sh
Test: build/soong/scripts/build-mainline-modules.sh
Test: build/soong/scripts/build-ndk-prebuilts.sh
Change-Id: I629ee7542687aaafd68eea50346b0cf53eafe1ef
2021-10-29 16:34:55 -07:00
Colin Cross
790ef35d1e Make HostToolPath, HostJNIToolPath and sboxPathForToolRel use pathForInstall
Use pathForInstall instead of PathForOutput for HostToolPath,
HostJNIToolPath and sboxPathForToolRel so that they internally produce
an InstallPath that can later support being converted to Make install
path.

Bug: 204136549
Test: m checkbuild
Change-Id: Ie16a62641d113873daeec4d1dd4261251bc0d0eb
2021-10-29 16:34:55 -07:00
Colin Cross
a44551fec6 Add PathForGoBinary
Add PathForGoBinary that uses pathForInstall to return the install
path of a GoBinaryTool.  This will replace various places that used
PathForOutput to reconstruct a path to a Go tool, and will support
moving Go tools to the Make install directory outside of the
PathForOutput directory in a future patch.

Bug: 204136549
Test: m checkbuild
Change-Id: I83a3be9f5c621975540f5ed601a0b9e2611c98b9
2021-10-29 16:34:55 -07:00
Colin Cross
b1692a3468 Remove PathForOutput from InstallPathToOnDevicePath
The next patches will make more InstallPaths use the Make output
directory, which is not inside PathForOutput.  Fix the assumption
that the InstallPath is relative to PathForOutput by storing
enough information in the InstallPath to find the on-device path
without PathForOutput.

Bug: 204136549
Test: soong tests
Change-Id: Icb3eeef3f1c72f773f333267f8a7dfc503feacb5
2021-10-29 16:34:55 -07:00
Colin Cross
1bc9412fb3 Make installable arch-variant and static vs shared variant for cc modules
libc's native_bridge shared variant needs to be installable: false,
make the installable property arch-variant and allow the static and
shared clauses to override it.

This relands I8e0eaed2169e89520b07bb0326bb97ac0b4a70ac with fixes to
allow only the shared variant to be marked installable: false.

Bug: 204136549
Test: m checkbuild
Change-Id: Ie579b5ba56ca21f4d9243088afd5078d8c122a68
2021-10-29 15:00:38 -07:00
Treehugger Robot
a471535c72 Merge "Revert "Do not build tests with ThinLTO"" 2021-10-29 19:48:30 +00:00
Paul Duffin
e8412c4e7a Merge changes I88ee9709,If2ea3fde,I657fbcde,I585a8861
* changes:
  Use toModule for all "standard" API surfaces
  Run TestClasspath test cases with Always_use_prebuilt_sdks=true/false
  Refactor TestClasspath to simplify follow up change
  Add tests for prebuilts of all API surfaces
2021-10-29 19:41:33 +00:00
Colin Cross
972e7ba893 Merge "Revert "Make installable arch-variant for cc modules"" 2021-10-29 18:10:59 +00:00
Colin Cross
545104bea1 Revert "Make installable arch-variant for cc modules"
Revert submission 1874144-native_bridge_libc_dup_rules

Reason for revert: breaks ndk_translation_all build
Reverted Changes:
I5379aa959:Make native bridge libc uninstallable
I2220d6b48:Remove BUILD_BROKEN_DUP_RULES from cuttlefish buil...
I8e0eaed21:Make installable arch-variant for cc modules

Bug: 204136549
Fixes: 204572444
Change-Id: I089a13675123a33a3ec3fea8eaa40c65fb2e5e59
2021-10-29 18:08:36 +00:00
Jiakai Zhang
65947f6fc4 Merge "Add a build-time check for dexpreopting system server jars." 2021-10-29 18:00:51 +00:00
Yi Kong
04e459d9ff Revert "Do not build tests with ThinLTO"
On second thought, this is actually counter-productive rather than an
optimisation. LTOing test code is usually quite cheap, however since
tests can pull in additional static libraries (e.g. to set up test
environment), we will have to create non-LTO variants for these
dependent libraries, which can be much more expensive.

This reverts commit 85d7297318.

Bug: 195134194
Bug: 203737712
Change-Id: I1a105136075133a5b5895d00847d8796c419964c
Test: m GLOBAL_THINLTO=true dist device-tests platform_tests
2021-10-29 17:53:27 +00:00
Paul Duffin
e9758b0415 Use toModule for all "standard" API surfaces
For the purposes of this change a standard API surface is one of the
following:
* public
* system
* test
* module-lib
* system-server

Test: m droid
Bug: 204189791
Change-Id: I88ee9709430ca455dd6c7d1523ae22f8c22b0b7e
2021-10-29 17:57:12 +01:00
Paul Duffin
132c3e61c0 Run TestClasspath test cases with Always_use_prebuilt_sdks=true/false
Previously, the TestClasspath test cases were only run with the default
setting of Always_use_prebuilt_sdks=false. That meant that some of the
code under test that depended on the setting of that variable was not
tested properly.

This change runs the test cases m with Always_use_prebuilt_sdks=true as
well. Those test cases whose behavior depends on the setting of that
variable are split into two separate test cases, each of which only
runs with the appropriate setting of that variable. All other test
cases are run for both settings of the variable.

That revealed a slight issue with the test setup (a missing
prebuilts/sdk/public/core/android.jar file) which broke the
core_current test when run with Always_use_prebuilt_sdks=true which has
also been fixed.

Bug: 204189791
Test: m nothing
Change-Id: If2ea3fde40c7573262e93691af0b5a57e4d54469
2021-10-29 17:57:12 +01:00
Paul Duffin
0b3b36a402 Refactor TestClasspath to simplify follow up change
Extracts classpathTestCase and extracts the logic for creating the test
into a separate testClasspathTestCases func.

Bug: 204189791
Test: m nothing
Change-Id: I657fbcde23a18f7f5651c174fbf17a9b7b7f1a9a
2021-10-29 17:57:12 +01:00
Paul Duffin
12e311d958 Add tests for prebuilts of all API surfaces
Previously, the tests only covered checking the sdk dependencies added
when using a prebuilt current (public) and system SDKs, i.e. with
sdk_version set to "current_30" or "system_30". This change adds tests
to cover other APIs, e.g. "test_30", "module_30" and
"system_server_30". It adds tests for "module" and "system-server" up
to API level 32 as following changes will give them different behavior
as API levels less than or equal to 31 do not have a separate
core-for-system-modules.jar file for the module-lib API.

Bug: 204189791
Test: m nothing
Change-Id: I585a88615439a24abf75250114a86113b5f5da57
2021-10-29 17:57:12 +01:00
Colin Cross
f87065fedb Merge "Make installable arch-variant for cc modules" 2021-10-29 14:58:03 +00:00
Treehugger Robot
3bf5986bb3 Merge "Remove MainNinjaFile()" 2021-10-29 13:12:00 +00:00
Jiakai Zhang
0a0a2fbea9 Add a build-time check for dexpreopting system server jars.
This change adds a Soong module that stores the paths to the compilation
artifacts of system server jars in a variable, which will then be
consumed by Make to check if the artifacts are installed. When the check
fails, it means that dexpreopting is not working for some system server
jars and needs to be fixed.

Bug: 201371822
Test: m nothing
Test: manual -
  1. Add "service-permission" to DEXPREOPT_DISABLED_MODULES (https://cs.android.com/android/platform/superproject/+/master:build/make/core/product_config.mk?q=DEXPREOPT_DISABLED_MODULES)
  2. m nothing
  3. See the error:
    Missing compilation artifacts. Dexpreopting is not working for some system server jars
    Offending entries:
    system/framework/oat/x86_64/apex@com.android.permission@javalib@service-permission.jar@classes.odex
    system/framework/oat/x86_64/apex@com.android.permission@javalib@service-permission.jar@classes.vdex
Change-Id: I4816f19668f1dae180a34adafdbfa448c97aa0db
2021-10-29 11:55:20 +00:00
Mohammad Islam
c874504988 Merge "Make car apex's library optional in PRODUCT_APEX_SYSTEM_SERVER_JARS" 2021-10-29 09:40:49 +00:00
Cole Faust
65f298ca63 Remove MainNinjaFile()
It was the same as SoongNinjaFile, except it used
shared.JoinPath instead of filepath.Join, which
doesn't matter here because the second argument
is never an absolute path.

Test: Presubmits
Change-Id: I95e64a8b8971bd168a1dc6670ae80f4d336045b5
2021-10-28 16:05:13 -07:00
Joe Onorato
83fdc94102 Add support for java streaming proto
Test: m services with ag/15937571 applied
Bug: 199427116
Change-Id: Icf262efa015900f14d9f4cabe3b8af7ec872a668
2021-10-28 15:08:34 -07:00
Treehugger Robot
c9e85ca825 Merge "Make a pass over our codebase cleaning up non-precise / non-inclusive language." 2021-10-28 20:51:17 +00:00
Colin Cross
1e7cfdea4a Make installable arch-variant for cc modules
libc's native_bridge variant needs to be installable: false, make
the installable property arch-variant.

Bug: 204136549
Test: m checkbuild
Change-Id: I8e0eaed2169e89520b07bb0326bb97ac0b4a70ac
2021-10-28 13:26:45 -07:00
Treehugger Robot
d0fb23705f Merge changes from topics "is_board_platform", "soong_config_namespace"
* changes:
  Convert is-board-platform2/is-board-platform-in-list2/is-vendor-board-qcom macros.
  Fix how the rule is displayed in the error message.
  Implement include path annotation.
  Better Soong config namespace support.
2021-10-28 19:57:09 +00:00
Joe Onorato
b4638c1ac3 Make a pass over our codebase cleaning up non-precise / non-inclusive language.
Test: treehugger
Bug: 204369779
Change-Id: I6e63cae824d323535cdb14b493edd868df47738d
2021-10-28 10:32:07 -07:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
7fa0696eb7 cc_prebuilt_library_shared converted via bp2build
Bug: b/203699063
Test: bp2build/cc_prebuilt_library_shared_test.go
Test: USE_BAZEL_ANALYSIS=1 m nothing
Test: mixed_{libc,droid}.sh
Change-Id: I4c58224e88c31507a4b285f8c55bdc066bd47232
2021-10-28 11:04:50 -04:00
Keun young Park
d64ab233e4 Make car apex's library optional in PRODUCT_APEX_SYSTEM_SERVER_JARS
- car-frameworks-service-module is added from com.android.car.framework
  apex which only exisits in car products.

Bug: 203233647
Bug: 203201931
Test: build, bootup
Change-Id: I5f13b0505c1ac4fd2fd7b6559e080c8a6fb526e0
Merged-In: I5f13b0505c1ac4fd2fd7b6559e080c8a6fb526e0
(cherry picked from commit 6e3ba9ac3b)
2021-10-28 06:12:26 +01:00
Treehugger Robot
2a524318e4 Merge "Add path of uwb NativeManager to rust allowlist." 2021-10-27 21:03:22 +00:00