This CL creates the build rules to invoke the standalone kotlin
extractor and create .kzip files for each kotlin translation unit. These
will be indexed by the internal g3 kotlin indexers
Implementation details
- Create a `kotlinKytheExtract` static rule. Its inputs will be source
.kt/.srcjar files and other verbatim args passed to kotlinc. The
extrator will serialize this information into a .kzip file. The .kzip
file should contain the necessary information to "replay" the
compilation
- Create a xref_kotlin phony rule to build all .kzip files corresponding
to kotlin translation units.
This implementation has one limitation. Since the kotlin indexers
"replay" the compilation using its own version of kotlinc-jvm, there
might be indexing issues caused by kotlinc version skew between android
builds and indexing builds. `-kotlin-home` can likely solve that, but
this CL defers that for now.
Bug: 265428637
Test: Built the phony `xref_kotlin`, and ran the indexer locally using go/kythe-git-local-run#indexing-to-local-xrefs-server
Change-Id: Ifb370f2df8aa46f71df3fe1ae7d7aa5da773cc8b
Skip combining jars into turbine-combined, combined, and withres jars
and instead collect transitive jars to use in the classpath and to
produce the final dexed jar.
This reduces the size of a `m checkbuild` in git_main by 11%, from
1300 KiB to 1154 KiB. It may also improve caching and reduce uplink
network bandwidth when building with RBE, as now the classpath inputs
to rules are themselves outputs of previous rules and so already in
the RBE CAS.
The downside is that the classpath inputs to each rule are now much
longer, increasing the Soong ninja file size 11%, from 4.6 GiB to
5.1 GiB. This could be mitigated in the future by supporting something
like depsets in the generated ninja file to reduce duplication.
Bug: 308016794
Test: TestSimple, TestKotlin, TestClasspath
Flag: build.RELEASE_USE_TRANSITIVE_JARS_IN_CLASSPATH
Change-Id: I2b7b4261375494370da70f98597c8719f1d561cf
The `m Run*RoboTests` test runner is no longer supported, remove the
code to generate the Make rules for it.
Also reduce the number of combining steps by passing the extra jars
that need to be combined into the compile step.
This relands Icf05079bf570bed7a10962cbf03459bd53c51f99 with a fix to
reorder the instrumented app classes after dependency classes.
Test: atest --host SystemUiRoboTests
Flag: EXEMPT refactor
Change-Id: I174ceb95af5557e1859eeb12a2f5fac2332975a8
collectTransitiveHeaderJarsForR8 visits direct dependencies and collects
all transitive libs and static_libs header jars. The semantics of the
collected jars are odd (it collects combined jars that contain the static
libs, but also the static libs, and it collects transitive libs dependencies
of static_libs), so these are only used to expand the --library arguments
to R8. Rename the method and all the fields it uses with a "ForR8"
suffix to avoid confusion when more transitive header jar collection is
added to support transitive classpaths.
Bug: 308016794
Test: all soong tests pass
Change-Id: I291a86fbbc2e2f088598fb309d1b992080d60941
Pass the name of Soong's checkbuild target for each module to
Make so that it can depend on it from the main checkbuild rule.
This will give better control over which files get built, allowing
checkbuild to skip the jar combining step when transitive classpath
jars are enabled. The per-module checkbuild targets are passed to
make instead of added directly as a dependency of checkbuild in order
to maintain the existing behavior of only building modules exposed
to make in checkbuild.
Also tweak the existing calls to CheckbuildFile and add
InstallFileWithoutCheckbuild to match the files that are
in the Make-based checkbuild.
Bug: 308016794
Test: m checkbuild
Change-Id: Ic5140819381d58f4d00f23a7a12447950c4cf268
The IDEInfo() methods read properties. To make those properties
configurable, we need a context to evaluate them with.
Bug: 362579941
Test: m nothing --no-skip-soong-tests
Change-Id: I26d4b7084439b3006e50b02277298f74a929e1aa
The jni_libs property of java_binary is documented to only install
the jni libraries on host. However, it was adding installable
deptag deps on those libraries, even if they were for device. This
caused soong to emit installation rules for the jni libraries, causing
them to be installed to the staging directory, but not emitting
proper information to make, so they wouldn't be included on the built
partitions.
Only install them if the module is built for host, so that they don't
erroneously appear in the staging directory.
Bug: 344695913
Test: Presubmits
Change-Id: Ic40b4fc6d7dbd51c03b08e28037be5428db49c33
java_import currently collects the "implementation and resource jar"
from each of its dependencies. This results in different resource
conflict resolution than other java modules, which collect
implementation jars and resource jars separately. Modify java_import
to match the other module types.
Flag: EXEMPT refactor
Test: all soong tests pass
Change-Id: I06134e2621582799b3a525d2c12b900b8d86d171
Similar to I40f28075ce151e4be80d6cfc7ec173dfa46f9bbf, convert
more long-lived WritablePaths to Paths.
Test: all soong tests pass
Flag: EXEMPT refactor
Change-Id: I9be448f811694fe0524fbbd7c5d4553cf69d533a
The kotlin standard library was added with a custom dependency tag,
which then required special handling later. We know up front whether
it will be treated the same as a static_libs or libs dependency, so
just reuse staticLibTag or libTag, and remove all the special handling.
Bug: 308016794
Test: TestKotlin
Flag: exempt REFACTOR
Change-Id: I2c5601faec284d88f28e13378d9cbffc24560efa
Replace the SystemModulesProvider interface with Provider
Test: all soong tests
Flag: EXEMPT refactor
Change-Id: If313580b28094d55b71f1635259bafa850ef7af5
`Jars` property in module_bp_java_deps.json should be relative to the
android build top, and not relative to the Android.bp file
Bug: 356572093
Test: m out/soong/module_bp_java_deps.json and verified that `jars` in
an arbitrary java_import module is relative to android top
Change-Id: Ifc905b14500dfa4254b27505fcc8af8f18349587
Convert JavaInfoProvider to return a *JavaInfo instead of a JavaInfo.
This will reduce copying when reading the provider, and also allows
JavaInfo to recursively contain a depset of JavaInfos from
dependencies.
Bug: 308016794
Test: go test ./java/...
Flag: EXEMPT refactor
Change-Id: Ibf6d9b797f760ad1fe815d59839839fdfad91733
Revert submission 2982300-java_api_library_non_sdk
Reason for revert: DroidMonitor: Potential culprit for b/357648959 - verifying through ABTD before revert submission. This is part of the standard investigation process, and does not mean your CL will be reverted.
Reverted changes: /q/submissionid:2982300-java_api_library_non_sdk
Change-Id: I5ef7afd9ec3e10ea99f82d02172843ad9b2cfda9
This change allows from-text stubs to be generated for all
java_sdk_library modules, unlike how it is currently limited to the
modules that contribute to the api surfaces (i.e. the SDK).
This change accomplish this by modifying the dependency of the
from-text stubs generation, so that the sdk_library generated from-text
stubs generating java_api_library modules no longer depend on the full
api surface stubs, but generate the stubs in the module level, and
combined later to generate the full api surface stubs.
This change also removes the java_api_library modules defined in
core-libraries/Android.bp, which are passed to generate the system
modules. Given that the from-text vs from-source toggle is done within
the java_sdk_library stubs level, these modules no longer need to exist.
Implementation details:
- Allow sdk_version to be specified in java_api_library modules. For
java_sdk_library-generated java_api_library modules, they inherit that
of the sdk_library module. Some java_sdk_library modules that do not
contribute to the api surface are allowed to set sdk_version to
something other than "none" or "core".
- Implement java_api_library to implement `SdkContext`. This allows
java_api_library to collect required deps from sdk_version (classpath,
bootclasspath, system modules), and pass the collected jars when
generating the stubs srcjar in metalava and compiling the stubs srcjar
in javac.
- Remove hardcoded list of sdk_library modules that are allowed to
genereate stubs from the api signature files, and allow from-text
stubs generation by default. Modules that are not able to generate
stubs from the api signature files are specified by setting the newly
introduced `Build_from_text_stubs` property to `false`.
Test: ENABLE_HIDDENAPI_FLAGS=true m
Bug: 327507877
Change-Id: Ia35d2f3cf9fae48fc8c4bd99a84ae18d7c0e7bee
Since only a single rule can write to a given WritablePath, it is
unecessary to hold on to the WritablePath once the rule has been
created. Keeping a WritablePath causes complications, as it
prevents using the input Path as the output when no modifications to
the input file are necessary. The normal pattern should be to create
a WritablePath using PathForModuleOut, build the rule that writes to
the WritablePath, and then store the WritablePath as a Path for use
as an input to any future rules.
WithoutRel previously only existed on OutputPath, which required
keeping the output path of the module as an OutputPath for as long
as possible in order to call WithoutRel on it at the end of the module.
Add WithoutRel to Path, make it always return a Path type, and implement
it on all the types that implement Path by using a helper in basePath.
Replace long-lived WritablePaths with Paths.
Test: all soong tests pass
Flag: EXEMPT refactor
Change-Id: I40f28075ce151e4be80d6cfc7ec173dfa46f9bbf
Adds a filegroup "metalava-config-files" whose contents are passed to
Metalava using the `--config-file` option.
Bug: 354699349
Test: m checkapi
Change-Id: I1e246517c2ef678e41214c975aecee97f9faca67
`Dependencies` in module_bp_java_deps.json currently only contains the
`libs` and `static_libs` of the corresponding module. This is only a
subset of the dependencies which are required to compile the java
module. Most notably, the SDK deps (determined using `sdk_version`) are
missing.
This CL adds all the compile dependencies to the json file. Implementation
details:
1. Add a `compileDepNames` to java.Module in base.go. Almost all the
module types in java package are composed with the java.Module
struct.
2. Populate this property in java.Module.collectDeps. This function is
used by java.Module.compile to determine the classpath used for
compilation.
3. Return `compileDepNames` in java.Module.CompilerDeps.
Since `compileDepNames` is populated in `GenerateAndroidBuildActions`,
update existing unit tests that check for correctness of IDEInfo.Deps
Bug: 353811607
Test: m nothing --no-skip-soong-tests
Test: cat out/soong/module_bp_java_deps.json | jq '."DebuggingRestrictionController".dependencies'
[
"android_system_stubs_current", <-- missing previously
"core-lambda-stubs", <-- missing previously
"core-public-stubs-system-modules", <-- missing previously
"com.google.android.material_material",
...
...
]
Change-Id: I166c0eaf6e87ca7adc98bb4e21447cf82612d103
Add an IsAddingDependency method to IncomingTransitionContext that
returns true if IncomingTransition is called after the transition
has already won while adding a new dependency. This will be used
as part of the transition mutators to support an apex use case where
incoming dependencies during the initial apex mutator need to be
rewritten onto the platform variant for modules that don't support
the apex, but a later call to OtherModuleDependencyVariantExists
must not rewrite the requested apex variation onto the platform
variant.
This should be used sparingly, all uses will have to be removed in
order to support creating variants on demand.
Bug: 319288033
Test: TestIsAddingDependency
Flag: EXEMPT refactor
Change-Id: Ib8e419d35ff8f7cbff9667c1cd40d05ccfacab8b
In the context of incremental soong, the output files
inter-module-communication will be through OutputFilesProvider.
The OutputFileProducer interface will be deprecated.
Test: CI
Bug: 339477385
Change-Id: I3c9f0b766325dd490bc903dc65259c3953d34606
In the context of incremental soong, the output files
inter-module-communication will be through OutputFilesProvider.
The OutputFileProducer interface will be deprecated.
Test: CI
Bug: 339477385
Change-Id: I51ccf59f97cb8a790642cb4463c479feaf18058a
Specifying min_sdk_version ensures backwards compatibility. There are
some existing violations, and those have been baseline'd using an
allowlist.
Bug: 156476221
Test: m nothing
Change-Id: I55098206da8ec5cfa0a9f5e7b41c9b8dedbdcbd2
So that users can use select statements with it.
Fixes: 347605145
Bug: 342006386
Test: m nothing --no-skip-soong-tests
Change-Id: Ica0ca6d1725b000b3748c0293e5a9f9b38ed87f9
This change modifies the contents of the generated Android.bp files so
that when generating a snapshot on a older platform, the "exportable"
modules are removed from the bp files, as the "exportable" modules are
first introduced in V and do not exist in older platforms.
Bug: 345162614
Test: ABTD
Change-Id: I2dba51b98deec7805bd796647a66981f237c55a9
Revert submission 3060229-ron-tmc-symlinks
Reason for revert: Droidmonitor created revert due to b/344045516
Reverted changes: /q/submissionid:3060229-ron-tmc-symlinks
Change-Id: If5045366677163560cdae95c0ab74256b4b49b9a
Now the derived modules will have symlink's to base's testcase dir
rather than copyfile files from base's intermediates dir.
I also removed storing the "base" module as dependency and only use
data from the provider in GenerateAndroidBuildActions and
AndroidMkEntries.
I did have to add two fields to the provider for this.
To ensure the symlinks always resolve to a physical file, I also added
goals such that building `derived-target` will also build `build-target`
Create new Make variable: LOCAL_SOONG_INSTALLED_COMPATIBILITY_SUPPORT_FILES
for tasks/general-tests.mk,device-tests.mk to read.
This allows "support" files that are installed with soong rules rather
than make rules to still end up in the zips.
Sample dir structure:
% tree -ls testcases aosp_cf_x86_64_phone[4:31:54]/0
[ 4096] testcases
├── [ 4096] FrameworksServicesTests
│ ├── [ 4096] data
│ │ └── [ 851] broken_shortcut.xml
│ ├── [ 2800] FrameworksServicesTests.config
│ ├── [ 12695] JobTestApp.apk
│ ├── [ 8599] MediaButtonReceiverHolderTestHelperApp.apk
│ ├── [ 16791] SimpleServiceTestApp1.apk
│ ├── [ 16791] SimpleServiceTestApp2.apk
│ ├── [ 16791] SimpleServiceTestApp3.apk
│ ├── [ 1017763] SuspendTestApp.apk
│ └── [ 4096] x86_64
│ └── [ 79827767] FrameworksServicesTests.apk
└── [ 4096] FrameworksServicesTests_contentprotection
├── [ 4096] data
│ └── [ 54] broken_shortcut.xml -> ../../FrameworksServicesTests/data/broken_shortcut.xml
├── [ 3005] FrameworksServicesTests_contentprotection.config
├── [ 41] JobTestApp.apk -> ../FrameworksServicesTests/JobTestApp.apk
├── [ 69] MediaButtonReceiverHolderTestHelperApp.apk -> ../FrameworksServicesTests/MediaButtonReceiverHolderTestHelperApp.apk
├── [ 52] SimpleServiceTestApp1.apk -> ../FrameworksServicesTests/SimpleServiceTestApp1.apk
├── [ 52] SimpleServiceTestApp2.apk -> ../FrameworksServicesTests/SimpleServiceTestApp2.apk
├── [ 52] SimpleServiceTestApp3.apk -> ../FrameworksServicesTests/SimpleServiceTestApp3.apk
├── [ 45] SuspendTestApp.apk -> ../FrameworksServicesTests/SuspendTestApp.apk
├── [ 36] test_module_config.manifest
└── [ 4096] x86_64
├── [ 64] FrameworksServicesTests.apk -> ../../FrameworksServicesTests/x86_64/FrameworksServicesTests.apk
└── [ 36] UNUSED-FrameworksServicesTests.apk
Test: m clean && atest FrameworksServicesTests_contentprotection
Test: m clean && atest CtsDevicePolicyManagerTestCases_ParentProfileApiDisabled
Bug: b/332320956
Change-Id: I8466f253fa559bc74cef4533edf263650e96bbfb
Currently, dexpreopt supports profile guided optimization. This does not
work well with r8/d8 optimization, since the checked-in profile will not
match the dex signatures after r8/d8 has optimized the dex code.
This CL introduces a new property `dex_preopt.enable_profile_rewrting`.
If set, the checked-in profile will passed as `input` to r8 via
`--art-profile <input> <output>`. The <output> from the previous command
will be used as the profile for dexpreopt.
Test: m nothing --no-skip-soong-tests
Test: m CredentialManager with https://ag.corp.google.com/27448930
and obfuscation turned on
Test: nm -U symbol.odex # contains obfuscated methods
Bug: 335418838
Change-Id: I53beed9ed76f013262f1c503de0f2b74997c2a7f
* changes:
Revert^2 "Refactor how jni_libs dependencies are added"
Revert^2 "Add make java.dependencyTag installable or not"
Revert "Revert "APK-in-APEX should set use_embedded_native_libs:..."
Revert^2 "Install jni symlinks in Soong"
b7646e4d4f
This is a relanding of I0930cb1ebb8ca8a6efd64b1ce2cdfd1c47fe19ef plus
some forward fix described below:
Export non-embedded JNI lib names via LOCAL_REQUIRED_MODULES
The non-embedded JNI libs are installed as the dependencies of the APK.
However, that dependency is not revealed to the Make world and as a
result, the JNI libs are dropped from the file_list.txt file which Make
uses to filter files to include in the image file.
Adding the lib names to LOCAL_REQUIRED_MODULES fixes it.
Bug: 341335305
Bug: 330276359
Test: m out/target/product/vsoc_x86_64_only/obj/PACKAGING/system_intermediates/file_list.txt
and check if libcarservicejni.so is there
Test: go test ./... under soong/java
Change-Id: If915a05909129c92fab7a6cbbd0c4c55f5ced598
This replaces the global override of min_sdk_version (via
`APEX_GLOBAL_MIN_SDK_VERSION_OVERRIDE`) with an min_sdk_version override
that can be set by each individual soong override_app.
The use case for this are go apps which are only installed in T and
above, even though the base AOSP apexes might be installable on < T
devices.
Test: added a unit test
Bug: 295311875
Change-Id: Ie2e738a6786bb24417c675617f7c78358017c96c
Revert submission 3088867
Reason for revert: On cf_auto build, we are seeing a symlink of jni lib pointing to lib64/libcarservicejni.so, which does not exist on the device.
Reverted changes: /q/submissionid:3088867
Bug: b/341335305
Change-Id: Ie4ff26fe47e1a0ab5e97b185c61cb94473cc5fb8
Revert submission 3088867
Reason for revert: On cf_auto build, we are seeing a symlink of jni lib pointing to lib64/libcarservicejni.so, which does not exist on the device.
Reverted changes: /q/submissionid:3088867
Bug: b/341335305
Change-Id: I60052dbb004df58fdfaf93be6089769ab2a670fc
java.dependencyTag now has a new field installable which contols whether
the dependency should be added as the install dep of the app or not.
This allows us to model all the dependency tags for Java using the same
struct.
Bug: N/A
Test: m
Change-Id: I9b1c7809a0bde3f15579aefd9bb34d828764b84b
The installation of the symlink
(<partition>/app/MyApp/lib/<arch>/libfoo.so) and its target
(/system/lib64/libfoo.so) are now done int Soong.
I gave up the idea of always embedding jni libs to apps, due to a
hard-to-fix regression in storage usage. Specifically, consider this
case.
app --(jni_lib)--> libfoo
--(jni_lib)--> libbar
libfoo --(shared_lib)--> libbar
Ideally, with the embedding idea, both libfoo and libbar should be
embedded into the app and there should be no libfoo or libbar outside of
the apk, unless there's no dependency to any of them from outside of the
apk.
However, the previous implementation installed libbar to /system/lib64,
leading two copies of the lib; one in /system/lib64, the other in the
apk.
This was happening because libbar was also considered as a transitive
dep of libfoo, and therefore a dependency to its installation path is
added to the apk. The problem here is that the app doesn't know that
libfoo depends on libbar. We in theory can write a very delicate
dependency traverse routine which filters libbar out of the transitive
deps, but that looked too complicated.
Bug: 339923078
Bug: 330276359
Test: Build and watch any bloatbuster warning
Test: m aosp_cf_system_x86_64
The following three files are found
* system/lib64/libnfc_nci_jni.so
* system/etc/libnfc-nci.conf
* system/priv-app/NfcNci/lib/arm64/libnfc_nci_jni.so
Change-Id: I0930cb1ebb8ca8a6efd64b1ce2cdfd1c47fe19ef
