Commit Graph

188 Commits

Author SHA1 Message Date
Cole Faust
7351875acb Merge "Add HasMutatorFinished" into main 2024-09-12 00:02:10 +00:00
Zi Wang
6ad2e30de3 Merge "Add framework for using provider on AndroidMKEntries" into main 2024-09-11 23:02:33 +00:00
Cole Faust
4e2bf9fb2d Add HasMutatorFinished
To enforce that selects are only evaluated after a certain point
in a followup cl.

Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: Ib215fedb904aa2e5f4d65cfd26a23f527eb4983e
2024-09-11 14:48:50 -07:00
Cole Faust
e8a8783154 Rename ConfigAndErrorContext to ConfigurableEvaluatorContext
I'm going to be adding some methods to this interface, give it
a name based on how it's going to be used, not based on what methods
it contains.

Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: I9bba04ba756c4dbe00625e2d04af81e78a11cae9
2024-09-11 11:35:46 -07:00
mrziwang
1842097928 Add framework for using provider on AndroidMKEntries
In the context of incremental soong, the inter-module communication
through AndroidMkEntries and AndroidMkData should be changed to
through the real provider.

This change adds the framework for the provider(AndroidMkInfoProvider).
A module will set its AndroidMkInfoProvider, which includes what's
in its AndroidMkEntries method, to be used in translateAndroidMkModule.

This change also changes module type vbmeta to use
AndroidMKInfoProvider.

The next step is to replace the AndroidMkEntries methods of all the
other module types to set the AndroidMkInfoProvider.

Test: m with tests and CI
Bug: 357907638
Change-Id: Icfd1363f97d31548bb78c6615e0f7076b22dcbe4
2024-09-11 10:38:41 -07:00
Neill Kapron
f1d63b8b6d Merge "SOONG: BPF: add libbpf_prog soong module" into main 2024-09-11 15:41:14 +00:00
Treehugger Robot
4301e29a02 Merge "Revert "Revert "Support BUILD_FRAMEWORK_COMPATIBILITY_MATRIX for..."" into main 2024-09-06 00:48:26 +00:00
Neill Kapron
41efab77d0 SOONG: BPF: add libbpf_prog soong module
This commit adds the libbpf_prog soong module which is heavily based on
the bpf soong module. Due to issues encountered and concerns around
compatibility and testability of mainlined bpf modules, support for
libbpf is added as a separate module.

Specifically, when changing the bpf module from
InitAndroidArchModule(... android.MultilibCommon) to MultilibFirst, the
soong apex bpfs functionality breaks. We have not identified a good
mechanism to fix this, and as we would then be creating architecture
specific bpf program binarys, additional testing and infrastructure
would be required.

It should be noted that libbpf_prog should not be used for features
which target 32bit kernel architectures at this point, as we do not
have a mechanism to identify the kernel arch at build time. I.E. 32bit
userspace could be on either 32bit or 64bit kernel arch, and we require
arch specific includes (vmlinux.h). Therefore, we default to 32bit
userspace on 64bit kernel to match the precedence set with the legacy
bpf programs.

Example use:
libbpf_prog {
    name: "xyz.o",
    srcs: ["xyz.c"],
    arch: {
        arm: {
            local_include_dirs: ["vmlinux/arm64"],
        },
        arm64: {
            local_include_dirs: ["vmlinux/arm64"],
        },
        x86: {
            local_include_dirs: ["vmlinux/x86_64"],
        },
        x86_64: {
            local_include_dirs: ["vmlinux/x86_64"],
        },
    },
}

Bug: 359646531
Test: manual
Change-Id: Ie58515d70abee061470cf4bb803228e00d496ac3
Signed-off-by: Neill Kapron <nkapron@google.com>
2024-09-05 22:37:19 +00:00
Bill Yang
3b3aac0088 Revert "Revert "Support BUILD_FRAMEWORK_COMPATIBILITY_MATRIX for..."
Revert submission 3235656-revert-3195743-compatibility_matrix-KLJOLWYIMW

Reason for revert: The root cause of the break is already be merged. Related changes: ag/29139095, ag/29139608

Reverted changes: /q/submissionid:3235656-revert-3195743-compatibility_matrix-KLJOLWYIMW

Change-Id: Ie23bdd343bebbd077553dc25f0bfd13539daf57a
2024-09-05 09:22:10 +00:00
Colin Cross
a6182ab2fa Move checkbuild targets of Soong modules into Soong
Pass the name of Soong's checkbuild target for each module to
Make so that it can depend on it from the main checkbuild rule.
This will give better control over which files get built, allowing
checkbuild to skip the jar combining step when transitive classpath
jars are enabled.  The per-module checkbuild targets are passed to
make instead of added directly as a dependency of checkbuild in order
to maintain the existing behavior of only building modules exposed
to make in checkbuild.

Also tweak the existing calls to CheckbuildFile and add
InstallFileWithoutCheckbuild to match the files that are
in the Make-based checkbuild.

Bug: 308016794
Test: m checkbuild
Change-Id: Ic5140819381d58f4d00f23a7a12447950c4cf268
2024-09-03 14:52:19 -07:00
Yu Liu
82a6d1464d Remove 6 install related fields from ModuleBase.
Bug: 358425833
Test: Manually verified all the generated ninja and mk files and CI.
Change-Id: If2c3417781953a1eef2b07a241424157fd73abff
2024-08-27 19:03:35 +00:00
Yu Liu
4b98396c64 Merge "Remove licenseMetadataFile and installFiles from ModuleBase." into main 2024-08-27 15:54:00 +00:00
Pindar Yang
1282d27291 Merge "Revert "Support BUILD_FRAMEWORK_COMPATIBILITY_MATRIX for soong"" into main 2024-08-27 03:25:23 +00:00
Pindar Yang
1558cc68ec Revert "Support BUILD_FRAMEWORK_COMPATIBILITY_MATRIX for soong"
Revert submission 3195743-compatibility_matrix

Reason for revert: Build Break, Bug: 361227456

Reverted changes: /q/submissionid:3195743-compatibility_matrix

Change-Id: Ib89cdc64eb816ca0df872473b1e9a9dfdb08ceee
2024-08-27 03:22:41 +00:00
Yu Liu
ec81054548 Remove licenseMetadataFile and installFiles from ModuleBase.
Bug: 358425833
Test: Manually verified all the generated ninja and mk files and CI.
Change-Id: I447ec74b7f9c3b8c686ed41511e2051f40ac88f8
2024-08-27 00:16:22 +00:00
Kiyoung Kim
04b64fc280 Append vintf_fragment_modules as required for AndroidMk
Current vintf_fragment_modules are not exposed to Kati, so modules from
the property is not being installed to the image while Kati collects
files to be installed from their dependencies. This change marks modules
from vintf_fragment_modules as required, so their dependencies can be
exposed to the Kati.

Bug: 322089980
Test: aosp cf build succeeded
Change-Id: I66fc8f9ee8c0b797a0c200b7070b83ec9bb0f616
2024-08-19 11:25:30 +09:00
Yu Liu
d46e5ae670 Remove more internal fields from ModuleBase.
Bug: 358425833
Test: Manual compare the generated ninja and mk files and CI.
Change-Id: I4edd0dc646ac035594f47cd9044cc4eb5f710e64
2024-08-16 00:20:27 +00:00
Yu Liu
e89b6dd497 Merge "Remove installFiles from ModuleBase." into main 2024-08-15 17:23:59 +00:00
Treehugger Robot
4483c0e7c0 Merge "Support BUILD_FRAMEWORK_COMPATIBILITY_MATRIX for soong" into main 2024-08-15 09:07:38 +00:00
Yu Liu
ddc2833b7f Remove installFiles from ModuleBase.
This is to limit the direct accesses to the internal fields of a
module in order to better support incremental caching. To access
the install files data from singleton context or other modules'
context use providers thru the provided wrapper; to access it
from the same module inside GenerateBuildActions use ctx which is
short-lived only inside this method.

Bug: 358425833
Test: CI
Change-Id: I337b07a2ef95fb2a898ac2f9277160a3f76a603c
2024-08-14 22:56:30 +00:00
Yu Liu
663e4508dc Merge SingletonProviderContext with OtherModuleProviderContext
Bug: 358425833
Test: CI
Change-Id: I8e3f40dc3cfc5337008b419801f8e6bf2d48e8b2
2024-08-12 22:50:19 +00:00
yangbill
66e555c8c7 Support BUILD_FRAMEWORK_COMPATIBILITY_MATRIX for soong
For building framework_compatibility_matrix.device.xml it will use
matrix file path from DEVICE_FRAMEWORK_COMPATIBILITY_MATRIX_FILE, and
for product_compatibility_matrix.xml it will use matrix file path from
DEVICE_PRODUCT_COMPATIBILITY_MATRIX_FILE.
This CL import those value from make to soong and will be used in
hardware/interfaces/compatibility_matrices/.

Also add vintf.vintfCompatibilityMatrixRule to the white list for using
custom make rule.

Bug: 352234038
Bug: 344445679
Test: m product_compatibility_matrix.xml
Change-Id: Ib7042ee2d771bd47a12f0565014b4006da678ddb
2024-08-02 08:38:41 +00:00
Cole Faust
43ddd08bd2 Make required a configurable property
So that users can use select statements with it.

Fixes: 347605145
Bug: 342006386
Test: m nothing --no-skip-soong-tests
Change-Id: Ica0ca6d1725b000b3748c0293e5a9f9b38ed87f9
2024-06-17 12:32:40 -07:00
Cole Faust
3f01580c04 Merge "Make the enabled property configurable" into main 2024-05-07 17:49:10 +00:00
Cole Faust
a963b94cde Make the enabled property configurable
This allows using select statements with it.

Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: I6f3efaaa3d82505e38a91ee4ba0e18e404360191
Merged-In: If355d24506e3f117d27b21442a6c02bca3402dc7
2024-05-02 15:41:24 -07:00
Jiyong Park
3f627e661a Use no_full_install: true instead of installable: false
So far, we have used `instalable: false` to avoid collision with the
other modules that are installed to the same path. A typical example was
<foo> and <foo>.microdroid. The latter is a modified version of the
former for the inclusion of the microdroid image. They however both have
the same instalation path (ex: system/bin) and stem (ex: foo) so that we
can reference them using the same path regardless of whether we are in
Android or microdroid.

However, the use of `installable: false` for the purpose is actually
incorrect, because `installable: false` also means, obviously, "this
module shouldn't be installed". The only reason this incorrect way has
worked is simply because packaging modules (ex: android_filesystem)
didn't respect the property when gathering the modules.

As packaging modules are now fixed to respect `installable: false`, we
need a correct way of avoiding the collision. `no_full_install: true` is
it.

If a module has this property set to true, it is never installed to the
full instal path like out/target/product/<partition>/... It can be
installed only via packaging modules.

Bug: 338160898
Test: m
Change-Id: Iee9be674951d0bf3d5e26432fcbae9afebb6007b
2024-05-01 22:43:45 +09:00
LaMont Jones
b509938e4f Propagate AconfigFiles in ModuleBase.
Aconfig file dependencies are module-independent, and properly part of
ModuleBase.

Bug: b/308625757
Test: manual

Change-Id: I38c5907d1671cc69bb198345201316ae781fdc9f
2024-01-24 23:45:12 +00:00
Tongbo Liu
c5f7b96ab5 Enable building whole MCTS and partial MCTS test suites.
Bug: 318608673
Test: m mcts
Change-Id: I2af74e319c40b0e36d8a71e1da3c6934f66a51be
2024-01-04 09:04:18 +00:00
Colin Cross
138d37dd6c Merge "Support generating module_info.json in Soong" into main 2024-01-03 17:55:47 +00:00
Nelson Li
f3c706868a Enable Soong to correctly add dependencies to .PHONY targets.
Previously, Soong's phony simply added "required" or "host_required"
to LOCAL_REQUIRED_MODULES and `include BUILD_PHONY_PACKAGE` without
correctly adding them to the dependency list.

This change add a new `phony_rule` module type to Soong that acts like
`.PHONY` instead of `BUILD_PHONY_PACKAGE`

Bug: 316838256
Bug: 309730110
Test: 1. Use `phony_rule` and adding "phony_deps: [dep_modules]" to
         Android.bp.
      2. make target and check the dep_modules be built.
Change-Id: I3e9e9ac26ecc456668f1b6baf5c08f9c9139b3b2
2024-01-03 02:01:36 +00:00
Colin Cross
d6fd013394 Support generating module_info.json in Soong
Generate module_info.json for some Soong modules in Soong in order to
pass fewer properties to Kati, which can prevent Kati reanalysis when
some Android.bp changes are made.

Soong modules can export a ModuleInfoJSONProvider containing the
data that should be included in module-info.json.  During the androidmk
singleton the providers are collected and written to a single JSON
file.  Make then merges the Soong modules into its own modules.

For now, to keep the result as similar as possible to the
module-info.json currently being generated by Make, only modules that
are exported to Make are written to the Soong module-info.json.

Bug: 309006256
Test: Compare module-info.json
Change-Id: I996520eb48e04743d43ac11c9aba0f3ada7745de
2024-01-02 16:03:43 -08:00
Colin Cross
5a37718c95 Convert ModuleProvder to generic providers API
Convert all of the callers of ModuleProvider/ModuleHasProvider to use the
type-safe android.SingletonModuleProvider API.

Bug: 316410648
Test: builds
Change-Id: I6f11638546b64749e451cebbf33140248dc1d193
2023-12-14 16:12:22 -08:00
Colin Cross
313aa5475f Convert OtherModuleProvider to generic providers API
Convert all of the callers of OtherModuleProvider/OtherModuleHasProvider
to use the type-safe android.OtherModuleProvider API.

Bug: 316410648
Test: builds
Change-Id: Id77f514d68761a262d9ea830a601dbed804bbbe5
2023-12-14 16:12:22 -08:00
Colin Cross
3c0a83d19f Use generics for providers API
Using generics for the providers API allows a type to be associated
with a ProviderKey, resulting in a type-safe API without that doesn't
require runtime type assertions by every caller.

Unfortunately, Go does not allow generic types in methods, only in
functions [1].  This prevents a type-safe API on ModuleContext, and
requires moving the API to be functions that take a ModuleContext as
a parameter.

This CL creates the new API, but doesn't convert all of the callers.

[1] https://go.googlesource.com/proposal/+/refs/heads/master/design/43651-type-parameters.md#no-parameterized-methods)

Bug: 316410648
Test: builds
Change-Id: I3e30d68b966b730efd968166a38a25cc144bd6de
2023-12-14 16:12:20 -08:00
Colin Cross
5c1d5fb21b Move test data installation to Soong
To generate module-info.json in Soong for b/309006256 Soong needs to
know the test data paths. Moving test data installation into Soong will
also help later for test suite packaging.

Add ModuleContext.InstallTestData that installs the files listed in a
[]DataPath alongside the test.  The files will also be passed to Make
to allow it to continue packaging them into the test suites for now.

Update the module types that are producing LOCAL_TEST_DATA entries
in their Android.mk files to go through InstallTestData instead.

Bug: 311428265
Test: atest --host toybox-gtests --test-timeout=120000
Change-Id: Ia8b964f86e584ea464667fd86a48d754d118bead
2023-11-30 13:38:49 -08:00
Colin Cross
a16bd6b714 Don't write license data to Android-${TARGET_PRODUCT}.mk
All the necessary license data is already propagated through the license
metadata file, and the entries in Android-${TARGET_PRODUCT}.mk are not
used.  Remove them, which should both reduce the size and parsing cost
of the Android-${TARGET_PRODUCT}.mk file and also removes a variable
that changes when dependencies are added or removed, which will
eventually allow running kati less often.

Bug: 309006256
Test: No change to out/Android-aosp_cf_x86_64_phone.ninja
Change-Id: Idfeb6fd10d77497ca274f8e74fe8e994ce1cafe6
2023-11-21 15:27:44 -08:00
LaMont Jones
0c10e4dcc0 Parallelize singleton execution
Bug: 281536768
Test: manual, presubmits
Change-Id: I57fdc76ba6b277e88e196b506af87127a530fd37
2023-05-19 20:31:32 +00:00
Jingwen Chen
8ac7d7d01d Revert "Export non-apex variants of modules to make"
This reverts commit 502da3987a.

Reason for revert: b/274195633

```
In file included from out/soong/installs-aosp_cf_x86_pasan.mk:134984:
In file included from out/soong/Android-aosp_cf_x86_pasan.mk:981696:
In file included from build/make/core/soong_cc_rust_prebuilt.mk:76:
build/make/core/base_rules.mk:342: error: packages/modules/Uwb/service/uci/jni: MODULE.TARGET.SHARED_LIBRARIES.libuwb_uci_jni_rust already defined by packages/modules/Uwb/service/uci/jni.
```

Change-Id: Ic1ea6969e54c23a7d126eb0fb47ab6f2e44ee965
2023-03-20 11:05:16 +00:00
Cole Faust
502da3987a Export non-apex variants of modules to make
Currently, non-apex variants of modules that are in apexes are not
exported to make unless they're apex_available to the platform. This
means that you can't `m` those modules directly.

However, there is a workaround in the apex androidmk implementation that
emits make rules for the removed modules, but just redirects them to
build the apex itself. We want to remove that, but one of the problems
with doing so is that you can no longer `m` many modules afterwards.

To fix that, unhide the apex's dependencies from make. To ensure they're
not installed, call SkipInstall() on them, and update SkipInstall() to
be more strict by setting `LOCAL_UNINSTALLABLE_MODULE := true`.

Bug: 254205429
Test: Presubmits
Change-Id: Ib971981559f3b642ce6be8890679e994e1b44be0
2023-03-16 11:13:23 -07:00
Iván Budnik
295da16b95 Revert "Export non-apex variants of modules to make"
Revert submission 2462194-androidmk_for_apex_dependencies

Reason for revert: DroidMonitor-triggered revert due to breakage https://android-build.googleplex.com/builds/quarterdeck?branch=aosp-master&target=full-eng&lkgb=9723453&lkbb=9724944&fkbb=9723508, bug b/272663333.

Reverted changes: /q/submissionid:2462194-androidmk_for_apex_dependencies

Bug: 272663333
Change-Id: I847fa910a8d54de19954f56fe92a80a9bcd0ee18
2023-03-10 16:11:26 +00:00
Cole Faust
ad222f271a Export non-apex variants of modules to make
Currently, non-apex variants of modules that are in apexes are not
exported to make unless they're apex_available to the platform. This
means that you can't `m` those modules directly.

However, there is a workaround in the apex androidmk implementation that
emits make rules for the removed modules, but just redirects them to
build the apex itself. We want to remove that, but one of the problems
with doing so is that you can no longer `m` many modules afterwards.

To fix that, unhide the apex's dependencies from make. To ensure they're
not installed, call SkipInstall() on them, and update SkipInstall() to
be more strict by setting `LOCAL_UNINSTALLABLE_MODULE := true`.

Bug: 254205429
Test: Presubmits
Change-Id: Ib094feb2c437ad50d8319c58caa997759e7ce32f
2023-03-09 11:20:10 -08:00
Wei Li
598f92d704 Export Soong module type to LOCAL_SOONG_MODULE_TYPE to Make for SBOM generation.
Bug: 266726655
Test: CI

Change-Id: If5b1a77930a591b6061ca2749c9c5ad29b4491fb
2023-03-06 22:51:03 +00:00
Cole Faust
39aabe94b9 Add information about what module produced a certain androidmk entry
For easier debugging.

Bug: 254205429
Test: Presubmits
Change-Id: I54fc53dd3b4f32eb3e4458386aa6caa5b76dbb5b
2023-02-23 16:57:43 -08:00
Sasha Smundak
dcb6129269 Streamline AndroidMk generation
Add AndroidMkEmitAssignList to emit a line to assign the items from
the given list of string arrays.

Test: treehugger
Change-Id: Id5acbef38ea4e91349bd2461f226db352d4b8123
2022-12-12 18:06:49 -08:00
Sasha Smundak
5c4729df93 Show module type in a module section of the Android-TARGET.mk
The first line of each section will be
```
include $(CLEAR_VARS)  # <module type>
...
```

Bug: 257037252
Test: treehugger
Change-Id: Iba586155b682fe4e1e5817d8397eda8f9d9c8789
2022-12-01 21:12:11 -08:00
Bob Badour
4660a9824c Integrate gen_notice with Make for dist.
Most exempt module types have no output. Support exempt module types
with output by requiring they must be 0p and have no license metadata.

Disallow `licenses: []` property on gen_notice.

Test: m cts dist

Change-Id: Ic992bd6420fa6898495866eac43495002ef4b6c8
Merged-in: Ic992bd6420fa6898495866eac43495002ef4b6c8
2022-09-14 14:21:44 -07:00
Bob Badour
0cc5facf79 Remove obsolete notice property.
Bug: 192434786

Test: m droid dist

Change-Id: I0f384ae895cebab9732448b82db600c67eb8f711
2022-07-29 17:54:40 +00:00
Bob Badour
9f847581b7 Keep Notice property from erasing license texts.
Remove "default to file in directory". Use explicit applicable license.

Bug: 192434786

Test: m droid dist

Change-Id: Id3d845b2073f151a8c0bad874e7ab005771bdfcb
2022-07-14 10:40:01 -07:00
Colin Cross
9a027be6bc Don't pass host cross linux musl modules to Make
Make doesn't understand host cross linux musl modules, don't pass
them to make.  Continue passing them to make when USE_HOST_MUSL=true
is set.

Bug: 236052820
Test: builds with linux musl arm64 host cross configured
Change-Id: Id8b90ca0fa698fdf658156b458cc385387768414
2022-06-28 15:16:15 -07:00
Bob Badour
5180438478 Make sure dist files have license metadata.
Bug: 151177513
Bug: 210912771

Test: m droid dist reportmissinglicenses

Change-Id: I0c85f6c49a3e9d9bb3219ed6ddfb939d90f80656
2022-06-01 21:07:25 -07:00