Some Android.bp modules have `enabled: false` but only use a product
variable such as `source_build` to enable the module. Currently b2build
does not handle this case at all. This commit adds the functionality
to support this use case.
Also, remove `__enabled` suffix in ProductVariable SelectKey.
Bug: 210546943
Test: go test ./bp2build
Topic: use_enabled_flag_product_variable_config
Change-Id: I459c17a84c172df010666391066bf4d11d19253e
ModuleBase.generalProperties served the pupose of being a subset
of ModuleBase.customizableProperties. But now with the removal of
the latter, the former is simply a redirection to Module.GetProperties()
Bug: 206961391
Test: m nothing and diff the ninja files
Change-Id: I6dd8b7ba74eb5e7ffb61029b0f9129eec2ccfdaf
- customizableProperties was being used to access all properties,
i.e. basically `GetProperties()`
- generalProperties was a subset of properties,
i.e. `initArchModule()` would only add new arch-specific properties
to `customizableProperties` (and `archProperties`) but not `generalProperties`
This PR sets generalProperties to contain all properties,
which then makes customizableProperties redundant.
And in arch.go, any iteration is done via `archProperties` instead.
Bug: 206961391
Test: m nothing and diff the ninja files
Change-Id: Ieb72b31c0a447fcbf34c92a04f002d7b9d777a86
Also fix some bugs pertaining to configurable attribute handling of bool
attributes and label sttributes, so that they may support values across
multiple different axes at the same time.
Test: unit tests for bp2build
Test: mixed_droid
Change-Id: I411efcfddf02d55dbc0775962068a11348a8bb2c
base.module = module is redundant in InitDefaultsModule() becase of the preceding initAndroidModuleBase()
minor reformat of initArchModel()
Test: m nothing
Bug: n/a
Change-Id: I16d5a400a33098a81ffbf6c81eba5b13db6debf1
Soong has enough information to build the license metadata files
without resorting to the fixups required in Make.
Bug: 207445310
Test: m checkbuild
Change-Id: I8e74108376162b8fdb87ba098ebe94350aa1f7c4
* The obj-* targets only call clang or clang++ to compile
C/C++ files, like the tidy-* targets only call clang-tidy.
* Add (obj|tidy)-dir_os and module_os-(obj|tidy) phony targets
to include only targets of the specified OS.
* Add (obj|tidy)-dir_os_subset and module_os_subset-(obj|tidy) phony targets
to include only a subset of that OS target group.
* Most clang-tidy runs produce the same warnings for all variants
of an OS. The *_subset targets reduce runs over multiple variants.
* The obj-*subset targets are useful for a quick test to compile
at least one variant of every C/C++ file for each OS.
* The (obj|tidy)-soong phony target is changed to include only
(obj|tidy)-top_level_directory targets instead of ALL
module-(obj|tidy) targets in all subdirectories.
* For aosp_arm64-eng make tidy-* targets; count NINJA commands:
tidy-soong 381K
tidy-soong_subset 106K
tidy-soong_android 294K
tidy-soong_android_subset 73K
tidy-soong_linux_glibc 84K
tidy-soong_windows 12K
tidy-bionic 12K
tidy-bionic_subset 2.4K
tidy-bionic_android 11.5K
tidy-bionic_android_subset 2.1K
tidy-bionic_linux_glibc 1K
tidy-device_android 7K
tidy-device_android_subset 5K
tidy-hardware_android 3K
tidy-hardware_android_subset 9K
libfec_rs-tidy 33
libfec_rs_subset-tidy 9
libfec_rs_android-tidy 18
libfec_rs_android_subset-tidy 3
* For aosp_arm64-eng make obj-* targets; count NINJA commands:
obj-soong 399K
obj-soong_subset 112K
obj-soong_android 305K
obj-soong_android_subset 75K
obj-soong_linux_glibc 90K
obj-soong_linux_glibc_subset 38K
obj-soong_windows 12K
obj-bionic 15K
obj-bionic_subset 3K
obj-bionic_android 14K
obj-bionic_android_subset 2.3K
obj-bionic_linux_glibc 1.3K
obj-system-core 6K
obj-system-core_subset 3K
obj-frameworks-base 16K
obj-frameworks-base_subset 6K
libfec_rs-obj 33
libfec_rs_subset-obj 9
libfec_rs_android-obj 18
libfec_rs_android_subset-obj 3
Test: NINJA_ARGS="-n" WITH_TIDY=1 make some_obj_tidy_target;
compare output commands from various phony targets
Bug: 199169329
Bug: 202769240
Change-Id: I186c45dc07c5884888c1063dfc09cf212ffb0ebf
Enable InstallBypassMake for all module types by switching its default
to true.
Bug: 204136549
Test: m checkbuild
Change-Id: Ied082995e74d71dee6aecea99689ac3f63600d3b
Add support for installing extra files from a zip file when installing
a primary file, and use it to support installing android_app_set modules,
which install a primary APK and then an unknown set of additional splits
APKs.
Test: app_set_test.go
Test: install test android_app_set
Bug: 204136549
Change-Id: Ia92f7e5c427adcef3bcf59c82a2f83450905c01d
Host symlinks cannot use order-only dependencies because they may be
used as part of the dependency chain on a tool, and an order-only
dependency would cause the target of the symlink not to be updated.
Use regular dependencies instead.
Bug: 204136549
Fixes: 205674000
Test: TestInstallBypassMake
Change-Id: Ib3f4ee143e94d1995ec6c60d314e7c91e57cc775
Test: lunch aosp_coral-userdebug; WITH_TIDY=1
make 7-libgptutils.coral-checkbuild 7-libgptutils.coral-tidy
Test: only coral libgptutils in out/build-aosp_coral.ninja
Test: lunch aosp_arm64-eng; make has no *libgptutils* tagets
Test: no libgptutils in out/build-aosp_arm64.ninja
Test: lunch aosp_cf_arm64_phone-userdebug; make libskia-checkbuild
Test: lunch aosp_arm64-eng; WITH_TIDY=1 make tidy-soong
Bug: 202763874
Change-Id: I2d5a3fdc627d62e43c3f10e61b338a03c10440be
Previously Soong's install rules have been disabled when embedded
in Make (ctx.Config().KatiEnabled() == true). The primary blocker
for moving installation into Soong has been the `required` proeprty,
which is too vague to be easily handled in Soong. Keeping
installation in Make has resulted in two host bin directories,
the Make-owned directory (e.g. out/host/linux-x86/bin), and the
Soong-owned directory (e.g. out/soong/host/linux-x86/bin). The
lack of knowledge in Soong about the final, Make-owned installation
location makes it hard to support NOTICE files entirely in Soong.
This patch begins to solve this problem by supporting the creation of
the installation rules into Soong, but rather than writing the rules
to the ninja file it writes them to a Makefile and lets Kati convert
them to ninja. This allows Kati to inject extra dependencies to
handle the `required` property.
Converting all modules to create their installation rules in Soong
would be too complex, so only modules that return true from
InstallBypassMake will use the Soong installation rules. This
is currently only set for robolectric tests.
Bug: 204136549
Test: m checkbuild
Change-Id: I28af9fa7fadece8ea1f98f5efd140c823751cae7
* RBE adds implicit dependent files as inputs
and creates too long commands.
* Adds .tidy files as validation paths.
* Change type of tidyFiles to WritablePaths.
Bug: 203577539
Test: lunch aosp_cf_arm64_phone-userdebug; USE_RBE=true WITH_TIDY=1 make
Test: USE_RBE=true WITH_TIDY=1 mm in external/skia
Change-Id: I5fc9d6f8c9c58abcd3721ca9fbae1bc785e4e609
The Android SDK has been built with Make up until now, monolithically,
then split up into several sdk-repo zip files for different purposes.
The Mac and Windows SDKs really only need to contain the platform-tools
and build-tools pieces, but due to this monolithic sdk zip, we had to
build the whole SDK first.
This adds an `android_sdk_repo_host` module that can build these
platform-tools and build-tools zips.
Bug: 187222815
Change-Id: I55809e1d7447dd65e22461f921b2b8abb6d5f822
Introduce `commonAttributes` & `fillCommonBp2BuildModuleAttrs used in
CreateBazelTargetModule
Adapt `bp2BuildInfo` to use `commonAttrs` instead of `Name`.
And thus also all downstream users of `CreateBazelTargetModule`.
As initial user, the Soong `required` property will be
translated to Bazel's `data`.
Bug: 198146582, 196091467
Test: build_converstion_test.go:TestCommonBp2BuildModuleAttrs
Test: go test
Test: mixed_{libc,droid}.sh
Change-Id: Ib500e40f7e2cb48c459f1ebe3188962fc41ec124
Previously, the archTypeSpecificInfo included an array of
*linkTypeSpecificInfo. This change replaces that array with an array of
*imageVariantSpecificInfo which themselves contain an array of
*linkTypeSpecificInfo.
That allows the sdk snapshot to handle image variants correctly, i.e.
collate their properties, optimize their properties and then detect if
there are any image variant specific properties for any image variant
other than the CoreImageVariant ("") and report it as an error.
The latter case is treated as an error because while Soong needs to
handle image specific variants there is currently no requirement to
handle generating a prebuilt with image specific properties. A follow
up change will test the error handling.
Image specific variants are needed because the "jni_headers"
cc_library_headers module provides a number of image variants (e.g.
recovery) that are used outside the ART module. Therefore, the sdk
snapshot needs to do the same.
At the moment image variants like the recovery variant are supported by
copying the property that creates the variant (e.g. recovery_available)
through to the prebuilt but that is not safe for a couple of reasons:
1. It ignores any differences between the recovery variant and the
other variants which could cause compatibility issues in modules
that build against the prebuilts.
2. It marks modules in the snapshot with recovery_available even when
they do not need it.
This change will allow follow up changes to address both those issues.
Bug: 195754365
Test: m nothing
Change-Id: I1c187d814f44b2cb7607cd43a6b215134be0faad
* In builder.go, share common flags in a module.
* This replaces the sharing of cflags/cppflags/asflags in cc.go.
* A unit test in apex_test.go now fails and is commented out.
It is a failing test hidden by old optimization in cc.go.
* In module.go, expand the reference variable $someflags<n>,
or ${someflags<n>} to keep many existing unit tests work as is.
* The build.ninja size was reduced from 8.1GB to 6.2GB,
for aosp_arm64-eng WITH_TIDY=1 USE_RBE=true,
and from 7.5GB to 5.6GB when USE_RBE is 0.
Content of build.ninja is also more readable and searchable.
Read/write build.ninja times are also reduced,
depending on disk I/O speed.
Test: make WITH_TIDY=1
Change-Id: I17f96adf4844136d52e5d40f57a19d9e290162b7
Support three options for converting modules with unconverted
dependencies
1. (default) Warn when converting a module if it has unconverted deps.
2. Error when encountering a module with unconverted deps. (not hooked
up yet)
Test: build/bazel/ci/bp2build.sh
Test: build/bazel/ci/mixed_libc.sh
Test: BP2BUILD_ERROR_UNCONVERTED=1 build/bazel/ci/bp2build.sh with
unconverted deps -- get appropriate error
Bug: 181155349
Change-Id: Ifaabf0cd2e43e963366dc137159c705294165c3d
* When WITH_TIDY=1, these targets allow quick check of C/C++
source code with clang-tidy, without building C/C++ binaries.
* For each module with tidy rules, add a module-tidy target, e.g.,
libart-tidy, libartd-tidy, bionic-benchmarks-tidy, libnativehelper-tidy, etc.
* Add a tidy-soong phony target that depends on all module-tidy targets.
* For each directory X/Y add a tidy-X-Y phony target that depends
on all *-tidy targets in X/Y and tidy-X-Y-Z for all X/Y/Z directories,
e.g., tidy-bionic, tidy-bionic-benchmarks, tidy-libnativehelper, etc.
* Only soong modules are collected for now.
Tidy rules in .mk files will be collected later.
* Some comment lines reformatted by gofmt.
Test: WITH_TIDY=1 make <some_module>-tidy tidy-<some_directory>
Test: WITH_TIDY=1 make tidy-soong
Bug: 199169329
Change-Id: I45aef3875f70288a8e070761e5f083dbbdfa6e94
This code to support bp2build-specific modules is no longer necessary as
we do not create bp2build modules after https://r.android.com/1792714
Test: build/bazel/ci/bp2build.sh
Change-Id: I20d3c42f7e3516554cfb537fef05ba40d222dbeb
Clarify that HostAndDeviceSupported builds _only_ Device by default,
and add information on how to built host variations.
For HostAndDeviceDefault, clarify how to disable different variants.
Change-Id: I20e30175ddf79038117f1df166d9b994237b27f5
A performance improvement for bp2build as Blueprint/Soong no longer have
the overhead of additional modules. The creation of these modules
results in:
* traversal of additional modules for each subsequent mutator
* synchronization over a go channel to collect newly created modules:
https://cs.android.com/android/platform/superproject/+/master:build/blueprint/context.go;l=2594,2600;drc=1602226f23181b8c3fbfcaf3358f0297e839d7d3
We avoid both of these by storing the information directly in the
underlying module.
Also as a fringe benefit, removes some necessary boilerplate for
conversion.
For benchmarks, reduces runtime ~1% for 1% converted, ~24% for 100%
converted. See more: go/benchmarks-for-https:-r.android.com-1792714
Test: ran benchmarks/tests in bp2build
Test: build/bazel/ci/bp2build.sh
Change-Id: Ie9273b8cbab5bc6edac1728067ce184382feb211
Calling this function for dependencies that are not android.Module
results in a hard error rather than a soft skip. Update the doc to
reflect this
Bug: 195369546
Test: TH
Change-Id: I18a0938c6536cf20f239cf98c601722aaecd51e6
Gathering packaging items from "SkipInstall" deps doesn't make sense.
Bug: 194403710
Test: soong test
Test: m microdroid # microdroid shouldn't have libandroidicu
Change-Id: If6c3ee82d588e2742c85cef7244c090c93f38b8e
Refactor bp2build to retrieve modules directly by name, instead of via
DirectDeps. This functions properly as bp2build has no need for variant
information of the blueprint graph.
Test: USE_BAZEL_ANALYSIS=1 m fmtlib
Change-Id: Ief4b67bc56f24929871af772f3a742f07085bf8c
Make prebuilt_etc module available for the snapshot. This change
includes implementing snapshot interface for the prebuilt_etc module so
it can be added to the snapshot from the fake snapshot, or when the
module is specified in the list.
Bug: 192430376
Test: m nothing passed
Test: Checked if the module is included in the snapshot properly
Change-Id: I3574d2a1b8f8f4e5f083f3913e8768f5088d0c46
The actual behavior of the method is slightly different from its text
right now. Update the text to describe the actual behavior.
Bug: 193228441
Test: m nothing (change is noop)
Change-Id: I755829af6aa1b3f95ac71f33d75751b94e71b54b
Previously, a module reference in a path property would be parsed into
two parts, the module name and the optional output tag, which defaults
to an empty string if not specified. The output tag would be stored in
a sourceOrOutputDependencyTag which would then be used, along with the
module name to add a dependency on the module.
Later, when the paths were processed the same module reference would be
parsed into the same two parts again and the module name used to find a
matching Module by comparing it against the name returned by either
Module.Name(), ctx.OtherModuleName() or ModuleBase.BaseModuleName().
Once the module had been found then if it supported OutputFilesProducer
then the tag would be passed to its OutputFiles(tag) method. Otherwise,
it would fall back to treating it as SourceFilesProducer.
The problem with that is the module name retrieved from the module in
some way (either directly or through a context name) could be different
to that originally supplied when adding the dependency. e.g.
1. If the original dependency was added onto a source module but there
existed a suitable and preferred prebuilt module then the dependency
onto the source module would have been replaced by the prebuilt
module which has a different name.
2. If the path property included a fully qualified name that included
a qualifying path then it would not match the name retrieved from
the module which would not include the qualifying path.
This change circumvents that whole issue by adding the module name that
was originally used to add the dependency into the DependencyTag. Now
the DependencyTag uniquely identifies the original module/outputTag
pair parsed from the module reference. The pathDepsMutator guarantees
that they are unique as it dedups them before adding the dependencies.
It is possible that calling ExtractSource(s)Deps() would add some
duplicate but if they did they would be identical, i.e. the same
sourceOrOutputDependencyTag would be used to add a dependency onto the
exact same module. In that case it would not matter which of the
dependencies was found as it would still return the same module.
Bug: 193228441
Test: m nothing
Change-Id: I661514a2984818e5c26577411cede53eb57bcd02
