The logic is not 100% provably the same since HEAD was quite
confusing at some points, but I did make an effort to preserve
functional equivalence.
In case that effort was not enough, it should be pretty easy to
tweak the logic at HEAD since it's still quite malleable.
Bug: 231370928
Test: Presubmits.
Change-Id: I17b2efbfb5c4d0aedd922caed54ff8d857e578df
Modules created by a LoadHookContext do not currently set a module type
when creating new modules, but it would make analysis of the
module-graph.json much easier if this module type information was available.
Bug: 174879786
Test: m json-module-graph && check that module which previously had a
blank module type now have the field populated
Change-Id: I3be5d259694a1540d21deb8a665ec7bea3464054
Each conversion required defining a separate mutator, which will each
operate on _all_ modules and requires each to repeat checks whether the
mutator should operator. Instead, we introduce a single mutator and
modules can define a ConvertWithBp2build to implement bp2build
conversion for that module.
Test: bp2build.sh
Bug: 183079158
Change-Id: I99d4b51f441c2903879092c5b56313d606d4338d
Refactor bp2build to retrieve modules directly by name, instead of via
DirectDeps. This functions properly as bp2build has no need for variant
information of the blueprint graph.
Test: USE_BAZEL_ANALYSIS=1 m fmtlib
Change-Id: Ief4b67bc56f24929871af772f3a742f07085bf8c
Without this fix, mixed builds are silently disabled, as a recent change
disabled mixed builds for all module types which are not listed in this
map. This map should be populated for both bp2build generation and for
mixed builds.
Test: lunch aosp_flame && USE_BAZEL_ANALYSIS=1 m libc
Change-Id: Ib2a3fb2de2e7a5204ef01355522ae569e0424dc7
This CL adds the support for specifying lists of directories in
build/soong/android/bazel.go, which are then written into
out/soong/bp2build/MANIFEST. Using this configuration,
modules/directories can either default to bp2build_available: true or
false, while still retaining the ability to opt-in or out at the module level.
It also ensures that ConvertWithBp2Build returns true iff the module
type has a registered bp2build converter.
Test: go tests
Test: demo.sh full
Test: TreeHugger presubmits for bp2build and mixed builds.
Change-Id: I0e0f6f4b1b2ec045f2f1c338f7084defc5d23a55
This is achieved by writing soong.environment.used in Main() instead of
as a side effect of a singleton. This makes a difference because build
actions are not generated when GENERATE_BAZEL_FILES=true is set,
therefore the side effect did not happen.
Arguably, Main() is made worse by this change, but I don't want to
tackle the problem of readably determining which mode soong_build is
running in in this change.
Test: Presubmits + the additional test.
Change-Id: I66af2429aedf008762173eaaa55b828b4cf4328b
It appears as though this is the first pre-singleton type to actually
be registered with the InitRegistrationContext as it failed due to an
uninitialized map, so this change also fixes that.
Bug: 182885307
Test: m nothing
Change-Id: Ibbf6d0db5f3c2fcc89291a16aa5f16b8b5009bd3
That is to allow the order in which singletons are registered at
runtime, including the ones added by default, to be used to define the
order in which singletons must be registered in tests.
Bug: 181953909
Test: m nothing
Change-Id: I22f90c90100a8d52a572d4451a1a5d256784b57a
This separates the collation of mutators from the registration of them
to allow the test infrastructure to sort the mutator order to match
that used at runtime.
Bug: 181953909
Test: m nothing
Change-Id: I01a073289d44417f327b0815c09eb1c033d464f2
In preparation for following changes that will ensure the order of
registration (and so execution) for mutators, singletons and
pre-singletons in tests match the order in runtime this change creates
the sortableComponent interface to allow those to be sorted in the same
way without having to resort to reflection.
By moving the registration code into each component type this change
also eliminates some unnecessary duplication of that code.
Bug: 181953909
Test: m nothing
Change-Id: I597b461b966c84faaeb13e7dff765f1fadd99981
This currently expands all globs, still need to support converting glob
syntax.
Test: go build_conversion_test
Test: GENERATE_BAZEL_FILES=true m nothing
Test: m nothing
Bug: 165114590
Change-Id: If7b26e8e663d17566fad9614ca87a8da1f095284
This CL creates the framework necessary for generating
BazelTargetModules from regular Soong Android modules.
BazelTargetModules are code-generated into Bazel targets in BUILD files.
See the follow-up CL for examples of creating filegroup/genrule
BazelTargetModules.
Test: GENERATE_BAZEL_FILES=true m nothing # creates out/soong/bp2build
with no BUILD files, because there are no BazelTargetModules in the
module graph.
Change-Id: I33a96365bd439043b13af6db9e439592e9983188
Implement bp2build codegen as a discrete step that runs after an
alternatively registered pipeline of mutators, instead of a
presingleton.
bp2build codegen requires a Context that supports VisitAllModules and
PathContext, so this CL also makes a BpToBuildWrapperContext that
conforms to PathContext by adding two method implementations.
Test: GENERATE_BAZEL_FILES=true m nothing && bazel query //... --config=bp2build | wc -l # 31433
Test: m queryview && bazel query //... --config=queryview # 63638
Change-Id: I0dd359746584b228046d2d0ff00895f28f9bdfc3
A SingletonModule is halfway between a Singleton and a Module. It has
access to visiting other modules via its GenerateSingletonBuildActions
method, but must be defined in an Android.bp file and can also be
depended on like a module.
Bug: 176904285
Test: singleton_module_test.go
Change-Id: I1b2bfdfb3927c1eabf431c53213cb7c581e33ca4
Splits queryview into queryview and bp2build. The latter runs as a
presingleton (could be converted to a singleton). This prevents needing
to run soong_ui a subsequent time to build the query. Queryview remains
as a separate step to prevent increasing runtime due to this generation
every time Soong runs.
Currently this is running as a presingleton as this gives a translation
of Android.bp files after only LoadHooks have run, no mutators.
Test: go tests
Test: m queryview && bazel query --config=queryview //...
Change-Id: If2ba28c7ef60fbd41f43bda6623d41c8c7d23a1b
This CL adds a new CONVERT_TO_BAZEL env var, and a bp2build goal. It
reuses the queryview architecture, but registers no mutators before
converting to BUILD files. This gives us a blank slate of the module
graph to work with, and create a shadow/alternate pipeline of mutators
to converge the Bazel BUILD graph to be semantically equivalent with the
Android.bp graph (after apply its current set of mutators).
The command to do so is:
$ CONVERT_TO_BAZEL=true m bp2build
To not clobber with queryview, the generated files are in
out/soong/bp2build.
Test: CONVERT_TO_BAZEL=true m bp2build && bazel query --config=bp2build //...
Test: m queryview && bazel query --config=queryview //..
Test: soong tests
Test: TH presubmit
Fixes: 174465461
Signed-off-by: Jingwen Chen <jingwen@google.com>
Change-Id: I4dd0ccc73abc345d70a50ca2803d6f400cd8c863
Store ninja file deps from ExistentPathForSource on a
PackageVarContext in the config instead of the PackageContext, as
the PackageContext may be shared between multiple tests running
in parallel.
Test: all soong tests
Change-Id: Ib1809a4dd4a82696e0fe48a87eac21a44684ecb5
Store SingletonMakeVarsProviders in the config instead of a global
variable to avoid races between tests running in parallel.
Test: all soong tests
Change-Id: I2ab64f368b5ac673fd985399d4421ed018abc562
Prepare for using Config when adding singletons by passing
Config to NewTestContext and NewContext instead of to ctx.Register.
This will enable a followup change to store SingletonMakeVarsProviders
registered on the Context in the Config, which is necessary to run
multiple tests in parallel without data races.
Test: all soong tests
Change-Id: Id229629a4e42ff4487d317241673837726c075fc
This retriggers soong_build whenever a new bzl, WORKSPACE, or
BUILD.bazel file is changed or added.
Test: Manually verified on bionic/libc genrules with manual changes to
related BUILD/bzl/WORKSPACE files -- these all retrigger builds.
Test: Updated finder_test.go
Change-Id: I634384f88781a6b6db32f5d6bf9c07e179e14c39
To support dist-for-goals in Soong, we need to define all phony rules
in Make so that dist-for-goals can insert additional dependencies on
them. Collect all the phony rules in phonySingleton and write them
out as Make rules when Soong is embedded in Make, or as blueprint.Phony
rules when Soong is run standalone.
Test: m checkbuild
Change-Id: I68201eff30744b0f487fc4f11f033767b53a627d
This is for use to add dependencies between specific module variants,
without getting the dependencies potentially duplicated as more
variants are created.
Moved genrule tool dependency registration to this phase, to avoid
potential splitting of its dependencies, and for consistency with the
same kind of dependency on dex2oat in https://r.android.com/1205730.
Test: m nothing
Bug: 145934348
Change-Id: I1f9e75354e359cef5f41d14e5016b0a67d7655d7
Some pre arch mutators are hard coded into mutator.go and so could not
share code for registering those mutators between tests and runtime.
This change adds a new HardCodedPreArchMutators(RegisterMutatorFunc)
method to RegistrationContext which allows hard coded mutators to be
registered alongside other build components during init() and testing.
The method is treated as a noop on the InitRegistrationContext and
behaves just like the PreArchMutators(RegisterMutatorFunc) method on
the TestingContext.
Bug: 146540677
Test: m nothing
Change-Id: I6f8b1e2d54d9dc4e86f951ced61d1ee7b0fe4b2e
This relands I12a0f907753fefd1997ab8b4ea2ac331234093cf along with
a fix to blueprint for absolute paths.
Store the current working directory and then change to the root
directory so that all file accesses must go through helpers in
the android package that properly track dependencies.
Change-Id: I24ac485677aa102eec1a2521d16820da6ee1ae77
Fixes: 146437378
Test: m checkbuild
Test: m OUT_DIR=/tmp/out nothing
Store the current working directory and then change to the root
directory so that all file accesses must go through helpers in
the android package that properly track dependencies.
Fixes: 146437378
Test: m checkbuild
Change-Id: I12a0f907753fefd1997ab8b4ea2ac331234093cf
Module type and singleton type names have to be unique but duplicates
are not checked when the relevant android.Register...() func is called.
Instead they are collated in lists and then registered with the Context
later on, at which point duplicates are detected. That loses
information that can be helpful in fixing the issue.
This is not an issue when testing as the module and singleton types
are registered directly with the Context.
This change adds duplicate detection to the initRegistrationContext
Register... methods prior to calling the android.Register... methods
which should make it easier to detect duplicates registered from an
init() function.
Test: m checkbuild
Bug: 146540677
Change-Id: I7f1a4b649072867717a9829c737a44454b12266c
The registration of module types and singletons is duplicated between
init() functions that register them for use in the build runtime and
test context creation code that registers them for testing.
This is a proof of concept for a mechanism that will allow the code
to be shared. It defines a RegistrationContext interface that is
implemented by both the TestContext and the new initRegistrationContext
type. An instance of the the latter is available through the
InitRegistrationContext variable.
The intent is that the registration of the module types and singleton
types will be extracted from the init() function into a separate
function that takes a RegistrationContext parameter. That method is
called from init() passing in the InitRegistrationContext and from a
test passing in the TestContext. Something like this:
func init() {
RegisterBuildComponents(android.InitRegistrationContext)
}
func RegisterBuildComponents(ctx android.RegistrationContext) {
ctx.RegisterModuleType(....)
....
}
A test would do something like this:
ctx := android.NewTestContext()
RegisterBuildComponents(ctx)
Test: m nothing
Change-Id: I97173cabb6d6cf7ce98fdb5f73418438b1997b35
Add support for TestingModule to return RuleParams for rules created
by the module.
Refactor TestingModule to use helpers, and use the helpers to
implement a similar TestingSingleton.
Use the new functionality to test RuleBuilder's module and singleton
rules.
Test: none
Change-Id: I8348c56ff5086d0c49401f5a00faf7c864e6b6f3
Register any Singletons that implement a MakeVars method as
MakeVarsProviders, and convert the hiddenapi singleton to
use it.
Test: m checkbuild
Change-Id: I6a2044ad34ef46a8b267762ddfeb51aa01d7734d
bootstrap.ModuleTypeDocs needs a mapping of module types to factories
to support factories that are wrapped in ModuleFactoryAdapter closures.
It also returns ModuleType objects grouped into Package objects.
Bug: 67909957
Test: m soong_docs
Change-Id: I70eac9f0f0e13075580da92d4219792ca0b18fbf
Move the makevars singleton after all dyamically registered
singletons, but before the env singleton, to ensure that it
can reference values initialized in other singletons.
Test: m checkbuild
Change-Id: I6ea45d3a174b94d4e89f3168894399e7022733e4
Wrap blueprint.PackageContext so that the *Func methods can provide
an android.Config instead of an interface{}. The modified signatures
means that every method in ModuleContext and SingletonContext
that takes a blueprint.PackageContext now needs to be wrapped to
take an android.PackageContext.
SingletonContext wasn't previously wrapped at all, but as long
as it is, wrap everything like ModuleContext does. This requires
updating every Singleton to use the android-specific methods.
Test: builds, all Soong tests pass
Change-Id: I4f22085ebca7def6c5cde49e8210b59d994ba625
Fix mutator registration for tests to allow different tests
in the same package to register different mutators.
Allow tests to track the resulting ModuleBuildParams objects
to use in assertions, and provide helpers for getting them.
For example:
config := android.TestConfig(buildDir)
ctx := android.NewTestContext()
ctx.RegisterModuleType(...)
ctx.MockFileSystem(...)
ctx.ParseBlueprintsFile("Android.bp")
ctx.PrepareBuildActions(config)
ctx.Register()
// Get the Inputs value passed to the javac rule for the foo module
inputs := ctx.ModuleForTests("foo".Rule("javac").Inputs
Test: java_test.go
Change-Id: I10c82967f5f3586d2c176f169906b571ed82fc73
Change module factories from returning a blueprint.Module and a list
of property structs to returning an android.Module, which holds the
list of property structs.
Test: build.ninja identical except for Factory: comment lines
Change-Id: Ica1d823f009db812c518f271a386fbff39c9766f
Calling android.NewContext() in tests results in a context that
contains all the mutators, module types, and singletons, which
causes unexpected interactions in unit tests. Create an empty
context instead, and add in only the necessary mutators, module
types, and singletons.
Bug: 36366816
Test: soong tests
Change-Id: Ic61262c37e3436b3ad4ccaca18b737021c304be6
Mutators are registered into a global variable, and then into the
context when it is created. Only call registerMutators once so that the
same mutator is not listed multiple times in the global variable when
multiple contexts are created.
Test: go test -v android/soong/android
Change-Id: Ie9e3ed09a89b848462b898476cdfb81a90c64bd3
