Commit Graph

128 Commits

Author SHA1 Message Date
Treehugger Robot
6fab06d56b Merge "Soong system image may update the $PRODUCT_OUT" into main 2024-09-12 00:25:21 +00:00
mrziwang
1842097928 Add framework for using provider on AndroidMKEntries
In the context of incremental soong, the inter-module communication
through AndroidMkEntries and AndroidMkData should be changed to
through the real provider.

This change adds the framework for the provider(AndroidMkInfoProvider).
A module will set its AndroidMkInfoProvider, which includes what's
in its AndroidMkEntries method, to be used in translateAndroidMkModule.

This change also changes module type vbmeta to use
AndroidMKInfoProvider.

The next step is to replace the AndroidMkEntries methods of all the
other module types to set the AndroidMkInfoProvider.

Test: m with tests and CI
Bug: 357907638
Change-Id: Icfd1363f97d31548bb78c6615e0f7076b22dcbe4
2024-09-11 10:38:41 -07:00
Justin Yun
2cc42505d9 Soong system image may update the $PRODUCT_OUT
A target must define "USE_SOONG_DEFINED_SYSTEM_IMAGE := true" and
"PRODUCT_SOONG_DEFINED_SYSTEM_IMAGE := <name of soong system image>"
to use the soong defined system image.
The system image must install the files to $PRODUCT_OUT for adb sync
and build verification. Instead of using 'update_product_out'
property in Android.bp, copy the files only if the name of the image
matches with PRODUCT_SOONG_DEFINED_SYSTEM_IMAGE.

Bug: 350599535
Bug: 365700376
Test: lunch aosp_x86_64-trunk_staging-userdebug && m &&
      m aosp_cf_system_x86_64
Test: lunch aosp_cf_x86_64_phone_soong_system-trunk_staging-userdebug
      && m
      and adb sync on the cf device
Change-Id: If8673ba2855a753a5559a588bd83068d950fd7e4
2024-09-11 14:34:44 +09:00
Justin Yun
34baa2e47c Update $PRODUCT_OUT from the soong system image files
When building a system image from 'android_system_image' module,
update the $PRODUCT_OUT files with the files installed to the system
image.
Soong updates the $PRODUCT_OUT files only if the android_system_image
module has "update_product_out: true" property.

Bug: 350599535
Test: lunch aosp_cf_x86_64_phone_soong_system-trunk_staging-userdebug
      && m
Test: adb sync
Change-Id: Ia3418c5380cac42d5c766e8304c8ed2bab3d835a
2024-09-05 11:02:57 +09:00
Jiyong Park
43b157a265 logical_partition.groups.partitions.filesystem can be empty
This fixes a minor bug that filesystem image was always needed for a
partition in a group. With this change, the filesystem image can be
omitted (as is the case for Make).

Bug: 339587835
Test: m nothing
Change-Id: I2f87612ec824e1298e3bddea8a4b90932b438f41
2024-08-27 18:00:40 +09:00
Yu Liu
d46e5ae670 Remove more internal fields from ModuleBase.
Bug: 358425833
Test: Manual compare the generated ninja and mk files and CI.
Change-Id: I4edd0dc646ac035594f47cd9044cc4eb5f710e64
2024-08-16 00:20:27 +00:00
Colin Cross
be891d5a52 Merge changes I5f2fd123,Ie8d8e229,Id2194f6b into main
* changes:
  Move RelativeToTop out of basePath
  Make PathForArbitraryOutput return an OutputPath
  Run TestClasspath subtests in parallel
2024-07-30 16:30:54 +00:00
Colin Cross
3b1c6847c4 Make PathForArbitraryOutput return an OutputPath
The only place basePath is used as a Path is when being returned from
PathForArbitraryOutput.  Using it as a Path requires implementing the
RelativeToTop() method on it, which then allowed that method to be
inherited by SourcePath, breaking the contract on RelativeToTop that
specifies that the returned Path is of the same concrete type as the
input Path.

Make PathForArbitraryOutput return an OutputPath, and update OutputPath
to support base paths that are not out/soong.  This also fixes
RelativeToTop, which was previously not working for PathForArbitraryOutput
paths.

Test: all soong tests pass
Flag: EXEMPT refactor
Change-Id: Ie8d8e2290961f35280e97137d2bd641c4d57ab87
2024-07-29 14:00:46 -07:00
Cole Faust
bf1d92ad20 Cleanup configurable getter usages
You don't have to call module.ConfigurableEvaluator(ctx) if ctx is
already a ModuleContext, you only need to do that for more restricted
contexts like SingletonContext.

Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: I7612290d43dae7decfae283a341882d9016c98a3
2024-07-29 12:24:25 -07:00
Inseob Kim
4daca6b242 Make vbmeta Partitions property configurable
This allows using select statements.

Bug: 354824866
Test: boot
Change-Id: I6cb41177620f85697eeaa77d3aec79f21674e245
2024-07-26 11:17:52 +09:00
Inseob Kim
33f95a959d Add support installing to root of filesystem image
If the partition of a spec is "root", that spec will be installed to
root instead. Normally that spec will be from prebuilt_root module with
install_in_root property.

Bug: 351258461
Test: USE_SOONG_DEFINED_SYSTEM_IMAGE=true m && cvd start
Change-Id: Iaaa9c2fb8a81fe0ba4710c641e1b65c5b71ad4a4
2024-07-15 17:39:44 +09:00
Colin Cross
e1a8555581 Add IsAddingDependency to IncomingTransitionContext
Add an IsAddingDependency method to IncomingTransitionContext that
returns true if IncomingTransition is called after the transition
has already won while adding a new dependency.  This will be used
as part of the transition mutators to support an apex use case where
incoming dependencies during the initial apex mutator need to be
rewritten onto the platform variant for modules that don't support
the apex, but a later call to OtherModuleDependencyVariantExists
must not rewrite the requested apex variation onto the platform
variant.

This should be used sparingly, all uses will have to be removed in
order to support creating variants on demand.

Bug: 319288033
Test: TestIsAddingDependency
Flag: EXEMPT refactor
Change-Id: Ib8e419d35ff8f7cbff9667c1cd40d05ccfacab8b
2024-07-11 16:59:34 -07:00
Kiyoung Kim
99a954d98d Generate file list installed from a filesystem
Generate file list installed from a filesystem, so it can be compared
with files installed from KATI.

Bug: 346873717
Test: File list generated with aosp_cf_system_x86_64
Change-Id: I8c8f555d57fee90753120f637dc3bc633f276a09
Merged-In: I8c8f555d57fee90753120f637dc3bc633f276a09
2024-07-02 02:22:37 +00:00
mrziwang
555d133433 Use OutputFilesProvider on filesystem module types
Test: CI
Bug: 339477385
Change-Id: I0902726cfef13c716d73c1abc5c9836bdcf3e10f
2024-06-07 11:15:33 -07:00
Cole Faust
d9c6a5b802 Make dirs a configurable property
The microdroid system image customizes dirs using soong config
variables, make it configurable so that selects can be used instead.

Bug: 342006386
Test: Presubmits
Change-Id: I4fbf4dc0a04ad1d1f6539476408606028390cd64
2024-05-21 14:54:00 -07:00
Jiyong Park
3ea9b65990 filesystem modules gathers first target only
With this change, the deps property in filesystem modules gather the
first target of the filesystem module only.

To gather dependencies across both targets, use multilib.both.deps
instead.

Bug: N/A
Test: go test ./...
Change-Id: Ie2ff0c48f08c61c8b219fc2c1540476ff8e4b1fc
2024-05-16 18:23:20 +09:00
Justin Yun
162098358c Build aconfig storage files from soong filesystem
`gen_aconfig_flags_pb: true` additionally generates, aconfig storage
files in `etc/aconfig` directory. That includes, package.map,
flag.map and flag.val files.

Bug: 329187549
Test: m aosp_cf_system_x86_64
Change-Id: Ibbfde390cd84ca06bdd345ce0d0b4baa30a60773
2024-05-16 13:25:37 +09:00
Jiyong Park
c6a773df64 Filter-out deps of unsupported arch
When a common-arch module (ex: phony) has the "required" dependencies on
native modules (ex: cc_library), both 32 and 64-bit variants of the
native modules are added as dependencies. This by itself is fine and
actually is intended, otherwise there's no way for us to install both
arch variants via required deps.

However, this imposes a problem when the common-arch module is depended
on by a filesystem module with compile_multilib: "first". Here, the
expectation is that only the first variant (64-bit) of the native module
is instaled, but in reality both variants are installed.

To handle this situation, make sure that the packaging routine filters
out packaging specs from unsupported architecture.

Bug: N/A
Test: go test ./... under soong/filesyste

Change-Id: Ie1ad5ace2e5d88e00183a115f4a76e5df87a8166
2024-05-14 21:56:42 +09:00
Jiyong Park
635e1218bd Merge "Fix: required deps from native module to phony module is respected" into main 2024-05-14 05:04:27 +00:00
Justin Yun
74f3f30ae4 Soong filesystem may install aconfig_flags.pb
By setting aconfig_flags: true, soong filesystem module installs the
aconfig_flags.pb file to its etc directory.

We need to define aconfigFilePaths to ModuleBase to store the list of
aconfig file paths that is generated from the GenerateBuildActions().
The aconfigFilePaths are collected when build the filesystem module
to build the aconfig_flags.pb for the filesystem image.

Bug: 335363964
Test: compare the cache list with those generated from make.
Change-Id: Ia32b566bf43174e94b9e610b9503608c6b583899
2024-05-13 23:14:42 +00:00
Jiyong Park
c4b1d55707 Fix: required deps from native module to phony module is respected
This change fixes a bug that required deps from native module to phony
module was ignored. It happened because addRequireDeps incorrectly
thought that both are native modules with different bitness (32->64),
which isn't.

Fix this by doing the bitness check only when both the current module
and the required module are native modules.

Bug: N/A
Test: go test ./... under build/soong/filesystem
Change-Id: I494ebc47e29001f174fa44d72809041f8ceffb0b
2024-05-13 16:47:30 +09:00
Jiyong Park
7c9ab4f5b1 Merge changes from topic "revert-3069323-revert-3066748-duplicate_entry_fs-CIBDJQYRHU-FVLDIVWUZV" into main
* changes:
  Revert^2 "Filesystem modules follow the installation semantic"
  Revert "Revert "Prevent multiple PackagingSpecs having same inst..."
2024-05-04 03:56:55 +00:00
Inseob Kim
53507d1559 Merge changes from topic "logtags_filesystem_module" into main
* changes:
  Add logtags support to filesystem module
  Export app's logtags too
2024-05-03 11:20:12 +00:00
Inseob Kim
b7b8457a60 Add logtags support to filesystem module
If build_logtags is set to true, etc/event-log-tags will be built by
collecting all logtags files from all dependencies.

Note that this is different from Makefile's current behavior that
collects all logtags files, including ones from uninstalled modules. But
the behavior is probably unintended, and ideally each partition image
should have its own logtags, rather than putting all tags to /system.

Bug: 336189540
Test: m aosp_cf_system_x86_64
Change-Id: I72b384cbb65e812af909eacd18799bd86f1dab13
2024-05-02 13:52:43 +09:00
Jiyong Park
7e7d4af6b5 Revert^2 "Filesystem modules follow the installation semantic"
54351867fa

Change-Id: I12010c1013fc9dad19d259ba7b0251ac8e8e0907
2024-05-01 12:36:10 +00:00
Jiyong Park
16ef7ac14e Revert "Revert "Prevent multiple PackagingSpecs having same inst..."
Revert submission 3069323-revert-3066748-duplicate_entry_fs-CIBDJQYRHU

Reason for revert: relanding with forward fix

Reverted changes: /q/submissionid:3069323-revert-3066748-duplicate_entry_fs-CIBDJQYRHU

Change-Id: Ica5a126446ca2dc07a40e4ff9866389b5d164eab
2024-05-01 12:36:10 +00:00
Jiyong Park
3d65d9cb90 Revert "Prevent multiple PackagingSpecs having same installation..."
Revert submission 3066748-duplicate_entry_fs

Reason for revert: b/338159248

Reverted changes: /q/submissionid:3066748-duplicate_entry_fs

Change-Id: I89906e160f62f1bf84eb65a1b99940f139626786
2024-05-01 05:25:41 +00:00
Jiyong Park
54351867fa Revert "Filesystem modules follow the installation semantic"
Revert submission 3066748-duplicate_entry_fs

Reason for revert: b/338159248

Reverted changes: /q/submissionid:3066748-duplicate_entry_fs

Change-Id: I8aacdb36efce8343c3713542eee209e71bde660b
2024-05-01 05:25:42 +00:00
Jiyong Park
742f54c4bc Filesystem modules follow the installation semantic
With this change, filesystem modules (android_filesystem,
android_system_image, etc.) follow the installation semantic which means
output from modules which don't produce installable files (i.e. modules
that return false on IsSkipInstall()) are not included in the filesystem
module.

Note that this bug has been there since the beginning but didn't trigger
an error due to another bug (b/335506668) which allowed multiple
packaging specs to contribute to the same path. That bug prevented the
non-installable variant of a module (e.g. stub) from being installed
simply because because it was "luckly" clobbered by the installable
variant of the same module (e.g. impl lib).

Bug: 335506668
Test: go test ./...
Change-Id: I2a42d178701d74bf6ccc5f33ba33918f8c608d40
2024-04-30 19:02:57 +00:00
Jiyong Park
a51c4ce10e Prevent multiple PackagingSpecs having same installation path
This fixes a bug that different PackagingSpecs having the same
installation path were silently allowed. Previously, a PackagingSpec
that comes the first for the given installation path won, effectively
eclipsing other PackagingSpecs destined for the same installation path.

Bug: 335506668
Test: go test ./...
Change-Id: Ia36f656e8364f95c4be78fff6e9dc16966307526
2024-04-30 18:50:06 +00:00
Jiyong Park
5c3c3274fe Remove an unnecessary test in filesystem
TestFileSystemShouldInstalCoreVariantIfTargetBuildAppIsSet is removed.
The test was originally introduced with aosp/2445946, but the change was
effectively reverted by aosp/2562192. The test however has been green as
a side effect of a bug (b/335506668) which will be fixed soon.

Bug: 335506668
Bug: 268582372
Test: N/A
Change-Id: I62f4746c605eba0b2f0037089a53d65922d65ef3
2024-04-30 22:35:14 +09:00
Jiyong Park
f21dd65c08 Revert^2 "Use Module interface in addRequiredDeps"
This reverts commit bbcdaa0c32.

Reason for revert: relanding with a forward fix

Change-Id: I14b83010a4b8d9cb9cb317f2aa07875c17f6cdaa
2024-04-17 05:22:37 +00:00
Daren Liao
bbcdaa0c32 Revert "Use Module interface in addRequiredDeps"
This reverts commit de18a3d55a.

Reason for revert: b/335332155

Change-Id: Ib5b74b1704c1446713466e8617c42b212c960d49
2024-04-17 03:07:53 +00:00
Jiyong Park
de18a3d55a Use Module interface in addRequiredDeps
Previously, addRequiredDeps directly called RequiredModuleNames directly on
ModuleBase. As a result, it failed to correctly track the dependencies
for the modules which are overriding RequiredModuleNames. cc_* were
those.

Fixing this by calling RequiredModuleNames via the Module interface.

Bug: 321626681
Test: go test ./... under filesystem
Change-Id: I79de616606b88277da0b3e86b21316ee83e0ec71
2024-04-13 10:51:18 +09:00
Jiyong Park
52a08486f3 Merge "android_filesystem_defaults and visibility rule" into main 2024-04-09 23:06:43 +00:00
Treehugger Robot
ccd5b5545b Merge "fix: required property doesn't track deps to java, apex, ..." into main 2024-04-09 03:12:39 +00:00
Jiyong Park
f46b1af969 android_filesystem_defaults and visibility rule
android_systemimage_defaults is deleted and android_filesystem_defaults
is created so that we can have defaults not only for
android_system_image, but also all android_filesystem modules and its
derivatives.

Also, change adds a check which ensures that the partition_type of the
filesystem and its defaults are the same.

Bug: 330665150
Test: m
Change-Id: If635c794534ed89d264eaf7dfc406a8245b8e9f0
2024-04-09 11:26:15 +09:00
Jiyong Park
73e5babafe fix: required property doesn't track deps to java, apex, ...
This change fixes a bug that the required property doesn't track
dependencies to modules whose arch is common.

Bug: 321000103
Bug: 321626681
Test: go test ./...
Change-Id: I3d2b3ad8cb2a9f1c5c3d5345bf05402a787f011a
2024-04-05 15:12:16 +09:00
Nikita Ioffe
2c8cdc6166 Add support for rollback_index to android_filesystem module
Bug: 287593065
Test: m
Change-Id: I01e986b2d8e3d992533dc7e2c07e4cdbeaca2c00
2024-04-04 13:03:37 +00:00
Inseob Kim
1e6afedb96 Add libs for BuildManifest.apk generation
Bug: 330282551
Test: m aosp_cf_system_x86_64
Change-Id: I4f99b5638490dae4f1da22287647ef4dfd47dcaa
2024-04-03 17:24:54 +09:00
Inseob Kim
f16c5a0a7d Merge "Add support to generate fsverity metadata" into main 2024-04-02 07:19:32 +00:00
Inseob Kim
5339184e32 Add support to generate fsverity metadata
By setting fsverity.inputs, android_filesystem can now generate
.fsv_meta and BuildManifest.apk. It has been done by Makefile because
Makefile is the only one knowing all installed files. But now
android_filesystem is aware of all artifacts, so we can move fsverity
metadata generation into Soong.

Bug: 330282551
Test: m aosp_cf_system_x86_64 and see output
Change-Id: Iae4dd83eaede960c263bfba537211df4ff4b36bd
2024-04-02 10:04:22 +09:00
Jiyong Park
eaac823f19 Add android_system_image_defaults
Bug: 330665150
Test: go test ./... under filesystem
Change-Id: Id32e6563491a3d129042de4a9517c545543a022d
2024-03-31 21:27:45 +09:00
Jiyong Park
9c540c82d4 fix: required deps of phony modules are correctly tracked
ArchType.Multilib of native modules are either lib32 or lib64.
Arch-neural modules have "" not "common".

Bug: 321626681
Test: go test ./...
Change-Id: Ie7e6a5203e9f671487dbf32ea2343ada7407a28f
2024-03-29 12:53:32 +09:00
Kiyoung Kim
078188774f Merge "Update linker.config.pb based on package dependency" into main 2024-03-26 00:42:17 +00:00
Treehugger Robot
e58af3da5a Merge "Add required, host_required, and target_required as dependencies" into main 2024-03-25 04:10:50 +00:00
Kiyoung Kim
ee599d6694 Update linker.config.pb based on package dependency
Current linker.config.pb from the package is generated with
auto-detected provide libs, but this misses require libs which can be
detected from module dependency. This change adds extra require libs to
linker.config.pb generated from system image so it can link with modules
outside of system image.

Bug: 324995772
Test: Link succeeded from Cuttlefish with soong defined system image
Change-Id: I8563ec9ddce2a1648cc9ee55704c9483e137b710
2024-03-25 11:20:11 +09:00
Cole Faust
9a24d90936 Add more specific partition visibility rules
//visibility:any_system_partition, //visibility:any_vendor_partition,
etc.

Then, if a partition visibility rule is not specificed, but the module
is installed on a non-system partition via the `vendor: true` or other
properties, the visibility rule for that partition will be added by
default.

This is so that "any_partition" doesn't imply that modules could be put
on the vendor partition when they weren't designed for that, and so that
modules that do need to go on the vendor partition don't need to specify
both vendor: true and visibility:any_vendor_partition.

Eventually, the partition properties should be deprecated, and replaced
with just these visibility rules.

Bug: 321000103
Test: go tests
Change-Id: I24dba36bbc20921941f892480bf7c050e93827c6
2024-03-20 10:55:05 -07:00
Jiyong Park
8bcf3c64f1 Add required, host_required, and target_required as dependencies
So far, the installation of required modules were handled by Make. This
prevents us from implementing the module installation and packaging
entirely in Soong.

This CL is the first step towards that goal. Soong now correctly tracks
the dependencies and they are correctly returned by
TransitivePackagingSpecs(), which is used by packaging modules like
android_system_image.

Bug: 321626681
Test: build
Change-Id: I9192b5333ceaa0b7d1c5c4abeec2af62febcd976
2024-03-20 17:05:17 +09:00
Cole Faust
4a2a7c98f6 Add include_make_built_files
If `include_make_built_files` is set to the name of a partition, the
make-built files from that partition will be incorperated into this
soong module. This is to ease the transition to soong built filesystems.

If any files are present in both the soong-built file list and the
make-built one, the soong ones will be preferred.

Bug: 329146343
Test: go test
Change-Id: I456b283e1189116e699ed75357cc056f5d217688
2024-03-14 16:40:42 -07:00