Commit Graph

244 Commits

Author SHA1 Message Date
Mårten Kongstad
3d76d1afe0 aconfig: remove support for --format=bool
This CL is part of a larger effort to simplify maintenance by removing
format aliases: with the new --format option, clients of `aconfig dump`
can specify the format they need themselves.

Bug: 317044914
Test: m
Change-Id: I11efc5d7280402f503ddd05f2dc65bb15ffd8ac7
2023-12-20 13:39:58 +01:00
Mårten Kongstad
ce09eb2f31 aconfig: add missing tests to TEST_MAPPING
A few new aconfig tests have been created. Add them to TEST_MAPPING as
postsubmit tests.

Bug: N/A
Test: atest :all
Change-Id: Ifa4d764d26114367227c9c5622fb31b304a6972c
2023-12-20 10:32:50 +01:00
Treehugger Robot
45e9ce5d49 Merge "aconfig: rename subcommand 'dump' -> 'dump-cache'" into main 2023-12-19 22:00:40 +00:00
Zhi Dou
533421c194 aconfig: update codegen templates
This change removes empty lines and spaces in generated code.

Bug: 311152507
Test: atest aconfig.test
Change-Id: Ib41e633323ffcce348299afa5a09bd0252db1bb9
2023-12-19 21:48:51 +00:00
Mårten Kongstad
c91ebf8153 aconfig: rename subcommand 'dump' -> 'dump-cache'
Rename the 'dump' subcommand to 'dump-cache' for better symmetry with
'create-cache', and to prepare for a potentially upcoming
'dump-storage' subcommand.

Keep 'dump' as an alias for 'dump-cache' for backwards compatibility.

Bug: N/A
Test: m
Change-Id: I521ce507985e4cf450ee1b8517993b798f11430c
2023-12-19 22:06:24 +01:00
Zhi Dou
eeed7990de aconfig: modify and filter flags before passing into java codegen
Before this change java codegen filter flags for exported mode in the
template. This change move the filter process to commands as other
codegen. Thus the codegen code will only generate code based on the
passed in flags.

Bug: 311152507
Test: atest aconfig.test aconfig.test.java AconfigJavaHostTest
Change-Id: I74045709cde19e6c687c3eb0d94050ea40cf5042
2023-12-19 20:32:59 +00:00
Zhi Dou
80040d2d2c aconfig: modify_parsed_flags_based_on_mode return Result
This commit changes modify_parsed_flags_based_on_mode to return Result
instead of vector. This change will have two consequences.
1. Error will be thrown if no flags fit into the current codegen mode
2. Error will be thrown if the flag declaration is empty

Bug: 311152507
Test: atest aconfig.test
Change-Id: I69b9a84312faed9f757bf3974b3cea49c5c5e285
2023-12-19 20:28:23 +00:00
Zhi Dou
8351b78234 Merge "aconfig: move CodegenMode to codegen crate" into main 2023-12-19 20:00:56 +00:00
Zhi Dou
dcfa0400c9 aconfig: move CodegenMode to codegen crate
This change moves CodegenMode to codegen crate.

Bug: 311152507
Test: ateset aconfig.test
Change-Id: I4f64cdbdc3d1b419412012560bfe03344b2b8faf
2023-12-19 18:59:26 +00:00
Dennis Shen
4575b273fe Merge "aconfig: package.map file serialization" into main 2023-12-19 18:57:24 +00:00
Treehugger Robot
8d1e52319e Merge "aconfig: add support for dump --filter='{fully_qualified_name}:...'" into main 2023-12-19 18:56:40 +00:00
Zhi Dou
0f1b9da4d9 Merge "aconfig: add exported mode in rust codegen" into main 2023-12-19 18:32:46 +00:00
Dennis Shen
adc7b732ca aconfig: package.map file serialization
For each container, there will be three storage files: package.map,
flag.map and flags.val. This change adds package.map creation.

package.map is a hash table in file that maps a package name to its
package id and package value offset. This file will be used by flag
storage client lib to locate the fixed offset of a flag in flag value
file. package.map provides the file byte offset to the start of all flag
values for this package. Together with flag value offset within its package
which will be provided by flag.map, we can locate a flag's value in the
value file.

The top level struct for this file is called "PackageTable". The struct
consists of three parts: (1) table header which includes the file
version and other metadata such as number of packages and etc. (2) table
buckets which is an array of u32, each bucket stores the package.map
file offset to the package table node. (3) package table node array.
each node stores package name, package id, package value offset and
offset to the next package table node.

The table uses fixed format serialization. All u32 are encoded using
little endian format as that is the Android format. All strings are UTF8
encoded. This is to ensure cross platform compatibility.

Bug: b/312243587
Test: atest aconfig.test
Change-Id: I1041405db42862573ec320c0e557948732c28eb8
2023-12-19 17:20:27 +00:00
Zhi Dou
e8a4ee7288 aconfig: add exported mode in rust codegen
This commit adds exported mode to rust codegen.

When the codegen mode is exported
1. only flags with exported: true will be generated
2. the generated getter should be in a read_write format and with
   default value as false, regardless the original permission and state
   of the flag

This commit adds integration test for rust codegen exported mode.

Bug: 316357104
Test: atest aconfig.test aconfig.test_mode.test.rust aconfig.exported_mode.test.rust
Change-Id: Ib7dae666e13eb8898289b06d42a4f89326e175c4
2023-12-19 16:58:06 +00:00
Zhi Dou
2909666ae4 aconfig: add c/c++ codegen exported mode integration test
This commit adds c/c++ codegen exported mode integration test

Bug: 316357680
Test: atest aconfig.test.cpp.exported_mode

Change-Id: Iadea7ba2459eb3e80c7ffc7dea83cc9c386798e5
2023-12-19 16:40:53 +00:00
Zhi Dou
b52465da31 aconfig: add new testing flag enabled_fixed_ro_exported
This commit adds a new testing flag enabled_fixed_ro_exported to test
the case of a exported and fixed_read_only flag.

Test: atest aconfig.test aconfig.test.java AconfigJavaHostTest
aconfig.test.cpp aconfig.test.cpp.test_mode aconfig.prod_mode.test.rust
Bug: 316357680

Change-Id: Iaedb8a6875166c6a6d24c7c3deee701a496b4964
2023-12-19 16:40:53 +00:00
Zhi Dou
c1d0a14b69 aconfig: add exported mode in c/c++ codegen
This commit adds exported mode to c/c++ codegen.

When the codegen mode is exported
1. only flags with exported: true will be generated
2. the generated getter should be in a read_write format and with
   default value as false, regardless the original permission and state
   of the flag

This change moves process parsed_flags funciton into codegen crate. The
process function is used to process parsed_flags based on the codegen
mode.

The template has also modified to keep readability.

Bug: 316357680
Test: atest aconfig.test aconfig.test.cpp aconfig.test.cpp.test_mode

Change-Id: I511e05ab93b07a04236055d956d1926f4ed89f36
2023-12-19 16:35:13 +00:00
Mårten Kongstad
009a494572 aconfig: add support for dump --filter='{fully_qualified_name}:...'
Add support for filtering by fully qualified flag name. (This should
have been part of aosp/2878034.)

Bug: 315487153
Test: atest aconfig.test
Change-Id: I4cd209d6f6940dcaa138e8e0b8e580cea99c350f
2023-12-19 10:24:35 +01:00
Dennis Shen
d555bbfa80 aconfig: update c/c++ codegen
1, Current codegen would still include an empty flag value cache even if
there are no read write flags. This is important as the flag value cache
is a std::vector. Thus adding this cache would automatically pull in
libc++ dependency which we should not.

2, There lacks of c/c++ codegen unit tests to lock down the generated
code when a flag package contains only read only flags. It should not
include server_configurable_flags header. And it should not include the
flag cache.

Bug: b/316614694
Test: atest aconfig.test
Change-Id: Iee4366672932ee507694f5fc73b526c66fcf0e9b
2023-12-18 13:43:22 +00:00
Ludovic Barman
8af1ac2a90 Remove unused use_protobuf3 flag
This is a No-op.
Protobuf3 is used regardless of this flag since https://android-review.git.corp.google.com/c/platform/build/soong/+/2817733

Test: m rust

Bug: 308790516
Change-Id: Ib84a01b76a2998249a836fb0278602a5229bd242
2023-12-16 10:56:26 +00:00
Mårten Kongstad
517ac4801b aconfig: dump --filter: implement predicates
The aconfig dump command can now limit which flags to print by passing
in one or more --filter=<query> commands.

If multiple --filter arguments are provided, flags that match any filter
will be included in the output.

The <query> syntax is <what>:<value>, where <what> is the name of a
ProtoParsedFlag field. Multiple queries can be AND-ed together by
joining them with a plus ('+') character.

Example queries:

  - --filter='is_exported:true' # only show exported flags
  - --filter='permission:READ_ONLY+state:ENABLED' # only show flags that are read-only and enabled
  - --filter='permission:READ_ONLY' --filter='state:ENABLED' # only show flags that are read-only or enabled (or both)

Current limitations that may be addressed in future CLs:

  - No support to invert a query, e.g. "flags *not* in the following
    namespace"

  - No support to quote strings; this makes description matching
    difficult

  - No support to glob strings, only exact matches are considered

  - No support to filter by description, trace or metadata fields

Bug: 315487153
Test: atest aconfig.test
Test: printflags --format="{fully_qualified_name}={state}" --filter=permission:READ_ONLY # manually verify output
Change-Id: Ie1e40fa444cec429e336048439da955f30e22979
2023-12-15 17:30:48 +01:00
Mårten Kongstad
49e4d6e73d aconfig: dump --filter: hook up command line args to dump.rs
This is the first step towards teaching dump to (optionally) filter
which flags to print.

A follow-up CL will implement dump::create_filter_predicate.

Bug: 315487153
Test: atest aconfig.test
Change-Id: Ibe0d4ce6563d3b5718fedd3ebfd45fbf5d935b92
2023-12-15 15:23:31 +01:00
Mårten Kongstad
ee58f98cc0 aconfig: support custom dump format specs
Teach `dump --format=<arg>` to format the output according to a
user-defined format string. The format string now accepts these
arguments:

  - "protobuf": output all data as binary protobuf (as before)
  - "textproto": output all data as text protobuf (as before)
  - any other string: format according to the format spec, see below

Custom format spec: placeholders, enclosed in { and } and named after
the fields of ProtoParsedFlag, will be replaced by the actual values.
All other text is output verbatim. As an example:

  - "{name}={state}" -> "enabled_ro=ENABLED"

Some fields support an alternative formatting via {<field>:<format>}. As
an example:

  - "{name}={state:bool}" -> "enabled_ro=true"

Note that the text replacement does not support escaping { and }. This
means there is no way to print the string "{name}" without expanding it
to the actual flag's name. If needed this feature can be introduced in a
later CL.

For backwards compatibility, the following format strings have special
meaning and will produce an output identically to what it was before
this change:

  - "text"
  - "verbose"
  - "bool"

A follow-up CL will add a new `dump --filter=` argument to limit which
parsed flags are included in the output.

Test: atest
Bug: b/315487153
Change-Id: If7c14b5fb3e7b41ea962425078bd04b4996318f4
2023-12-15 08:31:51 +01:00
Atneya Nair
a3a6bfb4a0 aconfig: Print flags with namespace, and sort them
For readability/convenience when looking at BRs/dev.

Test: adb shell printflags
Bug: 315402442
Change-Id: Ia8c447d99f6fe0a9ed6a33c4c68f2e97f5310dc0
2023-12-14 18:02:57 +00:00
Mårten Kongstad
0c6a2fac77 aconfig: pass ownership of ProtoParsedFlag items to Java codegen
Pass ownership of the ProtoParsedFlag iterator items to the Java
codegen: this removes the need for an explicit lifetime annotation.

Bug: N/A
Test: atest aconfig.test
Change-Id: I78705b65b4cf41c3e23b59afbd122a029ee45df4
2023-12-14 13:56:25 +01:00
Mårten Kongstad
49c49f365a aconfig: pass ownership of ProtoParsedFlag items to C++ codegen
Pass ownership of the ProtoParsedFlag iterator items to the C++ codegen:
this removes the need for an explicit lifetime annotation.

Bug: N/A
Test: atest aconfig.test
Change-Id: Ic6606a9ab01ddcb61aa668d7ac901469b1e25a1f
2023-12-14 13:53:52 +01:00
Mårten Kongstad
b7fc0d3e16 aconfig: pass ownership of ProtoParsedFlag items to Rust codegen
Pass ownership of the ProtoParsedFlag iterator items to the Rust
codegen: this removes the need for an explicit lifetime annotation.

Bug: N/A
Test: atest aconfig.test
Change-Id: Ia37ff9f1238684ea31cda6be70acde54fdb04881
2023-12-14 13:43:12 +01:00
Zhi Dou
82df71123a aconfig: CodegenMode::Exported: only generate code for exported flags
If the codegen mode is CodegenMode::Exported, the codegen should not
generate code including any non-exported flags. This change add a
function to filter the parsed flags before feeding to codegen.

Test: atest aconfig.test aconfig.test.java AconfigJavaHostTest
aconfig.test.cpp aconfig.test.cpp.test_mode aconfig.prod_mode.test.rust
Bug: 311152500

Change-Id: Ia88795bbd13774d02c76d8ab7ef20868149599b5
2023-12-13 20:02:32 +00:00
Dennis Shen
728fe988d8 Merge "aconfig: add create storage command" into main 2023-12-12 23:15:59 +00:00
Dennis Shen
0d1c5629f7 aconfig: add create storage command
Add a new aconfig command called create-storage which takes a number
of aconfig cache files that belong to a specific container and produces
storage files.

Add a new module called storage (src/storage/mod.rs) as the entry point
of storage files generation. FlagPackage struct is defined as an
intermediate data structure that will be used to drive all storage files creation.

Add a unit test to lock down FlagPackage creation behaviors.

Bug: b/312243587
Test: atest aconfig.test

Change-Id: Ia7e9f68237ea903f295ac7891c923f6a39f3422d
2023-12-12 21:46:57 +00:00
Ted Bauer
a27d46c590 Merge "aconfig: fix template bug in exported mode." into main 2023-12-12 20:47:19 +00:00
Ted Bauer
98562fded9 aconfig: fix template bug in exported mode.
FakeFeatureFlagsImpl still generated all flags in its mFlagMap field, but it should generate only exported flags in exported mode.

Test: atest aconfig.test.java
Bug: 311152507
Change-Id: I61843cd87b3bb5035772791a5869a91b07d574d8
2023-12-12 14:42:45 -05:00
Mårten Kongstad
0ae74f86ba aconfig: include all tests in TEST_MAPPING
The aconfig tests used to be implicitly run by Treehugger, but something
has changed and Treehugger no longer verifies aconfig uploads. Fix this
by explicitly listing all aconfig tests in the TEST_MAPPING file.

Treehugger does not allow new tests in presubmit before they have proven
themselves (in terms of flakiness and execution speed) in postsubmit.
For this reason this CL adds the tests to postsubmit; a follow-up CL
will move them to presubmit.

This has the added benefit of allowing developers to easily run all
tests locally, either by

  $ atest

if current working directory is build/tools/aconfig or one of its
subdirectories, or

  $ atest --test-mapping $(gettop)/build/tools/aconfig

from anywhere in the Android tree.

Also add all tests to "general-tests" to enable Treehugger to run them.
Move aconfig.test from "device-tests" to "general-tests"; the former
group is intended for tests that depend on device-specific
functionality. See [1] for more info.

1. https://source.android.com/docs/core/tests/development/test-mapping

Bug: N/A
Test: atest --test-mapping $(gettop)/build/tools/aconfig
Change-Id: If857733834c8ad43a16e6162e50e6b1f713f979d
2023-12-12 14:05:15 +01:00
Mårten Kongstad
0bedc1507d aconfig: fix broken test 'aconfig.test.cpp.test_mode:AconfigTest#ResetFlagValue'
The aconfig.test.cpp.test_mode test cases modify the local flag
overrides, but was not resetting the values between tests. This meant
that

  $ atest 'aconfig.test.cpp.test_mode:AconfigTest#ResetFlagValue'

would succeed, but

  $ atest aconfig.test.cpp.test_mode

would fail. Fix this by calling reset_flags() as part of the per-test
setup.

Bug: N/A
Test: atest aconfig.test.cpp.test_mode
Change-Id: Ic8cffff0deb279b7ae103c05642c67230afc040f
2023-12-12 10:29:58 +01:00
Oriol Prieto Gascó
13af7616bf Merge "aconfig: Respect flag naming convention on test flags" into main 2023-12-11 18:45:37 +00:00
Oriol Prieto Gasco
bbeda85949 aconfig: Respect flag naming convention on test flags
Test: atest aconfig.test
Bug: 311152507
Change-Id: I960867d32415007db5dacab57e97b813cbbee650
2023-12-11 18:45:06 +00:00
Dennis Shen
813267d1e7 aconfig: update aconfig cpp integration tests
Note a new cc_test target aconfig.test.cpp.test_mode is added instead of
reusing the old aconfig.test.cpp test target is becuase the test and
production version codegen flag lib has symbol conflict, one should only
link one at a a time.

Bug: b/291068147
Test: atest aconfig.test.cpp/aconfig.test.cpp.test_mode
Change-Id: Ifd52fd6eb7538fba90a13f0a6618a33a69b2358e
2023-12-11 13:20:12 +00:00
Dennis Shen
60eeb8578b Merge "aconfig: update aconfig owners" into main 2023-12-08 19:16:00 +00:00
Dennis Shen
102bec338a aconfig: update aconfig owners
Change-Id: Ie97963c68c789e64495db4ed6e198f17eab412f6
2023-12-08 19:14:17 +00:00
Treehugger Robot
a18d7b17b7 Merge "aconfig: move codegen into separate module" into main 2023-12-08 14:15:51 +00:00
Mårten Kongstad
12e593cd02 aconfig: move codegen into separate module
Consolidate the code generation sources into its own module. This is
done in preparation for when support for the new flag storage will be
added, which should also be placed in a separate module.

Bug: N/A
Test: atest aconfig.test aconfig.test.java
Change-Id: I8246729218ec60f2af4568c460e98329665a89fa
2023-12-08 11:24:54 +01:00
Mårten Kongstad
f3f20efd0c aconfig: consolidate how fully qualified flag name is calculated
Add an extension trait to consolidate how the fully qualified package
name (<package>.<name>) is calculated and use this where possible.

This CL is a semantic change only. The intention of this CL is to reduce
the risk of future bugs.

Bug: N/A
Test: atest aconfig.test
Change-Id: Ibab6641dda3843337fbea02631c31a1fdd8fb4ab
2023-12-08 11:02:32 +01:00
Jihoon Kang
a95f2b230c Add new dump format map in aconfig
The `aconfig dump --format map` lists aconfig keys mapped to its boolean
values that represents whether the flag is enabled or not, as seen below:
```
flag.name1=true
flag.name2=false
```

Test: aconfig dump --format map --cache out/soong/.intermediates/build/make/tools/aconfig/aconfig.test.flags/intermediate.pb && inspect output
Bug: 306024510
Change-Id: Ic4990c168f6fa9c87869113ba695c07394adbc67
2023-12-07 22:36:59 +00:00
Micha Schwab
1a1a08a378 Add metadata to aconfig
Flag metadata will be used both for release strategy, such as allowing faster rollouts for flags, and for test infrastructure such as which platform tests need running.

Doc: go/aconfig-metadata
Test: atest aconfig.test
Change-Id: Idc4b1c6b03c257a1cf92afdfb0b6e54b43741b06
2023-12-05 17:22:19 -05:00
Colin Cross
952df85c69 Merge "Support aconfig dump --dedup" into main 2023-12-05 19:12:11 +00:00
Oriol Prieto Gasco
7afc7e7b1c Add container field to flag_declarations
A container is software which is always built in its entirety using the
same build environment. In particular, all of its parts are built using
the same build-time default flag values. In addition, containers are
always installed as a single unit.

Bug: 312696545
Test: atest aconfig.test && m all_aconfig_declarations
Change-Id: I2ef3db836c4456f4f4fb5c066edf9094e38f89cc
2023-12-05 02:25:01 +00:00
Colin Cross
6befb342fd Support aconfig dump --dedup
Add a flag to aconfig dump that will allow identical flags to be merged
without erroring.  This will allow merging the aconfig cache files from
dependencies for each module, which requires passing less data to Make
from Soong, and thus reduces the percentage of builds require Kati
analysis.

Bug: 313698230
Test: aconfig.test
Change-Id: Id2d9b78225720ce01defad4b13ea9395f6ab1033
2023-11-29 12:04:36 -08:00
Ted Bauer
4a6af78b69 aconfig: Add exported mode to aconfig Java library generation.
This commit adds a third codegen mode, _exported_, in addition to
the existing modes, production and test.

When codegen mode is _exported_, getters are generated _only_ for
flags marked as exported as well. Also the getters always look
up DeviceConfig values at runtime, and have a default value of
false.

This only implements exported mode for Java codegen, follow-up CLs
will support Rust and C++.

Test: atest aconfig.test
Bug: 311152507
Change-Id: Ie39379b40de072180e05d84c76361b24cc0e0d83
2023-11-29 15:44:24 +00:00
Oriol Prieto Gascó
c96bf2caec Merge "Add is_exported field to aconfig.proto" into main 2023-11-22 13:41:09 +00:00
Wonsik Kim
8a64367cb5 Merge "aconfig: move cache_ as a memeber" into main 2023-11-21 17:57:39 +00:00