Commit Graph

71 Commits

Author SHA1 Message Date
Dennis Shen
e398144643 aconfig: cpp codegen update: Move server_configurable_flags header include
Move server_configurable_flags header include away from exported header,
instead put it in the flag provider headers. Otherwise, it can cause
compilation error if the source code who wants to use the generated c flag lib has
not added dependency on server_configurable_flags yet.

Bug: b/279483801
Test: atest aconfig.test
Change-Id: Ib75877ee88f995caf72b3fd2554c3da195032c14
2023-07-10 18:19:44 +00:00
Mårten Kongstad
206a382d4e aconfig: improve duplicate flags error message: include paths
Improve the error message returned when `aconfig dump` is fed multiple
declarations of the same flag: include the paths to the declaration
files.

In general all error messages from the protos::*::verify_* functions
should include paths to the offending files. This will be handled in a
follow-up CL.

Bug: 290300657
Test: atest aconfig.test
Test: manual: add duplicate flag and run `m all_aconfig_declarations`, inspect error message
Change-Id: I46dc23f7128dd5c68ced9f2e8518cfa89d81c2df
2023-07-07 08:52:52 +02:00
Wenshan Fu
a2d93ae1eb Merge "aconfig: create java libraries for aconfig proto" 2023-07-06 01:51:08 +00:00
Dennis Shen
8d544f7439 cpp codegen redesign, unit test support
cpp codegen iteration 2, based on discussions with three internal teams
that use c++. Refer to the design doc "aconfig c++ codegen" for detailed
design. At a high level, we generate two sets of code artifacts with the
same signatured api: one for production that without any local flag
override capability, one for unit test that allows local flag overrides.
It supports static methods style interface as well as injection pattern.

Refer to the test points in the codegen_cpp.rs for examples of generated
code.

for production target codegen: aconfig create-cpp-lib --cache <cache> --out <out dir>
for test target codegen: aconfig create-cpp-lib --cache <cache> --out
<out dir> --mode test

Bug: b/279483801
Test: atest aconfig.test
Change-Id: I92fefb9623d5435525339a74f57bbd36d0afef08
2023-07-05 14:05:44 +00:00
Wenshan Fu
89467c3109 aconfig: create java libraries for aconfig proto
Host side test infra needs the library to consume the dumped flags info
generated by aconfig.

Test: m libaconfig_java_proto_lite libaconfig_java_proto_full
Bug: 289906970
Change-Id: I9891cb8445b90f7269e0c679ceaae6bc102bbb0c
2023-07-05 13:57:19 +00:00
Wenshan Fu
e0b709b9a6 Merge "Create the TEST_MAPPING file" 2023-07-04 00:44:05 +00:00
Zhi Dou
7734879e69 Merge changes I4bf1d3ba,I33605054
* changes:
  aconfig: Add resetAll method to test mode FeatureFlagsImpl
  aconfig: remove extra new lines and spaces in generated java code
2023-06-30 13:23:28 +00:00
Wenshan Fu
13be0a5f98 Create the TEST_MAPPING file
Add FlagAnnotationTests in presubmit to ensure that test filtering rule
always works as expected.

Test: atest --test-mapping
Bug: 277819423
Change-Id: I7cc3a70495f8d62ed7a5b26edc4fd240e73ccf61
2023-06-30 03:28:02 +00:00
Zhi Dou
8d27cc39b1 aconfig: Add resetAll method to test mode FeatureFlagsImpl
Add resetAll method to test mode FeatureFlagsImpl. This method
is used to reset all the flags values to null. It provides convenient
way to the test tools to reset the flags values.

Bug: 280833463
Test: atest aconfig.test
Change-Id: I4bf1d3ba69ee106ef8d0c1cc62c00bbeca1b72aa
2023-06-29 15:15:32 +00:00
Zhi Dou
a41cc5ecf9 aconfig: remove extra new lines and spaces in generated java code
Change the java template to remove the extra new lines and spaces
in generated java code.

Bug: 280833463
Test: atest aconfig.test
Change-Id: I336050540f8a2b0966bf21cf6e034a8fa701458e
2023-06-29 15:05:18 +00:00
Mårten Kongstad
19776d19e6 aconfig: package fields must contain at least one dot char
Introduce a new requirement on package fields: a package must contain at
least one dot character.

Bug: 289336036
Test: atest aconfig.test
Change-Id: Idadcd2a76783a484cc5c6d6e94778c0248fa475f
2023-06-29 10:38:02 +02:00
Mårten Kongstad
c0d618c3a4 aconfig: fix incorrect check in create_device_config_ident
Use is_valid_name_ident to check the flag name, not
is_valid_package_ident.

Bug: 283910447
Test: atest aconfig.test
Change-Id: I3718e7de565b975a1696190e8effbcb077e5fde2
2023-06-29 10:18:20 +02:00
Mårten Kongstad
60ee8a633f aconfig: improve package identifier test readability
Rearrange the tests in test_is_valid_package_ident to make it more
apparent what constitutes a valid package pattern, and what doesn't.

Bug: 283910447
Test: atest aconfig.test
Change-Id: I3d2b87aed008d0ed3a0aa6e483e655178dda7277
2023-06-29 10:03:38 +02:00
Zhi Dou
8ba6aa71b1 aconfig: add java codegen test mode
Add java codegen test mode. The test mode will generate Flags.java and
FeatureFlagsImpl.java differently.
    * Flags.java will have getter and setter function to switch the
      FeatureFlagsImpl. Flags.java will not initialize the instance
      of FeatureFlagsImpl during initialization, thus it will force the
      user to set up the flag values for the tests.
    * FeatureFlagsImpl removes the dependency on DeviceConfig, and
      allows the caller to set the values of flags.

Command changes
This change adds a new parameter `mode` to `create-java-lib` subcommand.
The default value of `mode` is production, which will generate files for
production usage, and keeps the same behavior as before.

The new `mode` test is added to trigger the test mode. The command is
aconfig create-java-lib --cache=<path_to_cache> --out=<out_path>
--mode=test

Test: atest aconfig.test
Bug: 288632682
Change-Id: I7566464eb762f3107142fe787f56b17f5be631b7
2023-06-27 18:57:35 +00:00
Mårten Kongstad
3228b29cac aconfig: add dump --format=textproto
Teach `aconfig dump` to print its as text protobuf.

Bug: 288713733
Test: atest aconfig.test aconfig.test.java
Change-Id: Id31ce53226162a563fbe2d4841a8f6cf96c22436
2023-06-26 10:17:42 +02:00
Mårten Kongstad
37cb74b3e4 Merge "aconfig: add proto bug field" 2023-06-26 06:27:48 +00:00
Mårten Kongstad
1b8636bd7d aconfig: add proto bug field
Add a `bug` field on the flag_declaration and parsed_flag proto
messages. This field is optional in the sense that it can occur zero or
more times, and aconfig will simply pass any value through.

Bug fields are included in the aconfig dump format, which can be
processed by other tools.

Also unify how protos.rs checks that fields marked 'optional' in the
proto file, but in practice are 'required', are actually set.

Test: atest aconfig.test aconfig.test.java
Bug: 288261336
Change-Id: I93de0005674822c6ff4d699bdc2c6509763a7f7f
2023-06-22 16:50:33 +02:00
Joe Onorato
6aa48f8c5f Rename device_config --> aconfig and definitions --> declarations
Bug: 285303012
Test: for x in next trunk trunk_food trunk_staging ; do lunch aosp_panther-$x-eng ; m nothing ; done
Change-Id: I174ce3e609fa9077ea6bc61679ddb83e8efe57c8
2023-06-21 15:18:15 -07:00
Mårten Kongstad
a2e5ab82c7 aconfig: make proto fields optional
Change all required proto fields to optional. While the proto file is
supposed to be a backwards compatible API, and fields are not supposed
to be deprecated, this commit will allow for that option if needed.

Implementation wise this change doesn't matter much: any parsed data
needs additional verification outside what the protobuf crate's parser
provides anyway, so adding checks to verify that all required fields,
even though marked optional in the proto file, were found is a minor
increase in code complexity.

If in the future a proto field should no longer be used:

  - keep the field in the proto, still marked optional and clearly
    document that it is no longer in use
  - change protos.rs from checking struct.has_field() to explicitly
    dropping any value via struct.clear_field()

Bug: 286337317
Test: atest aconfig.test
Change-Id: Iad1ccfe50ecac286ff7a796aec909bec70b9520d
2023-06-19 16:53:22 +02:00
Mårten Kongstad
a2e152a139 aconfig: generate Java flag name constants
Generate Java constants for use in @FlaggedApi(flag = ...).

Also update the generated Java code to use the constants when reaching
out to DeviceConfig instead of hard-coding (duplicate) strings.

Bug: 285288440
Test: atest aconfig.test aconfig.test.java
Change-Id: I1127cacba650cc7a7896b1533e03631d7f5ec71b
2023-06-19 16:12:01 +02:00
Mårten Kongstad
403658f9cb aconfig: use proto struct directly
Remove the hand-crafted wrappers around the structures auto-generated
from protos/aconfig.proto, and use the auto-generated structs directly
intead. This gets rid of a lot of manual repetition, and its inherent
risk.

Also unify how individual fields read from text proto are verified (e.g.
is the flag.name field a valid identifier).

Also change the intermediate cache format from JSON to binary protobuf.

The concept of a 'cache' as an intermediate internal format to represent
parsed input stays. The command line interface still refers to caches.
At the moment a cache file is identical to a parsed_file protbuf, and
the code exploits this internally.

A couple of points regarding the auto-generated structs:

  - Vectors are named in the singular (e.g. parsed_flags.parsed_flag is
    a Vec<ProtoParsedFlag>) because this improves ergonomics for all
    devs working with aconfig input files

  - The auto-generated structs have fields that are of type Option<T>
    and convenience methods (named the same as the fields) to access T

Test: atest aconfig.test aconfig.test.java
Bug: 283910447
Change-Id: I512820cc4bc6c543dea9f6a4356f863120a10be3
2023-06-19 16:04:32 +02:00
Treehugger Robot
a99ac90eb5 Merge changes from topic "aconfig-prepare-for-proto-structs"
* changes:
  aconfig: fix incorrect source path in test cache
  aconfig: reduce number of #[cfg(feature = "cargo")] uses
2023-06-19 10:43:34 +00:00
Zhi Dou
af81e20653 aconfig: change java flag method name to camelCase
Before java code will directly use the flag name as the method name.
This change adds funciton to try the best to convert flag name to
camelCase, and then use the camelCase string as the method name in the
generated code.

Bug: 279483816
Test: atest aconfig.test aconfig.test.java
Change-Id: I45fc6df46c9d535cd38a657a41313202f9b660af
2023-06-16 12:40:40 +00:00
Mårten Kongstad
0cd8092376 aconfig: fix incorrect source path in test cache
Bug: 283910447
Test: atest aconfig.test
Change-Id: I9bc34e838a0945891f866e9788bcf9f4f6c23d3b
2023-06-15 11:43:33 +02:00
Mårten Kongstad
f94225266f aconfig: reduce number of #[cfg(feature = "cargo")] uses
Group statements with identical #[cfg(feature = "cargo")] attributes in
the same block. This reduces repetition and makes the code easier to
read and less error prone.

Bug: 284779868
Test: atest aconfig.test
Change-Id: Iebdcd20e7cd22cb641424a1af594f5c9ac57b623
2023-06-15 09:28:49 +02:00
Zhi Dou
06377d79ab Merge "aconfig: Java codegen iteration 1" 2023-06-14 13:21:16 +00:00
Zhi Dou
4655c967e1 aconfig: Java codegen iteration 1
This change includes
    - refactor generated java code to generate
        - Flags.java to support the static API
	- FeatureFlagsImpl.java to support injection API
	- FeatureFlags.java interface

Bug: 279483816
Test: atest aconfig.test aconfig.test.java
Change-Id: If0d4baf317b9174635cd0fff3832ab7091ee52ed
2023-06-13 14:45:53 +00:00
Mårten Kongstad
d18c978f31 aconfig: reject consecutive underscores in identifiers
The Java codegen may translate flag names (snake_case) to Java
camelCase, dropping the underscores. The flags a_b and a__b will
translate to the same camelCase form, which is ambiguous.

Circumvent this problem by disallowing consecutive underscores in flag
names, flag namespaces, and packages.

Bug: 284252015
Test: atest aconfig.test
Change-Id: I2586a38160723c06265a140193da8178655553e4
2023-06-13 13:30:58 +02:00
Dennis Shen
f1cd83bece Merge "aconfig: update cpp codegen to use static methods" 2023-06-12 12:57:57 +00:00
Dennis Shen
4f78f10015 aconfig: update cpp codegen to use static methods
Update c++ codegen to static methods interface.

Bug: b/279483801
Test: atest aconfig.test
Change-Id: I78da3bbca6240bee660c692807930d00f2242b0a
2023-06-12 12:57:38 +00:00
Joe Onorato
8b51859317 Fix build breakage
error: build/make/tools/aconfig/Android.bp:69:1: module "aconfig.test.java" variant "android_common": path dependency ":aconfig.test.flags{.srcjar}": unsupported device_config_definitions module reference tag ".srcjar"

Test: m aconfig.test.java
Change-Id: I3df980b06796c90dafee3940fe52809748305950
2023-06-09 13:39:41 -07:00
Mårten Kongstad
9c59c31499 aconfig: add Java integration tests
Add integration tests for Java. This test setup verifies that

  - the build system calls aconfig to generate a Java library
  - the Java test compiles against the auto-generated library
  - the auto-generated code returns expected values

Similar integration tests for C++ and Rust will be added in follow-up
CLs.

Note: the build does not currently support specifying that
tests/*.values should be applied, so the test flags will all be assigned
the defaults. A later CL will fix this.

Bug: b/283911467
Test: atest aconfig.test aconfig.test.java
Change-Id: Ia365e209261f4935a23e2dac9ef0ab5b60f76e52
2023-06-09 09:59:21 +02:00
Mårten Kongstad
202102f7be aconfig: include namespace in create-device-config-defaults
Update the output format of create-device-config-defaults to include the
flag's namespace. Also change the delimiters. The new format is

  <namespace>:<package>.<flag-name>=[enabled|disabled|

Bug: 285468565
Test: atest aconfig.test
Change-Id: I9b4ca1611cca8528dc341fc12812b614c86f6c08
2023-06-08 11:28:09 +02:00
Mårten Kongstad
b025507857 aconfig: improve code diffs in tests
Implement a helper function to make it easier for unit tests to diff
(and find the first difference) generated code and expected code.

Bug: 283910447
Test: atest aconfig.test
Change-Id: I460e8fbf05e8f33e8a62ecef67b2d9d77051e876
2023-06-08 11:27:59 +02:00
Mårten Kongstad
066575b95f aconfig: add namespace field to flag_declaration and parsed_flag
Add a new field to the proto messages flag_declaration and parsed_flag.

The new field will be used verbatim as a parameter when calling
DeviceConfig.getBoolean to read the value of a READ_WRITE flag. See the
DeviceConfig API for more info.

Note: not to be confused with the old namespace field, which has been
renamed to package.

Bug: 285211724
Test: atest aconfig.test
Change-Id: I2181be7b5e98fc334e5277fb5f7e386f1fe0b550
2023-06-08 11:27:57 +02:00
Mårten Kongstad
fbd71e2773 aconfig: allow dots in package fields
Allow package fields to include dots.

Update the generated code based on the package name: if the package name
is com.android.example:

  - java: package com.android.example; ...
  - C++: namespace com::android::example { ... }
  - Rust: mod com { mod android { mod example { ... } } }

Also, update examples to use dots in the package fields.

Also, remove unnecessary #include from the auto-generated C++ code: the
header should not include itself.

Bug: 285000854
Test: atest aconfig.test
Change-Id: I8a5352e25c64c34dee0725202a1b7c9957819de8
2023-06-08 11:25:43 +02:00
Mårten Kongstad
9fb58965af aconfig: rename namespace -> package
What used to be referred to as a namespace is now called a package.

This CL is a semantic change only.

Bug: 285000854
Test: m nothing
Test: atest aconfig.test
Change-Id: If3fca67c415af75b44f316e16666b97089407069
2023-06-07 14:53:04 +02:00
Treehugger Robot
2bb714ed2b Merge changes from topic "aconfig-gen-device-config-files"
* changes:
  aconfig: add create-device-config-sysprops command
  aconfig: add create-device-config-defaults command
  aconfig: add test utilities
  aconfig: cache.rs: remove unnecessary use statements
  aconfig: give commands ownership of all arguments
2023-06-07 09:50:23 +00:00
Mårten Kongstad
c31a6ff653 aconfig: add create-device-config-sysprops command
Add a new "create-device-config-sysprops" command that works like
"create-device-config-defaults" but for system properties.

DeviceConfig is a Java service, and will mirror (some of) its values by
setting system properties in the persist.device_config namespace. Native
code will access DeviceConfig (actually, the system properties) via the
server_configurable_flags library.

The new command writes a file that can be appended to /system/build.prop
to pre-populate persist.device_config before DeviceConfig has started.

Like create-device-config-defaults, the new command skips READ_ONLY
flags.

Bug: 285468565
Test: atest aconfig.test
Change-Id: I311c7c5e0b03dc897b09204137d43cc182324717
2023-06-02 16:47:04 +02:00
Mårten Kongstad
f02734e915 aconfig: add create-device-config-defaults command
DeviceConfig is the backend for READ_WRITE flags.

Add a new command "create-device-config-defaults" to create a file that
DeviceConfig will read to pre-populate its data store on first init.

This will be used to quickly switch flag values during CI tests:
rebuilding and reflashing a device would have the same effect, but would
be costlier. This feature is not expected to be used outside CI tests.

Note: because DeviceConfig only works with READ_WRITE flags, the
generated file excludes READ_ONLY flags.

Bug: 285468565
Test: atest aconfig.test
Change-Id: I4caff1a10647b8da0ce4e3615678993a957a92dd
2023-06-02 16:46:28 +02:00
Mårten Kongstad
83a8760bbc aconfig: add test utilities
Create a test utility function to create a Cache from the files in
testdata/*. A follow-up CL will update the unit tests to use this
instead of creating their own caches.

Bug: 283910447
Test: atest aconfig.test
Change-Id: Ice5064eadb0babde5eb38d292330d213ab136d96
2023-06-02 12:54:22 +02:00
Mårten Kongstad
1cd166cd31 aconfig: cache.rs: remove unnecessary use statements
Remove unnecessary use from the cache::test module: they already covered
by `use super:*;`.

Bug: 283910447
Test: atest aconfig.test
Change-Id: I9e03385629f38180c0f92080c7f097a8e0d9ef69
2023-06-02 12:52:48 +02:00
Mårten Kongstad
b27f2ce436 aconfig: give commands ownership of all arguments
Pass the Cache argument to command::create_<lang>_lib functions by value
instead of by reference, to align with other commands.

The intended ownership flow is as follows:

  - main creates objects based on command line arguments
  - main hands commands ownership of the objects
  - command processes the objects
  - command gives main ownership of any generated output
  - main writes the output to file

Rationale: commands.rs is a unit testable version of main, and to the
rest of aconfig, acts as the top level entry point; main.rs exists only
to parse command line arguments and perform I/O.

Bug: 283910447
Test: atest aconfig.test
Change-Id: I1e1dea7da8ecc2bb6e2f7ee4a6df64562c148959
2023-06-02 12:49:26 +02:00
Dennis Shen
6461673140 aconfig: update rust code gen to use libflags_rust
Bug: b/284096062, b/279483360
Test: atest aconfig.test
Change-Id: I4c96f35807de5cb2a745ba1c653513d9b22a5013
2023-05-31 14:35:59 +00:00
Mårten Kongstad
00cf045c85 aconfig: restrict valid namespace and flag names
The namespace and flag names will be used as identifiers in the
auto-generated code. Place restrictions on what constitutes a valid
name.

Valid identifiers are those that match /[a-z][a-z0-9_]/. aconfig
explicitly does not implement any automatic translation to make names
valid identifiers: this sidesteps potential conflicts such as "foo.bar"
and "foo_bar" mapping to the same name if dots were translated to
underscores.

Bug: b/284252015
Test: atest aconfig.test
Change-Id: I38d005a74311e5829e540063404d1565071e6e96
2023-05-26 17:09:10 +02:00
Mårten Kongstad
f73b963283 aconfig: first iteration of Rust codegen
Add a new `create-rust-lib` command to generate Rust code. The output is
a src/lib.rs file; the build system is assumed to set the generated
crate's name.

For READ_ONLY flags, the generated code returns a hard-coded true or false.

For READ_WRITE flags, the generated code reaches out to DeviceConfig via
the cc_library server_configurable_flags via the
libprofcollect_libflags_rust Rust bindings. The build system is assumed
to add this to the generated crate's dependencies.

Note: libprofcollect_libflags_rust seems generic enough that it should
be moved to an official Rust wrapper for server_configurable_flags. This
is tracked in b/284096062.

Summary of module the built system is assumed to wrap the auto-generated
code in:

  rust_library {
      name: "lib<namespace>_rs",
      crate_name: "<namespace>_rs",
      edition: "2021",
      clippy_lints: "none",
      no_stdlibs: true,
      lints: "none",
      srcs: ["src/lib.rs"],
      rustlibs: [
          "libprofcollect_libflags_rust",
      ],
  }

Also add a set of test input to be used in the unit tests for a more
coherent test strategy. A follow-up CL will migrate the code in
commands.rs, codegen_java.rs and codegen_cpp.rs.

Bug: 279483360
Bug: 283907905
Test: atest aconfig.test
Test: manual: create cache from files in testdata, create rust lib, add to module template above, verify the module builds
Change-Id: I02606aa3686eda921116e33f7e2df8fd1156a7aa
2023-05-25 16:18:59 +02:00
Mårten Kongstad
993111f91c aconfig: improve dump --format=debug output
Use Rust's {:#?} formatter for more readable output.

Bug: 279485059
Test: atest aconfig.test
Change-Id: I127f413e3d7aebfba96cad1dd58d9e261dd613a4
2023-05-24 14:56:45 +02:00
Mårten Kongstad
2f95444281 aconfig: sort items in cache by name
Introduce a builder pattern for constructing a cache from flag
declarations and flag values. Teach the builder to sort the flags by
name as the last step. This will ensure consistent dump output
regardless of the order flags are specified in the input files.

Bug: 279485059
Test: atest aconfig.test
Change-Id: Icdd62f51fa3761a469663f17581a83d9909e9ffe
2023-05-22 10:01:05 +02:00
Mårten Kongstad
af677038b3 aconfig: dump: support multiple caches
Teach `aconfig dump` to read multiple caches at the same time.

A cache represents a single namespace, and the Android build creates
multiple caches for different namespaces. By passing in all those cache
files to `aconfig dump`, aconfig will create a complete overview of all
flags in the Android tree.

Caches are traversed in order of namespace to produce the same output
regardless of the order the cache files are given on the command line.
If two caches use the same namespace, their order with respect to each
other is undefined.

Bug: 279485059
Test: atest aconfig.test
Change-Id: I54c3950bbb7b2be7d96c8928e78ae83aa626c2e2
2023-05-22 09:57:48 +02:00
Treehugger Robot
4266df817d Merge "aconfig: Add first iteration of cpp codegen to aconfig" 2023-05-17 19:37:28 +00:00