Commit Graph

27854 Commits

Author SHA1 Message Date
Liz Kammer
57f5b33ad3 Add test suite handling to central androidmk code
MTS is introducing partial MTS test suites that are per-module, with
names of the format: mts-${MODULE}. By centralizing the code for test
suites, we can automatically add "mts" test suite when an
"mts-${MODULE}" test suite is specified, reducing duplication.

Test: m mts
Bug: 170318013
Change-Id: I8ce9d3c252fcc0a937bb5f2826d21cb6c6932d82
2020-12-01 12:23:56 -08:00
Jiyong Park
fdea25781f Merge "rust modules can be included in apex" 2020-11-30 15:41:28 +00:00
Jiyong Park
99644e92c8 rust modules can be included in apex
We will have some APEXes having rust binaries and libraries. So, adding
the support for the types of modules.

rust.Module now inherits from android.ApexModuleBase and implements
the android.ApexModule interface.

Bug: 172414324
Test: m

Exempt-From-Owner-Approval: rebased after +2 from the owner
Change-Id: I356ef4c45f782a6460f001e83af96d1710642d80
2020-11-30 15:40:48 +00:00
Paul Duffin
8abcbdc0b3 Merge "java_sdk_library: Copy removed API specification file to dist" 2020-11-30 13:24:19 +00:00
Paul Duffin
11402caa47 Merge "Remove redundant setting of DistFiles by apexBundle" 2020-11-30 13:24:00 +00:00
Paul Duffin
a67ff601d9 Merge "Remove redundant setting of DistFiles by java.Library" 2020-11-30 13:23:50 +00:00
Paul Duffin
5e95589fe9 Merge "Generalize the handling of dist tags" 2020-11-30 12:42:55 +00:00
Paul Duffin
4326e0e094 Merge "Differentiate between no dist tag and an empty dist tag" 2020-11-30 12:31:53 +00:00
Thiébaud Weksteen
c9b71a446a Merge "Use 1.48.0 prebuilts" 2020-11-30 09:29:23 +00:00
Paul Duffin
040e906fcf java_sdk_library: Copy removed API specification file to dist
Uses the dists property to copy the removed API specification file for
each API scope alongside the API specification file in the dist.

Bug: 158544979
Test: m nothing
      m dist sdk - before and after this change, compare result to
      make sure that there are no significant differences other than
      the after now contains -removed.txt files for each scope of each
      java_sdk_library
Change-Id: If23c594d29a12e49d2c5a8945ad488874bb1590c
2020-11-27 15:17:44 +00:00
Paul Duffin
281bc00bbf Remove redundant setting of DistFiles by apexBundle
A previous change handles dist properties automatically for all module
types and as a result has made the apexBundle setting of DistFiles
redundant so this change removes that.

Test: m nothing
      m dist sdk - before and after this change, compare result to
      make sure that there are no significant differences.
      Test the effect on the apex by following instructions in
      http://b/172951145
Bug: 174226317
Change-Id: If6f741b5b73b3335f4c478a6528f5f311cd633e4
2020-11-27 15:17:44 +00:00
Paul Duffin
620680516d Remove redundant setting of DistFiles by java.Library
A previous change handles dist properties automatically for all module
types and as a result has made the java.Library setting of DistFiles
redundant so this change removes that and the tests that duplicate
tests of the general mechanism.

Test: m nothing
      m dist sdk - before and after this change, compare result to
      make sure that there are no significant differences.
Bug: 174226317
Change-Id: Ib79a3bdd46897efd84a9c456c37c374bd6036303
2020-11-27 15:17:44 +00:00
Paul Duffin
af970a2e7a Generalize the handling of dist tags
Previously in order to make use of dist entries with tag properties it
was necessary for each module type to be modified to store the results
of GenerateTaggedDistFiles(module) in the AndroidMkEntry.DistFiles
property. This change generalizes that mechanism to work across all
module types.

This change does improve the behavior of a couple of tests where
previously a dist with tag property was ignored because tagged dist
files were not available now it is used correctly.

Some module types do not implement OutputFileProducer interface and so
they cannot handle dist with tag properties. This change makes it an
error to attempt to do that. That necessitated adding OutputFiles(tag)
method to customModule in queryview_test.go as some of the tests in
that file used dist entries with tag properties.

Follow up changes will remove the code that was made redundant by this.

Test: m nothing
      m dist sdk - before and after this change, compare result to
      make sure that there are no significant differences.
Bug: 174226317
Change-Id: Ifc54d67db10ce0d0fe8179c05b97a2be2113be4e
2020-11-27 15:17:44 +00:00
Paul Duffin
74f05598eb Differentiate between no dist tag and an empty dist tag
Change https://r.android.com/1335521 added tag property to the Dist
struct so that it could be used to select one of a number of different
output files to copy to the dist instead of the single file that the
module type made available for dist. The output files were selected
by passing the tag to OutputFiles(tag).

Module types that wanted to support this new approach had to explicitly
set AndroidMkEntries.DistFiles = GenerateTaggedDistFiles(module).
Unfortunately, doing that had a side effect of changing the behavior of
dist entries without a tag.

That was because the change treated a tag that was not specified, as
being the same as "". So, prior to the change no tag meant use the
default dist file but after it meant use the paths returned by
OutputFiles(""). That changed the behavior of the java.Library type
which affected the behavior of the android_app module type.

Prior to the change the java_library would make the
Library.outputFile available for dist when no tag was specified. After
that change it would make Library.outputFile plus
Library.extraOutputFiles. The latter is usually empty except for
android_app which adds some extra files into there which will now be
copied to the dist. That change may have been intentional but there
was no mention of it in the change or the bug. Even if it wasn't
intentional it may still be beneficial.

Any module type that wants to add support for tags in dist runs the
risk of introducing similar changes in behavior. This change
differentiates between the tag not being set and the tag being set to
"" to avoid that possibility and to make the default behavior
explicit for those module types that have switched.

It does so as follows:
* Adds a DefaultDistTag constant that is used when the tag is not set.
  It is a string that is unlikely to be used as an actual tag as it
  does not start with a . and uses some special characters.
* The DefaultDistTag is used in MakeDefaultDistFiles(paths) to indicate
  that the supplied paths are the default ones and and also in
  GenerateTaggedDistFiles() for Dist structures that have no tag
  property set.
* The DefaultDistTag is passed to OutputFiles(tag) just in case the
  module type has explicitly defined the paths to associate with that
  tag in there. If it has then it overrides the legacy behavior. If it
  has not then it is just ignored and falls back to using the previous
  behavior.
* The java.Library.OutputFiles(tag) method explicitly handles the
  DefaultDistTag and returns Library.outputFile for it which restores
  the behavior from before the change that added dist.tag support.
* Similar change was made to apexBundle.OutputFiles(tag) in order to
  preserve its previous behaviour.
* The customModule used by TestGetDistContributions has been modified
  to also preserve its previous behavior after this change.

Test: m nothing
      m dist sdk - before and after this change, compare result to
      make sure that there are no significant differences.
      Test the effect on the apex by following instructions in
      http://b/172951145
Bug: 174226317
Change-Id: Ib8f0d9307751cc2ed34e3d9a5538d3c144666f6d
2020-11-27 15:17:44 +00:00
Paul Duffin
62269498be Improve test coverage of dist processing code
This change adds some additional tests to cover the different
combinations of setting OutputFile and DistFiles properties in the
AndroidMkEntry.

These tests will prevent follow up changes from inadvertently changing
the behavior that some modules rely on.

Test: m nothing
Bug: 174226317
Change-Id: I6ab6f2711a5fa416ea2b84a1e0b8754d525d7a0c
2020-11-27 15:17:44 +00:00
Paul Duffin
89968e3dec Expand dist property checks to cover dists
Previously, only the dist property's nested properties were checked for
correctness. This change also checks the dists property's nested dist
structures and adds some tests to verify that the checks are run and
correctly report the location of the incorrect property even when it is
within a slice of dist structs.

Test: m nothing
Bug: 174226317
Change-Id: If5a19360e1e4c98ee3b5afc813e35349d1fc6f6f
2020-11-27 15:17:44 +00:00
Paul Duffin
d83988dbad Remove duplicate tests from TestGetDistForGoals
A previous change duplicated the test cases from TestGetDistForGoals()
to test the getDistContributions() method. This change removes the
duplicate tests and leaves a single test to verify that
GetDistForGoals() uses getDistContributions() and
generateDistContributionsForMake() correctly.

Test: m nothing
Bug: 174226317
Change-Id: I545501016b932f9af0d5fda2f452a3a09932d3fb
2020-11-27 15:17:44 +00:00
Paul Duffin
8b0349c652 Separate dist processing from make specific output
Previously, the GetDistForGoals(Module) func combined the processing
of the dist properties with generating the make specific rules for
generating that dist. That has a couple of problems:
1. It combines two pieces of functionality into one method which is
   bad practice.
2. It makes it hard to test because the make specific output ends up
   containing absolute paths to temporary directories created by the
   test.
3. It makes switching to a non-make output difficult and fragile as
   changing the output will also require changing the tests.

This change adds an intermediate data structure to contain the result
of the dist processing. That processing is done by the new method
getDistContributions(Module) which returns the new intermediate
structure. It also adds generateDistContributionsForMake(..) to
generate the make output. The GetDistForGoals(Module) func uses them to
implement the previous behavior.

It adds identical tests to those in TestGetDistForGoals() but leaves
those tests alone to show that this refactoring does not change the
behavior. Follow up changes will clean up TestGetDistForGoals(). It
also adds a test for generateDistContributionsForMake(..).

Bug: 174226317
Test: m nothing
      m dist sdk - before and after this change, compare result to
      make sure that there are no significant differences.
Change-Id: I458b7c8e4485bf66d3498f50df85a8d65fc2ee00
2020-11-27 15:17:38 +00:00
Thiébaud Weksteen
c6fe19d637 Use 1.48.0 prebuilts
Bug: 173721343
Test: build external/rust/ and crosvm
Change-Id: Ifde24640fc262c5eebdcd623556e925e7d78ae85
2020-11-27 14:42:57 +01:00
Paul Duffin
3e9614198a Merge "Extract method to process bp file for androidmk_test.go" 2020-11-27 11:53:46 +00:00
Ulyana Trafimovich
af07f732e6 Merge "Assume any <uses-library> is shared, add only toplevel ones to manifest." 2020-11-27 10:44:54 +00:00
Paul Duffin
103aaae715 Extract method to process bp file for androidmk_test.go
Test: m nothing
Bug: 174226317
Change-Id: Ifc9ee5088fe766d971718fc112d1e95719a2b825
2020-11-27 01:24:02 +00:00
Treehugger Robot
9c4c534d25 Merge "Documenting apex/builer.go" 2020-11-27 00:55:40 +00:00
Paul Duffin
d35d92a7b5 Merge "Improve TestGetDistForGoals debuggability" 2020-11-26 17:16:53 +00:00
Thiébaud Weksteen
463ab7a27e Merge "Ignore variant dependencies in rust-project.json" 2020-11-26 09:15:09 +00:00
Treehugger Robot
5bbec1a371 Merge "Remove hardlink optmization in sbox" 2020-11-26 00:20:01 +00:00
Liz Kammer
0fd0e95e1b Merge "Refactor rust to use AndroidMkEntries" 2020-11-26 00:06:03 +00:00
Liz Kammer
c55dcfcc2b Merge "Refactor python to use AndroidMkEntries." 2020-11-26 00:05:52 +00:00
Liz Kammer
6953adeb95 Merge "Convert csuite config to use AndroidMkEntries" 2020-11-26 00:05:30 +00:00
Treehugger Robot
aa8ac24222 Merge "Soong: add -T to nanopb call to suppress timestamp" 2020-11-25 23:47:20 +00:00
Ulf Adams
82fd89b8ab Soong: add -T to nanopb call to suppress timestamp
By default, nanopb outputs a timestamp into the .c and .h files it generates.
This is usually considered bening. However, when using remote execution or
remote caching, this poisons all actions that have these files as inputs, and
also appears to affect transitive actions (actions depending on actions that
have these files as inputs).

This change adds the -T flag to suppress the timestamp, which in turn makes
the output files deterministic, which in turn allows the remote cache to
return previous results rather than requiring a re-execution of the
corresponding actions.

Test: ran the build a few times to check that the output is deterministic

Change-Id: Iecfa143dbf12a9bb7703dd0bc3f1b3e3fd56317c
2020-11-25 22:52:12 +01:00
Colin Cross
b0ce5757b9 Merge "Use a unique depfile for each gensrcs command" 2020-11-25 21:18:13 +00:00
Colin Cross
d03797e48a Remove hardlink optmization in sbox
The hardlink optimization attempts to save I/O by hardlinking input
files into the sandbox instead of copying them, but it never works on
a Linux build due to nsjail always making out and the source tree into
separate bind-mounted filesystems.  If the optimization did work it
would actually cause build failures when it hardlinked the relative
bionic/libc/fs_config_generator.py symlink into the sandbox directory
without the target.  Just remove the optimization for now since it
never works.

Test: m checkbuild
Change-Id: I96e8c0c145e7c99958639594edf8a93b69ae90eb
2020-11-25 18:28:58 +00:00
Paul Duffin
0cc047ad5d Improve TestGetDistForGoals debuggability
Previously, TestGetDistForGoals tested multiple test cases within a
single test so when it failed it was difficult to determine which test
case was the cause. This change runs each test case as its own
nested test.

It also corrects the order of expectedLine and line format parameters
to match the order in the message.

Test: m nothing
Bug: 174226317
Change-Id: I1408ec4125afc5c0b392cd7643dd3f630fe468e5
2020-11-25 17:02:32 +00:00
Thiébaud Weksteen
3c5905b0cb Ignore variant dependencies in rust-project.json
Source-generated library modules will have a dependency on the source
variant. This creates a cycle in the dependency graph which triggers a
warning from rust-analyzer. Ignore this type of dependency.

Test: m nothing
Change-Id: I13365093ebb88b00f6a72734b01114ec3e9a320e
2020-11-25 16:11:49 +01:00
Ulya Trafimovich
78a7155c17 Assume any <uses-library> is shared, add only toplevel ones to manifest.
This patch reworks the approach introduced in
https://r.android.com/1450819. That patch based the decision which
libraries should be added to the manifest <uses-library> tags by the
manifest_fixer on the "shared" status of the library.

That approach is incorrect for two reasons:

  - It doesn't make sense to have a non-shared library in class loader
    context ("shared" libraries are those specified in
    frameworks/base/data/etc/platform.xml, and they are the only ones
    that PackageManager knows about).

  - It doesn't make sense to add anything but the top-level of the
    class loader context tree to the manifest, because this part of the
    tree is flattened to a sequence, and PackageManager cannot restore
    it to the previous tree shape (there is an information loss).

This patch removes the "shared" bit of information from class loader
context elements and assumes that all libraries that end up in class
loader context are shared. Consequently, only the top-level libraries
should be passed to manifest_fixer.

Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Bug: 168686456
Change-Id: I902690f0f38f1047fa79cf6ccbe956077eceaab0
2020-11-25 14:47:05 +00:00
Jingwen Chen
65d5b7f0e9 Merge "Comment android/config.go" 2020-11-25 04:18:56 +00:00
Treehugger Robot
b21906cd70 Merge "Minor cleanup of environment.go." 2020-11-25 03:28:33 +00:00
Treehugger Robot
bf6d82f929 Merge "Minor cleanup of cleanbuild.go (and partially build.go)." 2020-11-25 03:27:25 +00:00
Treehugger Robot
c2acb8cbf0 Merge "Add comments about sysprop_library" 2020-11-25 00:54:14 +00:00
Jiyong Park
b81b99032d Documenting apex/builer.go
Mostly documentation changes, but includes a few refactorings like
changing the variable names, reordering functions, reordering statements
in logical order, etc.

Bug: 173472337
Test: m
Change-Id: Iefdc8e463bcda60187b98e8c90661e220e8cdf40
2020-11-25 09:53:46 +09:00
Colin Cross
3ea4eb8d35 Use a unique depfile for each gensrcs command
gensrcs modules run the same command once for each input file to
produce each output file.  Each command needs its own depfile instead
of reusing one per shard.  The depfiles will be merged together
by RuleBuilder into one depfile per shard.

Test: TestGenSrcs
Change-Id: Iaf4f2cf9f5592c20e32944ddf34e0a61aff17ba8
2020-11-24 15:05:28 -08:00
Liz Kammer
e2a17a5a40 Convert csuite config to use AndroidMkEntries
This is to facilitate consolidation of compatibility suite logic.

Test: m nothing and compare soong Android mk files -- no diffs
Change-Id: Ibc60d14fdde93835124e18a5c484bbda496bdb9e
2020-11-24 13:41:28 -08:00
Christopher Parsons
dc71a6e28a Merge "Improve commenting of cc/binary.go." 2020-11-24 20:34:39 +00:00
Colin Cross
38214f5ac5 Merge changes I8af00c8c,Ia526ae89
* changes:
  Reland: Rewrite sbox to use a textproto manifest
  Fix gensrcs subdirs
2020-11-24 19:27:35 +00:00
Liz Kammer
d8dceb0f2d Refactor python to use AndroidMkEntries.
This is to enable consolidating test compatibility suite handling in
Soong's AndroidMk files.

Test: go test soong tests
Test: m nothing and compare soong/Android mk files -- only diffs is that
      lines matching "^LOCAL_SHARED_LIBRARIES := $" are absent
Change-Id: I4922a1c8b0231cd98ebd2bc7257956eb9c0d240c
2020-11-24 10:10:03 -08:00
Liz Kammer
4018a8d4d7 Refactor rust to use AndroidMkEntries
This is to facilitate consolidation of compatibility suite logic.

Test: go soong tests
Test: m nothing & compare soong Android mk files -- only diffs are
      elimination of duplicate LOCAL_UNINSTALLABLE_MODULE := true lines
Change-Id: I76211505b2b6afde5d0431586b77d799cf70c6b9
2020-11-24 09:44:51 -08:00
Colin Cross
e5218b6be6 Merge "Annotate dependency tags for dependencies of installed files" 2020-11-24 16:01:35 +00:00
Rupert Shuttleworth
1f304e674d Minor cleanup of cleanbuild.go (and partially build.go).
Test: TH presubmit.
Change-Id: I439c16687b1877d367ff3d0f27e30c9bac97ef58
2020-11-24 14:50:25 +00:00
Rupert Shuttleworth
80115fa6b0 Minor cleanup of environment.go.
Test: TH presubmit.
Change-Id: I89415d6b1219d2204b4fcce4b3a7456250cd0967
2020-11-24 13:16:30 +00:00