Commit Graph

38223 Commits

Author SHA1 Message Date
Paul Duffin
280bae6d20 Filter blocked entries from modular flag files
Previously, the sdk snapshot would include all the entries from the
stub-flags.csv and all-flags.csv modular files generated by a single
bootclasspath_fragment. That included implementation details, i.e.
class members that are not part of a stable API or the hidden API which
meant that the sdk snapshots were implementation dependent.

This change removes the implementation details from the modular flag
files, i.e. those entries that are only marked as "blocked". When
comparing the files against the corresponding subset of the monolithic
files it assumes that any entries missing from the modular flag files
are blocked.

Bug: 194063708
Test: atest --host verify_overlaps_test signature_patterns_test
      m out/soong/hiddenapi/hiddenapi-flags.csv
      - manually change files to cause difference in flags to check
        that it detects the differences.
Change-Id: I6b67b2253cf029d6830b58a06ebb0c8fcaa0dd71
2021-08-11 17:29:47 +01:00
Paul Duffin
726d44a86d Merge "Remove member signature and inner classes from signature-patterns.csv" 2021-08-11 09:08:01 +00:00
Treehugger Robot
1dde1fade7 Merge "[scripts] Print full clang prebuilt dir from get_clang_version.py" 2021-08-11 06:57:27 +00:00
Lukács T. Berki
3eff609379 Merge "Make multiproduct_kati call soong_ui.bash ." 2021-08-11 06:12:06 +00:00
Treehugger Robot
95048e9625 Merge "Removed commented out code" 2021-08-11 05:30:45 +00:00
Colin Cross
0bcba80823 Merge "Add symbols for start and end of embedded linker" 2021-08-11 02:28:38 +00:00
Christopher Parsons
9f7c388337 Merge "Support rtti in bp2build" 2021-08-10 20:05:44 +00:00
Yabin Cui
39ff202748 Merge "Revert^2 "Switch to clang r428724 13.0.1."" 2021-08-10 19:22:15 +00:00
LaMont Jones
b02043b84d Merge "Improve handling of errors from zipinfo" 2021-08-10 18:12:27 +00:00
Chris Wailes
90c4b14a30 Removed commented out code
A previous CL commented out code instead of removing it.  This CL fixes
this grave error.

Test: TH
Change-Id: I1daeaaa1ad90230aec639d603e7cdb9b61af251a
2021-08-10 10:21:02 -07:00
Romain Jobredeaux
d9eb0f0aa6 Merge "Exempt prebuilts/jdk and build/bazel/examples/java BUILD files from bp2build overwriting" 2021-08-10 16:29:35 +00:00
Treehugger Robot
6fd3047115 Merge "Support exporting device arch toolchain info" 2021-08-10 16:11:52 +00:00
Chris Parsons
2c7883941e Support rtti in bp2build
Rtti was already handled by bazel macros, so this change simply
propagates the rtti bit to these macros.

Test: Run bp2build, build //external/libcxxabi:all
Change-Id: I63296db2db868202874c8bd0b1de6310f7bb85c7
2021-08-10 11:58:07 -04:00
Yabin Cui
10bf3b8c00 Revert^2 "Switch to clang r428724 13.0.1."
786c1d41ef

Change-Id: Ic143d1b3375c8258e0e4274339524380de391676
2021-08-10 15:42:10 +00:00
Lukacs T. Berki
cef87b62e4 Make multiproduct_kati call soong_ui.bash .
This serves to not link parts of soong_ui (and eventually soong_build)
into a separate, weird binary. This is in turn good because they contain
any number of global variables and no one really thought about what
happens when two instances are executing at the same time in the same
address space.

This comes with a slight performance hit: 5 aosp_* projects build 152
seconds instead of 146. I suppose this is a price worth paying for a
clean design?

Test: presubmits.
Change-Id: I5623dcab2290f0fc392dd2ede597b9794a3d2a4e
2021-08-10 17:35:14 +02:00
Paul Duffin
6ffdff853c Remove member signature and inner classes from signature-patterns.csv
Previously, the signature-patterns.csv file included a lot of
implementation details, e.g. the signatures of dex members or inner
classes that are not part of any API, including the hidden API.

This change will remove all member signatures and inner class names
from the file and replace them with just the outermost qualified class
name.

That will still leave some implementation details, e.g. the names of
implementation only classes and packages.

Bug: 194063708
Test: atest --host verify_overlaps_test signature_patterns_test
      m out/soong/hiddenapi/hiddenapi-flags.csv
      - manually change files to cause difference in flags to check
        that it detects the differences.
Change-Id: I9de6a2a6129e875e19f7ded5fae578cbdb584660
2021-08-10 14:01:51 +01:00
Paul Duffin
8f34b0e19d Merge "Use trie to store monolithic hidden API flags" 2021-08-10 09:26:56 +00:00
Chris Wailes
353405d9fb Clean up generated rust-project.json files
This CL removes the roots section from the rust-projects.json file as it
is no longer required by the rust-analyzer tool.

Bug: 162055556
Test: prebuilts/rust/tests/rust-analyzer-integration-test.sh
Change-Id: I85c3277632c88394bc52e709ecce83f6aa1f2e0e
2021-08-09 15:50:39 -07:00
LaMont Jones
4224c6216d Improve handling of errors from zipinfo
Bug: 153900481
Test: manual
Change-Id: Ib60578ead4ccaa7ba09f628a6c9281c0cc70e6ce
2021-08-09 19:31:39 +00:00
Christopher Parsons
67d6ccecfc Merge "bp2build: handle system_shared_libs" 2021-08-09 19:11:00 +00:00
Paul Duffin
c11f6679d8 Use trie to store monolithic hidden API flags
Previously, a simple map from dex signature to the flags was used to
store the monolithic hidden API flags. This change replaces that with a
trie that uses packages and class names extracted from the signature
to map from the signature to the flags.

The signature is broken down into pieces consisting of package names,
class names and the remaining member signature. They are used in turn
to navigate through nodes in the trie to a Leaf that contains the flags
associated with the signature.

The flags can be retrieved from the trie in a number of ways:
* Using the complete signature to retrieve the flags for a single
  member.
* Using a class name to retrieve the flags for all the members in a
  class and any nested classes.
* Using a package wildcard to retrieve the flags for all the members in
  the classes in that package.
* Using a recursive package wildcard to retrieve the flags for all the
  members in the classes in that package and nested packages.

This will allow a bootclasspath_fragment to select a subset of the
monolithic flags to compare against itself without specifying the
signatures of every member of that set.

Before creating the trie an attempt was made to compute the subset by
iterating over all the signatures in the monolithic flags and matching
against a regular expression created from the patterns but that was too
slow. It took minutes to run whereas using the simple map or the trie
only takes a few seconds.

Bug: 194063708
Test: atest --host verify_overlaps_test
Change-Id: I36f5e319d3e7d62dd34305de1eec990a93cb3a89
2021-08-09 19:51:33 +01:00
Chris Parsons
51f8c39261 bp2build: handle system_shared_libs
- If no system_shared_libs is specified, bp2build writes no attribute
value. In this case, the bazel library macros determine the correct
default behavior.
- If any system_shared_libs is specified for any variant, then bp2build
writes the value verbatim. This includes if an empty list is specified,
as this should override defaulting behavior.

Note this defaulting behavior is incomplete and will be incorrect in
corner cases. For example, if, in an Android.bp, system_shared_libs is
specified for os.linux_bionic but not for os.android, then the bazel
default for os.android will be incorrect. However, there are no current
modules in AOSP which fit this case.

As a related fix, supports static struct for cc_library_static.

Also, removes some elements from the bp2build denylist.

Test: mixed_droid CI
Change-Id: Iee5feeaaf05e8e7209c7a90c913173832ad7bf91
2021-08-09 11:41:09 -04:00
Yabin Cui
01cad0cddc Merge "Revert "Switch to clang r428724 13.0.1."" 2021-08-08 21:02:29 +00:00
Yabin Cui
786c1d41ef Revert "Switch to clang r428724 13.0.1."
Revert submission 1743633-switch-to-clang-r428724

Reason for revert: fix build
Reverted Changes:
I824f3bc1c:Update bazel's clang version to clang-r428724
I6c4c26267:Adjust test for compiler update.
I94ab1f2dc:Switch to clang r428724 13.0.1.

Change-Id: I846e904b6e868d4e2be1bd8260ed59fc5464ef68
2021-08-08 20:59:59 +00:00
Yabin Cui
baeca672c1 Merge "Switch to clang r428724 13.0.1." 2021-08-08 18:20:22 +00:00
Romain Jobredeaux
748652e350 Exempt prebuilts/jdk and build/bazel/examples/java BUILD files from bp2build overwriting
Test: b build prebuilts/jdk/...
Test: b build build/bazel/examples/java/...
Change-Id: I6f82e4984ab2d8ec4c21edbf2dcba5f37b515fc9
Bug: 195786195
2021-08-06 22:09:36 +00:00
Colin Cross
795a9f99f4 Merge "Reland: Pass jars with resources to R8" 2021-08-06 18:33:23 +00:00
Yabin Cui
e87329c4c5 Switch to clang r428724 13.0.1.
Bug: http://b/190099451
Test: build and boot devices

Change-Id: I94ab1f2dc194ffcec5681708b558af26ced30aae
2021-08-06 18:28:08 +00:00
Colin Cross
03349188bc Merge changes Ia0c3826a,I92ad6635
* changes:
  Revert "Time out and dump stacks from R8 processes after 30 minutes"
  Add more debugging to run_with_timeout
2021-08-06 17:47:49 +00:00
Stephen Hines
fe3bab726d Merge "Revert "Make DWARF v5 the default debug information format."" 2021-08-06 17:04:41 +00:00
Paul Duffin
8d007e9919 Export signature patterns to sdk snapshot
Bug: 194063708
Test: atest --host verify_overlaps_test signature_patterns_test
      m out/soong/hiddenapi/hiddenapi-flags.csv
      - manually change files to cause difference in flags to check
        that it detects the differences.
Change-Id: Ic819def9b0c41e3b22fb98582cd39f6d27080a32
2021-08-06 14:11:14 +01:00
Paul Duffin
67b9d61ac2 Separate creation of signature patterns from overlap checking
Previously, the signatures used to select the subset of the monolithic
flags were simply the signatures read from the modular flags file. This
change moves the creation of the signature list into a separate script
that outputs the signatures to a file and then passes the path through
Soong from the bootclasspath_fragment modules that create it to the
platform_bootclasspath module that uses it to compare the modular
flags against the monolithic flags.

Currently, the signatures are the full signatures but follow up changes
will replace them with patterns (hence the name) that avoids having to
include implementation details in the hidden API flags that are output
as part of a bootclasspath_fragment's snapshot.

This change moves the stub flags related code next to the all flags
related code as they are treated in a similar way.

Bug: 194063708
Test: atest --host verify_overlaps_test signature_patterns_test
      m out/soong/hiddenapi/hiddenapi-flags.csv
      - manually change files to cause difference in flags to check
        that it detects the differences.
Change-Id: I2855bf6d05c91b8a09591664185750361c7e644f
2021-08-06 13:40:34 +01:00
Paul Duffin
47c456228c Merge "Extract a subset of the monolithic flags for comparison" 2021-08-06 09:08:29 +00:00
Paul Duffin
335f02ad21 Merge "Clarify symbols used in verify_overlaps.py" 2021-08-06 09:06:23 +00:00
Paul Duffin
a26cf315a8 Merge "Add tests for verify_overlaps script" 2021-08-06 09:05:45 +00:00
Colin Cross
a79a52c7c4 Reland: Pass jars with resources to R8
R8 will parse proguard files out of resources in injars.  Use the jar
with resources instead of the classes jar so that R8 can see the
proguard files.

R8 fails when an input jar contains dex files, so filter out dex files
from the input jar before passing it to r8.

This relands Ibb870ee9c70470336f542a3b7542dab86716dbf8 with a fix to
move the temporary input jar outside the output directory.  Putting it
in the output directory interacted poorly with RBE in local comparison
mode, which incorrectly deleted the output directory.

Bug: 195558228
Test: m checkbuild
Change-Id: Id4b091c3bd72d4c2904883e8793ec4b9499e4b78
2021-08-05 23:16:38 -07:00
Stephen Hines
bb1a17aecb Revert "Make DWARF v5 the default debug information format."
This reverts commit 5e13f86fc8.

libabigail isn't quite ready for DWARF v5 yet, so let's revert this
temporarily.

Bug: http://b/182004844
Bug: http://b/195152239
Bug: http://b/195698910
Test: Builds
Change-Id: I4b1dd8f9e64d19f37c90b8a2433cbb302c7d5105
2021-08-05 17:05:01 -07:00
Colin Cross
a832a04db2 Revert "Time out and dump stacks from R8 processes after 30 minutes"
This reverts commit 29c294b2cf.

The deadlock in R8 was fixed, and the timeout causes problems when
RBE is enabled and the R8 action gets stuck in a queue for 30
minutes.

Test: checkbuild
Change-Id: Ia0c3826aea3288e9db3bdbd4955e1e16c0a38fe3
2021-08-05 17:04:06 -07:00
Colin Cross
11ec0a8f8b Add more debugging to run_with_timeout
Print more messages in run_with_timeout to make it easier to
determine which log statements came before the timeout and the
on_timeout command.

Test: run_with_timeout_test.go
Change-Id: I92ad663529fa03d91185ba382654a73f0eba1ab7
2021-08-05 16:51:50 -07:00
Pirama Arumuga Nainar
238ca381c4 [scripts] Print full clang prebuilt dir from get_clang_version.py
e.g. clang-r416183b1 instead of just r416183b1.

Test: manually run the script
Change-Id: I993202ea97433f841e3127ecef46dec66fb9a8c9
2021-08-05 23:39:00 +00:00
Colin Cross
93fb53f5d0 Merge "Revert "Pass jars with resources to R8"" 2021-08-05 15:25:25 +00:00
Colin Cross
1d73c7316c Revert "Pass jars with resources to R8"
This reverts commit 05ca7c8769.

Reason for revert: b/195644067

Flaky failure with "missing in.jar" error.

Change-Id: Id2d6452de1e43951bc58f0bfb8e20d5ad65ce830
2021-08-05 15:16:11 +00:00
Paul Duffin
53a7607f7d Extract a subset of the monolithic flags for comparison
Previously, the compare_signature_flags compared the full set of
monolithic flags against the modular flags and just ignored any
monolithic flags for a signature that was not present in the modular
flags. It essentially performed two tasks, identifying the subset of
the monolithic flags against which the modular flags should be compared
and then doing the actual comparison.

This change separates those two tasks, by first creating a dict that is
the subset of the monolithic flags against which the modular flags will
be compared and then do the comparison.

At the moment the subset is simply the set of monolithic flags
associated with the signatures that are present in the modular flags.
Follow up changes will provide other ways to select the subset.

Bug: 194063708
Test: atest --host verify_overlaps_test
      m out/soong/hiddenapi/hiddenapi-flags.csv
      - manually change files to cause difference in flags to check
        that it detects the differences.
Change-Id: Icb547f071b814dd955f31255b0d35597f7ba4aa1
2021-08-05 14:19:57 +01:00
Paul Duffin
7be96337f2 Clarify symbols used in verify_overlaps.py
Switches symbols to use monolithic... and modular... to try and clarify
the purpose of those symbols.

Bug: 194063708
Test: atest --host verify_overlaps_test
Change-Id: Ifc646edebe3d43b51dc3528fa883718399cb5254
2021-08-05 14:19:57 +01:00
Paul Duffin
428c65156b Add tests for verify_overlaps script
Refactor verify_overlaps to make it testable and add tests for the
comparison. It does make one significant change in behavior which is to
read each of the files produced by a bootclasspath_fragment into a dict
before comparison, rather than reading and comparing them a row at a
time. That allows it to reuse the code to read a CSV into a dict.

Bug: 194063708
Test: atest --host verify_overlaps_test
      m out/soong/hiddenapi/hiddenapi-flags.csv
      - manually change files to cause difference in flags to check
        that it detects the differences.
Change-Id: Ib70ac87fe089fc25e3bef18f367d4939bfc0cb8d
2021-08-05 14:16:51 +01:00
Colin Cross
ac33aac0aa Merge "Pass jars with resources to R8" 2021-08-05 01:41:32 +00:00
Colin Cross
05ca7c8769 Pass jars with resources to R8
R8 will parse proguard files out of resources in injars.  Use the jar
with resources instead of the classes jar so that R8 can see the
proguard files.

R8 fails when an input jar contains dex files, so filter out dex files
from the input jar before passing it to r8.

Bug: 195558228
Test: m checkbuild
Change-Id: Ibb870ee9c70470336f542a3b7542dab86716dbf8
2021-08-04 14:57:45 -07:00
Treehugger Robot
27ac52198d Merge "Update the Rust prebuilt version number to 1.54.0" 2021-08-04 21:28:20 +00:00
Treehugger Robot
1e28e36589 Merge "[rust] [coverage] Don't pass -C link-dead-code" 2021-08-04 20:51:47 +00:00
Mathew Inwood
bf49705608 Merge "Update error message for new policy." 2021-08-04 09:59:33 +00:00