Commit Graph

505 Commits

Author SHA1 Message Date
Patrice Arruda
0cc5b21516 Soong: Fix the package name in metrics.proto file
The metrics.proto needs to be imported to Google3 in order to unmarshal
the build metrics data from bigstore. Cleaned up the enum names to use
the Camel naming convention and renamed the build_metrics to
soong_metrics to be more specific to soong and to allow quering
soong_metrics files from bigstore.

Bug: b/135280521
Test: lunch andf m -j. Checked if soong_metrics was generated correctly
      by using printproto command.

Change-Id: I998c8d05db592e94a653d6ca32250b80df3c9b21
2019-06-14 15:27:46 -07:00
Colin Cross
ceec6a2295 Merge changes I555ad21a,I9d46761e,I917bdeee,I5bc5476a,I59bbdae4, ... am: 6ebe07def7
am: 1337decdff

Change-Id: I2f27851391ad68137cc66133de575d75a3c83a74
2019-06-14 01:30:19 -07:00
Colin Cross
e0df1a36b2 Move all output through StatusOutput
Write log output through StatusOutput so that the status implementation
can synchronize it with its own output.

Test: status_test.go
Change-Id: I917bdeeea4759a12b6b4aa6d6d86ee18a2771723
2019-06-12 21:30:56 -07:00
Colin Cross
097ed2a37c Remove terminal.Writer
terminal.Writer is now just a wrapper around stdio.Stdout() without
any useful functionality.  Replace it with stdio.Stdout() as an
io.Writer.

Test: ui/terminal/status_test.go
Change-Id: I5bc5476afdca950b505642f0135a3af9d37fbe24
2019-06-12 21:30:56 -07:00
Dan Willemsen
7488174f2c Merge "Refactor .aar resource compilation" am: 588857b418
am: 2758050dc4

Change-Id: I98dfbdbef71972a040fd53f9e77eb321fb6f5e0e
2019-06-10 21:16:55 -07:00
Dan Willemsen
304cfec778 Refactor .aar resource compilation
Instead of extracting the aar, and passing the res directory to another rule
(along with a dependency to a different known file), don't keep the
extracted res directory, and pass the aar directly to the resource
compliation rule, which will extract just the res directory.

I need this for my RBE experiments, where non-listed output files won't
exist in other rules.

Test: m
Change-Id: I99074381052cbcebb6a402484abae9ab2e40284f
2019-05-29 16:54:43 -07:00
Patrice Arruda
d2e1149cc5 resolve merge conflicts of a3898089ba to qt-dev-plus-aosp
Bug: None
Test: Ran soong_ui.bash which compiles the soong_ui.
Change-Id: I0b8083e3e5ee69c8cbf17443fbc038ec7b5a0b10
2019-05-06 19:24:32 -07:00
Treehugger Robot
565ee6ed87 Merge "Soong: Refactor the soong_ui arguments processing to be more like bazel." 2019-05-07 00:49:21 +00:00
Patrice Arruda
a5c2542f68 Soong: Refactor the soong_ui arguments processing to be more like bazel.
Currently, the command line argments is being processed in multiple
places. In the main soong_ui arguments, there are several if statements
that checks if a specific argument was specified in order to execute
the requested operation. This does not scale well when adding or
removing a command in the near future. In order to support the build
commands (m, mma, etc...) from the envsetup.sh in soong_ui, a refactor
was required in order to add a command rather quickly and to have the
flexibiity to unit test the command. The soong_ui arguments format is
as follows:

soong_ui <command> [<arg 1> <arg 2> ... <arg n>]

The <command> is a specific operation to be executed. The arguments
after the command are processed by the command itself.

Below is the list of changes made in this commit:

 * Created a new command infrastructure that allows adding
   or deprecating a command easily.

 * Fixed a bug when running ./soong_ui.bash directly would cause
   a panic due to index being out of range for args list.

Bug: b/130049705
Test: Below is the list of testing done on this commit:
    * Ran lunch and verified that the output is the same as the the output
      without the modifications. lunch indirectly runs soong_ui passing
      in the --dumpvar-mode (to read makefile variables such as
      TARGET_PRODUCT) and --dumpvars-mode (to build the build cache).
    * Ran ./soong_ui.bash directly (with unsupported flags and no flags)
      and the proper message appeared that soong native UI is not yet
      available.
    * Ran m, mm, mmm, mma, mmma commands.
    * Ran the make installclean command:
         make -j80 TARGET_PRODUCT=aosp_arm64 TARGET_BUILD_VARIANT=eng dist DIST_DIR=/tmp/helloworld installclean
    * Ran ./out/soong_ui -j80 --make-mode PRODUCT-aosp_arm64-eng dist checkbuild tests

Change-Id: Iee4de7ec0fa4661206fda8ae1fe6fa4487d9bb22
Merged-In: Iee4de7ec0fa4661206fda8ae1fe6fa4487d9bb22
2019-05-06 14:05:52 -07:00
Colin Cross
f41ed9ae63 Merge "Remove services.art from known_nondeterminism.whitelist" am: 42603c4ab5
am: 828a488720

Change-Id: Idbdeec76c0d8207890afbf94ea20ebac4c3d5379
2019-04-30 15:24:25 -07:00
Colin Cross
497d4c24b1 Remove services.art from known_nondeterminism.whitelist
Bug: 120039850
Test: none
Change-Id: I70d5243f427a4da5c8126be6b8ebc90221133ad3
2019-04-29 17:36:19 +00:00
Dan Willemsen
7d155094be Merge "pom2bp: Implement -static-deps to match pom2mk" am: 80ae6816ae
am: 9181e08738

Change-Id: I331a7bb1cd82315f673d9007c2240da423864034
2019-04-22 20:17:35 -07:00
Dan Willemsen
52c90d868e pom2bp: Implement -static-deps to match pom2mk
This changes the default behavior of pom2bp to match pom2mk, and not
require pre-extraction of all of the manifests.

Test: convert an internal pom2mk user to pom2bp
Change-Id: I08120cce1f923ef28519563ffd772032b2e500ed
2019-04-21 21:46:42 -07:00
Dan Willemsen
8a4c0da74d Add -jetifier to pom2mk and pom2bp am: 7fdab6eeec
am: fd9fa1a56e

Change-Id: Ifaa39849ef2d858dd63745e2a8a0b9e5587bbd66
2019-04-21 02:47:34 -07:00
Dan Willemsen
7fdab6eeec Add -jetifier to pom2mk and pom2bp
Test: use pom2mk -jetifier to replace some hand-modified makefiles
Change-Id: I2a1988d0120fee1edf2d85c8b3294041840efe7c
2019-04-20 21:47:14 -07:00
Paul Duffin
0018e7f48b Merge "Rename -extra-deps to --extra-static-libs and add --extra-libs" am: 38b122021a
am: 2b9f534fba

Change-Id: I1a607c72ad7f1a71b7dcc0aa216beee2b868776f
2019-04-17 07:32:57 -07:00
Paul Duffin
38b122021a Merge "Rename -extra-deps to --extra-static-libs and add --extra-libs" 2019-04-17 14:06:48 +00:00
Paul Duffin
babaf07b13 Rename -extra-deps to --extra-static-libs and add --extra-libs
Allow callers of pom2bp to add dependencies on runtime libraries as
well as static libraries. Renames -extra-deps to --extra-static-libs
to make it clearer what it does.

Bug: 130287656
Test: prebuilts/sdk/update_prebuilts.py --stx 5280039
      prebuilts/misc/common/androidx-test/update-from-gmaven.py

Change-Id: I8a3ee8056aa4ed9935a4f47d563ec423672cf38b
2019-04-16 11:55:48 +01:00
Jaewoong Jung
a7d23ae804 Merge "Embed CSS entries in soong_build.html." am: 0a0b3fa605
am: 6d48aa4186

Change-Id: I0e162fdc0eabafc152686f637436b41295540935
2019-04-15 18:17:14 -07:00
Dan Willemsen
891af48379 Support RuleBuilder.Sbox to wrap commands in sbox am: 633c502295
am: b933c57635

Change-Id: I07cb4998e8d860f97c4ae00f9f776744cc8a5adf
2019-04-15 17:50:48 -07:00
Treehugger Robot
0a0b3fa605 Merge "Embed CSS entries in soong_build.html." 2019-04-16 00:49:18 +00:00
Jaewoong Jung
5f867c0580 Embed CSS entries in soong_build.html.
This will prevent it from being affected by future css location changes.

Test: m soong_docs
Fixes: 130567020
Change-Id: I80fd7b8a12a32bcea7b270423b8d8f8dc0976177
2019-04-15 15:21:55 -07:00
Dan Willemsen
633c502295 Support RuleBuilder.Sbox to wrap commands in sbox
This essentially allows you to declare that everything in a directory
will be created by the rule, and we'll ensure that your command actually
writes out all of the claimed outputs, and remove any other files that
previously existed in that directory.

Test: built-in tests
Change-Id: I990dce2b3a0d89ebd2736ac1a0cadfb5864c6e73
2019-04-15 14:52:05 -07:00
Colin Cross
537cfd2477 Merge "pom2bp: add jar deps to -nodeps modules" am: 34df71446f
am: 694b4629e1

Change-Id: I519921487ae1813091e08074df70376fa39efb61
2019-04-11 16:23:49 -07:00
Colin Cross
1aa7f26a22 pom2bp: add jar deps to -nodeps modules
The dependency type in the pom file is not always accurate, just
add all jar and aar dependencies to both the -nodeps and final
module.

Test: cd prebuilts/sdk/current/androidx && pom2bp -regen Android.bp && mma
Change-Id: I1784b8ebf3485db19405cb353c9bb5fb3ae837d8
2019-04-10 18:31:55 +00:00
Colin Cross
11fbb890c9 Fix package path of android/soong/android pctx am: cc0ce80ed5
am: a1c57b4664

Change-Id: Ifba10892f902e70bd0a52a82a20bd5f1b64a61de
2019-04-04 01:39:57 -07:00
Colin Cross
cc0ce80ed5 Fix package path of android/soong/android pctx
android/soong/common was renamed to android/soong/android long
ago, but the pctx package path was still "android/soong/common".
This required all users of rules defined in android/soong/android
to import "android/soong/android" and then
pctx.Import("android/soong/common").

Test: m checkbuild
Change-Id: I20d096522760538f7cfc2bec3d4bfeba99b275d4
2019-04-04 03:59:34 +00:00
Colin Cross
a50a2464c6 Merge changes I7dd4ba7f,I7887a67a am: 92cc71606c am: 900574f0cd
am: 6c99a30bb7

Change-Id: If557897ed64ecdf8967299b5512392e5485767ae
2019-04-02 08:21:42 -07:00
Colin Cross
92cc71606c Merge changes I7dd4ba7f,I7887a67a
* changes:
  Add depfile support to RuleBuilder
  Add RuleBuilderCommand.Flags
2019-04-02 15:04:06 +00:00
Colin Cross
f37f6b064c Merge changes Ie7f776a4,I3ca5dd1f,I2e911262 am: 11581cf6b3 am: 89d99337fb
am: 92270109eb

Change-Id: Ic7ac18a1d60138e909c6045fd5313620256989d4
2019-04-01 17:48:29 -07:00
Colin Cross
11581cf6b3 Merge changes Ie7f776a4,I3ca5dd1f,I2e911262
* changes:
  Add a whitelist for services.art changing between builds
  Add diff_target_files tool
  Import files from compare_target_files for use in diff_target_files
2019-04-02 00:03:05 +00:00
Colin Cross
93599cdb93 Merge "Touch output files in sbox" am: 80eb6907ac am: 9457c97c40
am: 5cb90e6517

Change-Id: I07587e71632567f553f417db40b791e4d335aaae
2019-04-01 11:15:17 -07:00
Colin Cross
d1c1e6fd6c Touch output files in sbox
Some tools used by genrules set the timestamp of the generated file
to an old value.  For example, when using tar to extract files it
will set the timestamp of the extracted file to the value stored in
the tar file.  This can cause unnecessary rebuilds due to output files
that are older than their inputs.  Touch output files in sbox before
moving them into the output location.

Test: m checkbuild && m checkbuild
Change-Id: Idb0e71d2bf0ac3618338371132cc71be74bf9d9d
2019-04-01 10:43:20 -07:00
Colin Cross
d93112ae26 Add a whitelist for services.art changing between builds
Bug: 120039850
Test: manual
Change-Id: Ie7f776a4e2743883ea006c66148fc4f40f90ce07
2019-04-01 10:07:25 -07:00
Colin Cross
91ee4c793d Add diff_target_files tool
Add a tool that lists files that have changed between two builds by
comparing the target-files.zip artifacts.

Bug: 121158314
Test: unit tests
Change-Id: I3ca5dd1fdf652dffef923ad7307fd71e49e2e95a
2019-04-01 10:07:21 -07:00
Colin Cross
c45c3b5ec6 Import files from compare_target_files for use in diff_target_files
Copied from cl/240594925.

Bug: 121158314
Test: copied unit tests
Change-Id: I2e91126285dcd33171ff8b8dbfcfa5d48501f535
2019-04-01 10:02:40 -07:00
Colin Cross
1d2cf0494a Add depfile support to RuleBuilder
Allow rules built with RuleBuilder to use depfiles.  Ninja only
supports a single depfile with single output.  If there are
multiple outputs in a rule, move all but the first to implicit
outputs.  If multiple depfiles are specified, use new support
in dep_fixer to combine additional depfiles into the first depfile.

Test: rule_builder_test.go
Change-Id: I7dd4ba7fdf9feaf89b3dd2b7abb0e79006e06018
2019-03-29 16:35:06 -07:00
Jaewoong Jung
9b4cd0c39b Build doc improvements. am: 12c02a63d0 am: dde127fdb6
am: 8c822915fb

Change-Id: If8b0c3432f659b0739f92cf07f4a0020a6c1cbd8
2019-03-13 10:40:57 -07:00
Jaewoong Jung
12c02a63d0 Build doc improvements.
Bring property type strings to the start of property comments. This is
how source.android.com shows variables and paratemeters, and it is also
easier to read, especially when a comment ends with a code example.

Test: m soong_docs
Change-Id: Ic3e9db3a3711495998919ee95bbef6c7cb33b215
2019-03-12 13:31:11 -07:00
Jaewoong Jung
e4d2d08930 Merge "Change module type text to HTML in build docs." am: 0d2c1d1e23 am: 7c8d1b2400
am: dfd6745462

Change-Id: I56e69eb99966564a754efa45c15726cbc8e52590
2019-03-11 19:27:54 -07:00
Jaewoong Jung
238be3807c Change module type text to HTML in build docs.
To add code format (<pre>...</pre>) support to module type texts.

Fixes: 124332708
Test: m soong_docs
Change-Id: Ibe9d3845a8f952fdef5007893d4af5836809ce1f
2019-03-11 14:35:41 -07:00
Dan Willemsen
6b783c8391 Fix recent builds with bad dangling symlinks
It appears that there's a bug in repo where when a linkfile is removed
from the manifest, the symlink remains in the source tree.

Cleaning this up needs to happen before we start scanning the source
tree for Android.bp/Android.mk files, so this has to be done very early
-- way before CleanSpec processing could handle it.

Bug: https://bugs.chromium.org/p/gerrit/issues/detail?id=10583
Test: ln -s missing hardware/qcom/sdm710/Android.bp; m
Change-Id: Ib68f2507ffe58ccdd9fbc88925f8a4f6150f2f7d
2019-03-08 19:56:57 +00:00
Colin Cross
0b9f31fb08 Replace *Escape with *EscapeList
Follow the change to blueprint to make *Escape take and return a string
and add *EscapeList that take and return slices of strings.  Fix up
a few places that were unnecessarily converting a string to a slice
and back to a string.

Test: m nothing
Change-Id: I3fa87de175522205f36544ef76aa2f04aef1b936
2019-03-04 18:11:53 +00:00
Jaewoong Jung
d10f484eee Add a filename to anchor links in build docs.
The href base added by I8a8ac0f9ba6c0d57ed83db2155955a95e070a265
doesn't really play nice with anchor links. This fixes the issue.

Fixes: 126550408
Test: m soong_docs
Change-Id: If62411a6391fd8151fa40cb2ecf32c5006797d6c
2019-02-27 11:15:00 -08:00
Jaewoong Jung
90e1155135 soong_docs build link error fix.
This adds a script to copy the current page's base url to hrefs in the
doc.

Fixes: 123473548
Test: m soong_docs
Change-Id: I8a8ac0f9ba6c0d57ed83db2155955a95e070a265
2019-02-25 10:26:46 -08:00
Jaewoong Jung
6c29688691 Split up Soong build doc.
Previously the build doc was a gigantic list of modules and properties,
which can be overwhelming to new users. This change breaks it up by
packages, so that it is easier to look up and feels more coherent.

Bug: 123521276
Test: m soong_docs
Change-Id: I1a1a331f4b4deb8782d698e9076098c5a27b9566
2019-02-20 15:43:06 -08:00
Treehugger Robot
a2b41a6e02 Merge "Improve documentation page layout" 2019-02-20 07:50:16 +00:00
Dan Willemsen
707542f298 Simplify python launcher, use __main__.py
Uses more python rather than C++, and skips less of Py_Main.

Test: build/soong/python/tests/runtests.sh
Change-Id: I03997d88e2e16047c96bb4e00e530229c42b3325
2019-02-15 14:33:59 -08:00
Sasha Smundak
ff483393ac Improve documentation page layout
It now has module list on the left and the main panel containing
a section for each module. Each section contains the navigable list
of the module's attributes, grouped by attribute's origin.

Test: Manual
Bug: 120512870
Change-Id: Ib62ef58c61daa88950f0934a70ba8ed16f433bf2
2019-02-13 21:40:21 -08:00
Colin Cross
7089c27c07 Update Soong docs generator for blueprint changes
bootstrap.ModuleTypeDocs needs a mapping of module types to factories
to support factories that are wrapped in ModuleFactoryAdapter closures.

It also returns ModuleType objects grouped into Package objects.

Bug: 67909957
Test: m soong_docs
Change-Id: I70eac9f0f0e13075580da92d4219792ca0b18fbf
2019-02-13 20:39:26 +00:00