Commit Graph

76019 Commits

Author SHA1 Message Date
Joe Onorato
c35895676c Orchestrator can build end to end.
This reduces the scope of the demo to just building and installing
a single .so, but it makes the demo actually build that single .so.

Next up, writing some unit tests and fleshing out functionality.

Test: see the README
Change-Id: I560904b786fbf69d3a83dbb08d496dba5a3192ca
2022-05-26 15:11:23 -07:00
Kelvin Zhang
4117e78575 Merge "Install new snapuserd_ramdisk stem" 2022-05-25 15:39:17 +00:00
Chih-hung Hsieh
c15b738a7c Merge "Add date time, checks on android_root, etc." 2022-05-24 22:53:00 +00:00
Usta Shrestha
0e46244250 Merge "Allow sourcing envsetup from anywhere" 2022-05-24 17:01:12 +00:00
Usta Shrestha
1433fb357b Allow sourcing envsetup from anywhere
Bug: 232555676
Test: cd build && source envsetup.sh
Change-Id: I8491628406e494f31589249d1cc851b672dd7cae
2022-05-23 18:50:52 +00:00
Treehugger Robot
a3dc792131 Merge "Add a doc about dex_preopt_check." 2022-05-23 10:25:43 +00:00
Treehugger Robot
7668e5a13c Merge "Remove "@1.0-" from android.system.suspend service's name" 2022-05-20 18:49:39 +00:00
Jiakai Zhang
500671f61c Add a doc about dex_preopt_check.
The doc lists some possible causes of a `dex_preopt_check` error, which
can be useful for developers to understand how to debug the error.

Bug: 232542678
Test: m nothing
Change-Id: I2e9868a6a470e41e8c3743c045e0cf4cf7367efa
2022-05-20 16:21:46 +01:00
Anton Hansson
eb483046c0 Merge "Revert^2 "Slim down the sdk product"" 2022-05-20 13:16:13 +00:00
Yi Kong
05b2217940 Merge "Rename modulesAddedWall to modulesWarningsAllowed" 2022-05-20 00:26:18 +00:00
Colin Cross
3bbbe3f04e Merge "Don't depend on art's boot-image-profile.txt in thin manifests" 2022-05-20 00:12:49 +00:00
Chih-Hung Hsieh
dd58ab0dc1 Add date time, checks on android_root, etc.
* Dump current date/time to the output .html file
  to distinguish files from different build dates.
* Guess android_root from the path of warn_common.py,
  but only if it contains build/make and build/soong.
* Do not depend on the TOP variable,
  which is no longer emitted by build/soong make.
* Do not depend on the USE_RBE variables,
  which is not emitted in local makes.

Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Bug: 198657613
Change-Id: Ib75c60216f9e06f9a768e63b536495fb9658843f
2022-05-19 15:28:41 -07:00
Yi Kong
ee99ddf900 Rename modulesAddedWall to modulesWarningsAllowed
MODULES_ADDED_WALL is deceiving and it actually meant which modules are
allowed to have warnings (do not automatically append -Werror flag).

Test: presubmit
Bug: 233183337
Change-Id: Ia2eb972d3d76ba245840f9872cbcc66d2894dab3
2022-05-19 20:16:00 +08:00
Dmitrii Merkurev
494358dec8 Merge "Introduce EROFS compress hints to the Android build system" 2022-05-19 11:00:11 +00:00
Joe Onorato
9327b9cb4d Merge "Generate ninja files for api assembly and run ninja." 2022-05-19 05:56:48 +00:00
Chih-hung Hsieh
b28e4c2235 Merge "Handle more mixed warning lines from RBE" 2022-05-18 21:31:22 +00:00
Colin Cross
37823d1e6e Don't depend on art's boot-image-profile.txt in thin manifests
Use $(wildcard) to avoid referencing art/build/boot/boot-image-profile.txt
in thin manifests that don't have the art project.

Bug: 207813628
Test: tradefed build
Change-Id: I5d23834f4e8484d2c51b916b6e8395ee23315487
2022-05-18 14:15:59 -07:00
Chih-hung Hsieh
89703b6374 Merge "Add warning type into TopDirs and TopFiles index" 2022-05-18 21:03:32 +00:00
Frederick Mayle
cb577c2efc Remove "@1.0-" from android.system.suspend service's name
Bug: 232447926
Test: TH
Change-Id: If177707ae0459cd3e0521d3b8fba8417c5609c5f
Merged-In: If177707ae0459cd3e0521d3b8fba8417c5609c5f
2022-05-18 20:18:14 +00:00
Anton Hansson
6a6092cab0 Revert^2 "Slim down the sdk product"
8d4de90683

Change-Id: I34ffd88204cb0ee668abf26ecb7bb20f663ef7d9
2022-05-18 14:11:44 +00:00
Issei Suzuki
3a294b6ca2 Merge "Revert "Slim down the sdk product"" 2022-05-18 12:26:01 +00:00
Joe Onorato
51497185fd Generate ninja files for api assembly and run ninja.
Change-Id: I003536e4ed4481acbdc041a6450106f8459d5978
2022-05-18 05:24:48 -07:00
Issei Suzuki
8d4de90683 Revert "Slim down the sdk product"
Revert submission 2096601-sdkprod

Reason for revert: DroidMonitor-triggered revert due to breakage https://android-build.googleplex.com/builds/build-details/8608272/targets/sdk, bug b/233036676
Reverted Changes:
I5ef671781:Update sdk parent
I940655c39:Slim down the sdk product
I926d520ce:Update sdk parent

Change-Id: I16953ba601a22b469c312dc481c45ec351ac8cbc
2022-05-18 11:36:23 +00:00
Anton Hansson
8033f34ec2 Merge "Slim down the sdk product" 2022-05-18 10:07:49 +00:00
Dmitrii Merkurev
8ab6603a43 Introduce EROFS compress hints to the Android build system
Make it possible to provide a prebuilt hints file to specify different compression strategies on a file basis for all partitions using:

BOARD_EROFS_COMPRESS_HINTS := <path>/erofs-hints.txt

or separately for every partition using:

BOARD_SYSTEMIMAGE_EROFS_COMPRESS_HINTS := <path>/system-partition-erofs-hints.txt
BOARD_VENDORIMAGE_EROFS_COMPRESS_HINTS := <path>/vendor-partition-erofs-hints.txt

Hints format: Each line is defined by tokens separated by spaces in the following form:

<pcluster-in-bytes> <match-pattern>

pcluster-in-bytes can be specified as 0 which mean file will not be compressed

Bug: 231934752
Test: used dump.erofs -S system.img to verify the system image isn't compressed after specifying BOARD_SYSTEMIMAGE_EROFS_COMPRESS_HINTS := no-compress.txt, where no-compress.txt contains 0 .*
Test: used dump.erofs --nid=<inode-for-all-apk-files> vendor.img to verify apk files are not compressed (the rest of the partition content is compressed)
Test: used dump.erofs -S <all-images> to verify all images are not compressed after specifying BOARD_EROFS_COMPRESS_HINTS := <path>/no-compress.txt
Change-Id: I64054e26af5ea5781c45d2183b7796a9ad2b7927
2022-05-17 23:10:37 +00:00
Chih-Hung Hsieh
a55837c7d1 Handle more mixed warning lines from RBE
* Use android_root or __file__ path to find
  the source tree root and its sub directories.
* Use the sub directory list to clean up RBE-returned
  warning lines, removing all mixed prefix before
  the top sub directories.
* Change some re.match rules and order to save
  unnecessary comparisons.

Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Bug: 198657613
Change-Id: I78cc17d04b8ab9e12935ef04797f3272298d5267
2022-05-17 11:46:58 -07:00
Chih-Hung Hsieh
4b020cd1d3 Add warning type into TopDirs and TopFiles index
* The index was a directory or file path.
  This change adds additional entries that have
  warning type prefix in the index.
* In the "Directories/Files with at least 1% warnings"
  section, the list will include number of warnings for
  (1) "all warnings in a directory/file"
  (2) "warnings of a type in a directory/file"
  (3) "warnings of a type in all directories/files"
  Examples:
    4273 (8.7%) frameworks/av/...
    4130 (8.4%) packages/...
    2427 (5.0%) [google-explicit-constructor] */...
    1628 (3.3%) [cert-err34-c] */...
    1099 (2.2%) [google-runtime-int] bionic/tests/math_data/...
     608 (1.2%) [cert-err34-c] external/...

Bug: 231245501
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: I13bb54c846ad514334f78c5a71e994a131a92963
2022-05-17 11:46:15 -07:00
Treehugger Robot
d528b3c553 Merge "Create $(zip_root)/PVMFW & archive prebuilt used" 2022-05-17 16:34:22 +00:00
Treehugger Robot
3eb3ca0c9b Merge "Add keep rules for @KeepForWeakReference annotation" 2022-05-16 21:19:03 +00:00
Treehugger Robot
266225ccb2 Merge "Revert "Do not compress zip file for unit tests."" 2022-05-13 23:44:57 +00:00
Sam Saccone
7d7ae94bef Revert "Do not compress zip file for unit tests."
This reverts commit a24c755f66.

Reason for revert: not worth the additional storage cost.

Change-Id: I8ed24ddef4ac3f640bf35e79cddf433e0df0854d
2022-05-13 19:55:25 +00:00
Anton Hansson
59f95bcd1c Merge "Remove unused dependencies from sdk goals" 2022-05-13 08:55:55 +00:00
Treehugger Robot
69b1b0c10a Merge "Lightweight ninja writer in Python" 2022-05-12 23:46:55 +00:00
Treehugger Robot
a96be433c4 Merge "Checkpoint new build orchestrator" 2022-05-12 22:55:33 +00:00
Anton Hansson
af94854296 Slim down the sdk product
The sdk_phone_armv7 product is designed to build system images, which
adds unnecessary dependencies and enables unnecessary checks (such as
verifying dexpreopt artifacts).

Bug: 205008975
Test: m sdk sdk_repo dist && diff android-sdk.zip (only build.prop diffs)
Change-Id: I940655c39a3ae58839521416abafaf461df44921
2022-05-12 22:41:02 +00:00
Joe Onorato
7cf6f97775 Checkpoint new build orchestrator
Test: rm -rf out && multitree_build
Change-Id: Ic274182f0925f30d56227597b65e5b9ef3b19707
2022-05-12 12:12:38 -07:00
Spandan Das
aacf2376a5 Lightweight ninja writer in Python
Summary:
- Create python classes for ninja vocbulary in `ninja_syntax.py`. These
  classes will be serialized to a ninja file
- Create a Writer class in `ninja_writer.py`. The current API supports
  adding variables,rules,build actions, etc. This can be extended in the
  future (See `test_ninja_writer.py` for examples)

Future Work:
 - Update the `Subninja` class once chDir is supported (aosp/2064612)
 - Support a width parameter that will be used to wrap long lines of
   text. This will improve readability of the generated files

Expected Use Case: Multi-tree build orchestrator

Test: python ./test_ninja_syntax.py
Test: python ./test_ninja_writer.py

Change-Id: I90c7ee69ddeb7c20c3fd4fca5a911dddbf2253bd
2022-05-12 17:47:29 +00:00
Michael Groover
9d1a0a47d5 Merge "Add support for -providerArg in signapk" 2022-05-12 17:40:31 +00:00
Colin Cross
544aedda27 Merge "Remove HOST_DALVIK_*" 2022-05-12 15:44:20 +00:00
Pierre-Clément Tosi
c7b837872d Create $(zip_root)/PVMFW & archive prebuilt used
It might be useful to have access to the prebuilt that the Android build
used to create the pvmfw partition so add a new directory (similarly to
other partitions) containing the prebuilt (which is also the only
content of the partition). This is particularly useful in ATC, when
detecting if the prebuilt was updated by the build.

Bug: 232253742
Test: m
Change-Id: I1116d91ffb073821568bcaeed30bbb43873dcafa
2022-05-12 16:37:46 +01:00
Anton Hansson
55a8f02ea8 Remove unused dependencies from sdk goals
The SDK no longer includes system images, so no longer needs to depend
on targets that are only relevant for the system images.

Remove all the explicit dependencies that end up depending on
"everything" but are not included in the sdk zip.

Two positive side-effects:
- fewer unused things get built (now ~32k targets, down from ~108k)
- apexes no longer get built, eliminating the risk of mismatch between
  module sdk prebuilts and source-built AOSP apexes.

There are some potential risks with this CL:
- because we no longer forcefully rebuild everything, if there are bugs
  in dependencies of exising files in the SDK, incremental builds may
  not rebuild them.
- some files that get built into the SDK may be being built as a
  side-effect of other files included in the SDK, so removing one file
  may cause another to not get built. The solution to that is to add
  the file not getting built to ALL_SDK_FILES.

Bug: 205008975
Test: m sdk sdk_repo dist && diff sdk.zip before/after (identical)
Change-Id: I97284864f66aa88556fbe16864f45b04be97634e
2022-05-12 12:28:49 +00:00
Treehugger Robot
b4eef93771 Merge "Remove -j1 for aidl-freeze-api" 2022-05-11 23:12:07 +00:00
Michael Groover
2f8aad4f7f Add support for -providerArg in signapk
signapk currently accepts a Provider class that can be instantiated
and inserted before the signing. This commit adds support to specify
a -providerArg parameter that can be used to configure the Provider.
Prior to JDK 9 a Provider would accept a providerArg in a constructor
accepting a String; in JDK 9+ a Provider should first be instantiated
with the zero-arg constructor, then the configure method should be
called with the providerArg.

Bug: 142334653
Bug: 190974913
Fixes: 232134730
Test: Manually verified new Provider can be inserted with pre- and
      post-JDK 9 behavior.
Change-Id: I96f027640c59d3357e8dcf656626d1601bfef861
2022-05-11 21:55:24 +00:00
Treehugger Robot
a09c684e27 Merge "Remove all_product_makefiles" 2022-05-11 18:31:14 +00:00
Kelvin Zhang
823d695d3a Install new snapuserd_ramdisk stem
This installs snapuserd to /system/bin/snapuserd_ramdisk instead of
/system/bin/snapuserd

Test: th
Bug: 219841787
Change-Id: I517e251bf1f895b9de993ec7e5f57f11205172c8
2022-05-11 11:30:26 -07:00
Jared Duke
7d0c90b51f Add keep rules for @KeepForWeakReference annotation
Ensure annotated fields are kept when treeshaking-related optimizations
are enabled. Note that this still allows field removal if tree shaking
reveals no references, but will avoid removal if there are references
that might impact downstream usage, e.g., weak references.

Bug: 227455445
Test: atest InternalAnnotationsTest
Change-Id: I59c6277c43b14c5de8154fa1eefad01f0702e413
2022-05-11 09:47:12 -07:00
Colin Cross
a9203e6d87 Remove HOST_DALVIK_*
There are no more BUILD_HOST_DALVIK_* modules, remove the host dalvik
support.  The equivalent support in Soong just copies the device dex
file to the host dalvik location.

Test: m checkbuild
Change-Id: I4bcf916518317b28d3217994b2fe9ee95d7f50d6
2022-05-11 08:17:45 -07:00
Jeongik Cha
26e9dd30a4 Remove -j1 for aidl-freeze-api
Because bpmodify is safe in multi-process env.

Bug: 190577319
Bug: 229413853
Test: aidl-freeze-api
Change-Id: Ie47afc25e2259bbcbbff50f98fa9d36887558d27
2022-05-11 01:25:19 +00:00
Cole Faust
4edc0b30ff Remove all_product_makefiles
It was unused, along with all_product_configs.

Bug: 228518445
Test: Presubmits
Change-Id: Ibd105241581f8b03254b8fee382d916ea3a98890
2022-05-10 15:39:10 -07:00
Cole Faust
8ccac5198d Merge "Add function for flattening 2d lists" 2022-05-10 05:26:03 +00:00