Commit Graph

2441 Commits

Author SHA1 Message Date
Dan Willemsen
0df1517259 Add build_test.sh, split common parts of soong_ui.bash
Move common microfactory functions to cmd/microfactory/microfactory.bash
so that they can be used to build both soong_ui for normal build and
multiproduct_kati for build system tests.

Test: m -j
Test: build/soong/build_test.bash
Change-Id: I9512642d846ce54d05a027b6d33a2b3029b3f90b
2017-05-15 17:26:52 -07:00
Dan Willemsen
5ed900be48 Allow specifying a build variant
Test: multiproduct_kati --variant userdebug
Change-Id: I6548889caf868cad6270110c038753d029247572
2017-05-15 16:34:46 -07:00
Dan Willemsen
a4e43a77d8 Improve multiproduct_kati output
It now uses the same output style as ninja, overwriting status lines in
smart terminals.

Test: multiproduct_kati
Test: multiproduct_kati | cat
Change-Id: I8db5198ffdc5ebc5503241ac492379753d92978e
2017-05-15 15:24:10 -07:00
Dan Willemsen
02f3add3a3 Clean OUT_DIR
Otherwise Make and soong_ui disagree about where the soong.variables
file should be. For example, when "OUT_DIR=out/":

Error dumping make vars: Failed to parse make line: "make: Nothing to be done for `out/soong/soong.variables'."

We would need to pass out//soong/soong.variables into make, since it
doesn't attempt to match clean/unclean paths.

Bug: 37553659
Test: OUT_DIR=out/ m -j
Change-Id: Iebeea4868b96b11e8b08735f272e22be5db1d701
2017-05-12 21:12:13 +00:00
Colin Cross
67a5c132c5 Prettify soong ninja build descriptions
Descriptions currently look like:
[  0% 4/29328] cc out-soong/.intermediates/external/clang/lib/Sema/libclangSema/android_arm_armv7-a-neon_denver_static_core/obj/external/clang/lib/Sema/SemaCodeComplete.o

This is not very helpful - most of the characters are used to show the
output path, which contains useful information like target architecture,
but also contains most of the path to the source files twice, and less
useful information like the exact variant name used by soong.
Make the descriptions look like:
[  0% 3/29329] //external/clang/lib/Sema:libclangSema clang++ SemaTemplateInstantiate.cpp

This is //path/to/module:modulename tool relative/path/to/source/file

Test: builds, looks pretty
Change-Id: I3087aa7d4eb1860ef6239d77407b8b35445616d7
2017-05-10 11:04:16 -07:00
Colin Cross
570e18ffaa Merge changes If90975c8,I2a873fc6
* changes:
  Use ctx.ModuleBuild for darwin ar
  Run gofmt
2017-05-10 17:28:21 +00:00
Treehugger Robot
c7fd1aa491 Merge "Fix genrule documentation" 2017-05-10 04:58:37 +00:00
Treehugger Robot
f2a9a3c582 Merge "soong: add __clang_analyzer__ to clang-tidy" 2017-05-10 04:34:42 +00:00
Treehugger Robot
7335e16774 Merge "Work-around for crt objects with sdk: current" 2017-05-10 03:57:22 +00:00
Treehugger Robot
fb13d59528 Merge "Add bootable/recovery to root.bp" 2017-05-10 03:05:25 +00:00
Dan Willemsen
014de6a033 Fix genrule documentation
Change-Id: Ieffb2bd7d739527db129ff300d8aec70d82c278a
2017-05-09 23:09:17 +00:00
Steven Moreland
90a4b752bc Add bootable/recovery to root.bp
Test: soong finds things in bootable/recovery
Change-Id: I88f012ab344046d2307a6f08dc618ad424fe870e
2017-05-09 15:19:08 -07:00
Jayant Chowdhary
6e8115acb1 Work-around for crt objects with sdk: current
Test: convert libcups from Android.mk->Android.bp, mm -j64.

Change-Id: I4fddcfa5806d0ce27a229c37a078b0d93e9082a8
2017-05-09 14:38:10 -07:00
Colin Cross
5b52959c99 Use ctx.ModuleBuild for darwin ar
Test: builds
Change-Id: If90975c8545158012bc6201acadd136363c21260
2017-05-09 14:32:45 -07:00
Colin Cross
51d4ab2d5e Run gofmt
Test: builds
Change-Id: I2a873fc65cf414a14d6b58ef4ed6b9f5e57d75f7
2017-05-09 13:44:49 -07:00
Colin Cross
329a42cec6 Merge "androidmk: support cfi, diag, and misc_undefined properties" 2017-05-09 20:39:29 +00:00
Andreas Gampe
9707116867 Soong: Change Sanitize.Recover order
Add Sanitize.Order cflags after the general sanitizer options, so
it can override the global -fno-sanitize-recover=all.

Bug: 38128476
Test: m SANITIZE_HOST=address
Test: manual test
Change-Id: Ied5600507b66bd25d9d2a8bba428162ed4eb01e4
2017-05-08 23:37:51 +00:00
Colin Cross
254514eb73 androidmk: support cfi, diag, and misc_undefined properties
Support more sanitize related properties.  Also remove integer,
since it isn't a supported property, and rewrite it to be
deterministic by not going through splitBpList.

This reapplies Ie8fba06c36fc121912c65a77ee495ceef4962df0.

Bug: 37547301
Test: androidmk_test.go
Change-Id: Id637ffdb7d36bed8a2c2fe76bd6f54662709e3b3
2017-05-08 16:03:17 -07:00
Colin Cross
8ff9ef4c30 Declare dependency on sanitizer runtime libraries
Moving the last users of a sanitizer runtime library to soong was
causing the runtime library to stop getting installed.  Declare
the dependency so make keeps installing it.

Test: builds
Change-Id: Ieeb9ad5c04ac8df0d1a74239da393dac5cab2b03
2017-05-08 15:57:05 -07:00
Colin Cross
ceeff0fd39 Add product_sdk_version.cflags property
libwilhelm needs PRODUCT_SDK_VERSION=%d.

Test: builds
Change-Id: Ie3caf9e4fdcb8e241c8743b1675520e93cf20cae
2017-05-08 15:57:05 -07:00
Colin Cross
e6d4cb3652 Merge "Revert "androidmk: support cfi, diag, and misc_undefined properties"" 2017-05-08 22:11:02 +00:00
Colin Cross
0fdee74f5b Revert "androidmk: support cfi, diag, and misc_undefined properties"
This reverts commit a8688c4652.

Change-Id: Ida4050f3ddc8303d614f2bd2a01470ed5459fe2c
2017-05-08 22:10:25 +00:00
Dan Willemsen
411a363c5a Merge "Revert "Ensure environment dependencies are correct"" 2017-05-08 21:13:53 +00:00
Dan Willemsen
4068a5db6c Revert "Ensure environment dependencies are correct"
This reverts commit e78473c33c.

Reason for revert: Broke mac builds, which rely on $PATH to find xcrun

Change-Id: I58711b94d472ab828823afaf53394a1aa869f3f5
2017-05-08 21:11:55 +00:00
Dan Willemsen
6a76b144a9 Merge "Ensure environment dependencies are correct" 2017-05-08 20:50:16 +00:00
Treehugger Robot
6d9d964eb7 Merge changes I7270fef0,Ie8fba06c
* changes:
  Automatically add frameworks/rs includes to modules with .rs files
  androidmk: support cfi, diag, and misc_undefined properties
2017-05-08 19:54:40 +00:00
Colin Cross
2101f4aa06 Automatically add frameworks/rs includes to modules with .rs files
C++ sources generated from .rs files need includes in
frameworks/rs and frameworks/rs/cpp.  Add them automatically
instead of requiring modules to add them manually.

Test: builds
Change-Id: I7270fef0b36c1956475d6d83ab8aa2e3c362e688
2017-05-08 19:37:12 +00:00
Colin Cross
a8688c4652 androidmk: support cfi, diag, and misc_undefined properties
Support more sanitize related properties.

Bug: 37547301
Test: androidmk_test.go
Change-Id: Ie8fba06c36fc121912c65a77ee495ceef4962df0
2017-05-08 19:36:57 +00:00
Dan Willemsen
95f4dbb330 Ignore disabled modules when generating ndk sysroot
When linux_bionic is turned on, there are two host modules for the
preprocessed ndk headers, but only one is enabled. So exit early for the
disabled one.

Bug: 31559095
Test: Diff out/soong/build.ninja before/after, no change.
Test: Turn on linux_bionic, no longer see a panic.
Change-Id: I204a884a83ccf21c96088e97c19a0cffe029ec99
2017-05-05 23:26:01 -07:00
Dan Willemsen
e78473c33c Ensure environment dependencies are correct
Store the os Environment on init, then clear it so that it's only
available via a call to Config.Getenv(). That way we're guaranteed to
know about all dependencies.

Test: Add some os.Getenv/Config.Getenv calls and ensure correct behavior
Test: cs/ shows no os.Getenv / os.LookupEnv / os.ExpandEnv calls that
would be affected
Change-Id: Ic519e0c871f340e752df03b8af0599d47c1d91d8
2017-05-05 23:08:16 -07:00
Treehugger Robot
528d5e5d28 Merge changes Ic3a55909,I70462293
* changes:
  Always run asan ckati on the build servers
  Add exec.Cmd wrapper for logging / sandboxing
2017-05-06 01:50:31 +00:00
Colin Cross
03cc1b6d97 Use -isystem to export ndk stl includes
Hides:
In file included from frameworks/rs/cpu_ref/rsCpuExecutable.cpp:4:
In file included from prebuilts/ndk/current/sources/cxx-stl/llvm-libc++/include/fstream:169:
In file included from prebuilts/ndk/current/sources/cxx-stl/llvm-libc++/include/ostream:138:
In file included from prebuilts/ndk/current/sources/cxx-stl/llvm-libc++/include/ios:216:
In file included from prebuilts/ndk/current/sources/cxx-stl/llvm-libc++/include/__locale:21:
prebuilts/ndk/current/sources/android/support/include/locale.h:97:9: error: 'LC_GLOBAL_LOCALE' macro redefined [-Werror,-Wmacro-redefined]
       ^
out-soong/ndk/sysroot/usr/include/locale.h:115:9: note: previous definition is here
       ^

Test: builds
Change-Id: I766e8645db33e6e71f44a131aa0b1649d83e8c51
2017-05-05 18:22:30 -07:00
Colin Cross
b2123aa1df Add a variable to disable modules in the PDK
Not plumbed through yet, but will at least prevent us from losing
track of which modules will not build in the PDK.

Test: builds
Change-Id: I4c1a0e39104b104dc1ff23c4391d5e83b3002d3a
2017-05-05 18:22:30 -07:00
Colin Cross
c602b7d881 Allow "required" property to vary by arch
Renderscript needs to require different bitcode files based
on architecture.

Test: builds
Change-Id: Ifffa16c892a7f8e309f1c5652472fba44cd2e786
2017-05-05 18:22:30 -07:00
Colin Cross
2a252bef50 Support .rs and .fs files in cc_* module srcs lists
Translate .rs and .fs files to .cpp files using llvm-rs-cc.

Test: builds
Change-Id: I242cea0d09c9985730a512cec7705c3f1479f4ed
2017-05-05 18:22:30 -07:00
Colin Cross
7e0eaf15b9 Support %s in product variable properties
Support using strings as product variable substitutions, and
add tests for printfIntoProperty.

Test: varaible_test.go
Change-Id: I06cfadfb1d3fc81da72fb71323706df20426c8b7
2017-05-05 18:22:30 -07:00
Colin Cross
2030a8cbcb Merge "Support override_rs_driver product variable" 2017-05-06 00:15:02 +00:00
Dan Willemsen
a3e6c520ac Always run asan ckati on the build servers
We're seeing some rare crashes that appear to be stack overflows. To get
better debugging, use the asan version of ckati when 'dist' is
specified.

Bug: 36182021
Test: m -j
Test: m -j dist
Test: make -j dist (w/o lunch)
Change-Id: Ic3a5590974bfd718bf1929355d344b5933ac1d4f
2017-05-05 16:59:10 -07:00
Dan Willemsen
269a8c78e7 Add exec.Cmd wrapper for logging / sandboxing
Wrap os/exec.Cmd to use our Context and Config interfaces for automatic
logging and error handling. It also simplifies environment modification
based on the Config's environment.

This also adds sandboxing on Macs using sandbox-exec. A simple profile
is provided that only logs on violations, though multiproduct_kati on
AOSP has no violations. This isn't applied to ninja, only make / soong /
kati to start with. I measured <5% time increase in reading all
makefiles, and no noticable difference when kati doesn't regenerate.

I'd like to spin up a process to dump violation logs into our log file,
but the log reporting changed over the range of Mac versions that we
support, so that's going to be more complicated. Opening Console.app
works in all cases if you're local -- just search/filter for sandbox.

Linux sandboxing will be implemented later -- the sandbox definition is
opaque enough to support a different implementation.

Test: multiproduct_kati on AOSP master on Mac
Change-Id: I7046229333d0dcc8f426a493e0f7380828879f17
2017-05-05 16:25:40 -07:00
Treehugger Robot
a731969dc1 Merge "Supported python build in host side." 2017-05-05 22:14:19 +00:00
Nan Zhang
db0b9a3cf3 Supported python build in host side.
The base module handles all the common functionalites, such as version
compatibilty check, version variations split, source file format check,
source/data file duplicate check.

The library/binary module focuses on how to generate binary build actions,
such as setting up stub script, zipping, filling in __init__.py in
runfiles dir tree.

Bug: b/31676493
Test: go test under python package

Change-Id: I06608369f350f7195873d459e1c8d1bdb811e77e
2017-05-05 13:27:56 -07:00
Josh Gao
06ec87082f Merge "Globally enable libcxx's thread safety annotations." 2017-05-04 23:47:32 +00:00
Colin Cross
954364256f Support override_rs_driver product variable
Test: builds
Change-Id: Ifd9bf09bb54418d62f6886b0d8ed84dd493a3162
2017-05-04 15:44:11 -07:00
Colin Cross
37a2aeb95a Merge "Remove -isystem frameworks/av/include" 2017-05-04 04:09:43 +00:00
George Burgess IV
561a3fe4af soong: add __clang_analyzer__ to clang-tidy
We have code that acts slightly differently when the static analyzer is
running, so that it can produce more accurate diagnostics (e.g. less
false positives). It uses __clang_analyzer__ to detect the static
analyzer.

When the static analyzer is run via clang-tidy, __clang_analyzer__
doesn't get defined.

Bug: None
Test: WITH_TIDY=1 m. clang-tidy now acts as expected in code made for
the static analyzer

Change-Id: I460ff410640524633c0a60b71d34927d17e1ed98
2017-05-04 01:19:02 +00:00
Colin Cross
307d145ed5 Remove -isystem frameworks/av/include
Remove the last -isystem for global include directory (besides
bionic).

Bug: 31751828
Test: builds
Change-Id: I8f38332de8f0cffd5c56b9b5ebeacbaaf3a4faea
Merged-In: I8f38332de8f0cffd5c56b9b5ebeacbaaf3a4faea
(cherry picked from commit d13a457a45)
2017-05-03 23:56:47 +00:00
Colin Cross
bbc9f4d586 Export proto deps from reused libraries too
The deps have to be exported alongside the headers to make sure
the headers are in place when a module that depends on the shared
library builds.

Bug: 37555583
Test: use protos in a cc_library
Change-Id: I8f5b8d7937868458c1cf02b990a348aa395634da
2017-05-03 16:39:31 -07:00
Colin Cross
7c34c4c8eb Merge "Check reused source files in hasSrcExt" 2017-05-03 21:24:09 +00:00
Colin Cross
10d2231d44 Check reused source files in hasSrcExt
hasSrcExt is used to determine extra flags and dependencies when
generated files are used.  If the generated files are being handled
in a static library whose objects are reused in a shared library,
the flags and dependencies still need to apply.  Instead of clearing
the source files in the shared library, move them to an OriginalSrcs
property, and check that in hasSrcExt along with the Srcs property.

Also pass extra exported include directories from the static
library to the shared library.

Bug: 37555583
Test: use protos in a cc_library
Change-Id: I709779ec03b66b220b7bd58a1f6f0b9b5067d955
2017-05-03 12:28:27 -07:00
Treehugger Robot
12d713e434 Merge "Supported minor features in soong_zip" 2017-05-03 08:23:56 +00:00