Commit Graph

6708 Commits

Author SHA1 Message Date
Dan Willemsen
58be946140 Merge changes from topic "rm_ONE_SHOT_MAKEFILE"
am: 8b8b9b6bf8

Change-Id: I5764b2ce848710584d2d837e9e33fb9fb71c8d33
2019-08-01 14:31:32 -07:00
Dan Willemsen
8b8b9b6bf8 Merge changes from topic "rm_ONE_SHOT_MAKEFILE"
* changes:
  Remove [jni_]link_type files
  Remove support for ONE_SHOT_MAKEFILE
2019-08-01 21:11:02 +00:00
Tao Bao
7d223c6ddd releasetools: Add FORCE_RUN_RELEASETOOLS.
This allows test runner to take care of the external tools setup, then
to start the test without a lunch'd environment. This is needed before
having supports like python_test_helper that packs dependents into
releasetools_test module.

Bug: 138791766
Test: Don't lunch any target. Use `FORCE_RUN_RELEASETOOLS=1 python -m
      unittest -v test_add_img_to_target_files` and check there's no
      skipped testcases.
Change-Id: Iafeaba54fc228d8e1fc5ff3c91f7ffebef18ad7f
2019-08-01 12:22:54 -07:00
Mitchell Wills
e2f0f13049 Merge "Add build script to create a self extracting archive with a click through license"
am: 9a1d60f5cb

Change-Id: I412c669ed7291288911f258232519c20fa136f8c
2019-07-31 19:43:27 -07:00
Treehugger Robot
9a1d60f5cb Merge "Add build script to create a self extracting archive with a click through license" 2019-08-01 02:17:33 +00:00
Dan Willemsen
0df0f6bbf1 Remove [jni_]link_type files
Now that mm/ONE_SHOT_MAKEFILE have been removed, we can expect to know
about all of our dependencies at the end of the build.

This removes 19k nodes from our build graph (aosp-master
aosp_arm64-eng), though in a default build, only 3k of those are used.

Test: ALLOW_MISSING_DEPENDENCIES=true, then trigger a missing dependency
Test: treehugger
Test: create link_type files, then apply CleanSpec.mk, ensure they're removed
Change-Id: I9506331e4a9911d2f26e59a2f72a97aef1644073
2019-07-31 18:09:22 -07:00
Dan Willemsen
93b7c1b547 Merge "Stop supporting wrapping the build with make"
am: d69e849c9b

Change-Id: If6e369f82348d9d7aebe37104072a6878f7281ea
2019-07-30 15:44:36 -07:00
Dan Willemsen
d69e849c9b Merge "Stop supporting wrapping the build with make" 2019-07-30 21:48:52 +00:00
Dan Willemsen
f02f1f16f0 Merge "Remove the PRODUCT-* goal"
am: a2d19cfd05

Change-Id: I8c83c177104117a88fa9822c48ec7fe45b69365c
2019-07-30 10:34:33 -07:00
Dan Willemsen
a2d19cfd05 Merge "Remove the PRODUCT-* goal" 2019-07-30 17:04:07 +00:00
Dan Willemsen
893948955d Stop supporting wrapping the build with make
The build servers are no longer using this. It's hard to search for
others using it -- I've cleaned them up as I've run across them, but
we'll just have to see who breaks once this goes in.

This was the last user of makeparallel, so we can remove it.

Test: make
Test: treehugger
Change-Id: If6df3f1a67d6a6df36afaa4b07cd88a48a364fe1
2019-07-29 22:43:35 -07:00
Daniel Norman
3f9554b59a Merge "Adds new merge builds script for use in merging two non-dist builds."
am: a85f0ee8a2

Change-Id: Ib7b66d5ee82df1d9f31d521917490ac1bac4a4e2
2019-07-29 17:36:57 -07:00
Dan Willemsen
8a5d597bbf Remove the PRODUCT-* goal
I've switched all the build server configs to using TARGET_PRODUCT /
TARGET_BUILD_VARIANT and explicit goals instead.

Remove tools/check_builds.sh which relied on this, but hasn't been
touched in a long time.

Test: m PRODUCT-test
Test: treehugger
Change-Id: If5f8c714567b33aeb38223c176ca24ea649eb57d
2019-07-29 14:27:04 -07:00
Mitchell Wills
1c790ca655 Add build script to create a self extracting archive with a click through license
Bug: 130257221
Test: generated an archive using the script and ran the result
Change-Id: If1147cd41fa939b0d9958196e627042b5731dc14
2019-07-29 11:04:57 -07:00
Daniel Norman
bfc51efa97 Adds new merge builds script for use in merging two non-dist builds.
Bug: 137853921
Test: python -m unittest test_common
Test: python -m unittest test_merge_target_files
Test: Built two partial builds without dist. Ran out/host/linux-x86/bin/merge_builds.
Flashed using `fastboot flashall`. Device boots.
Change-Id: Iffd0a447cdf19a7775a813b4b896178aa6f861f3
2019-07-29 16:30:56 +00:00
Tao Bao
ac280723c2 Merge "releasetools: Move recovery-two-step.img to OTA/."
am: 98efd8290e

Change-Id: I21419dc4a13271c589b8bcb4a4be7da02ef1587d
2019-07-28 11:07:07 -07:00
Tao Bao
0480850f0b releasetools: Move recovery-two-step.img to OTA/.
It used to be packed at IMAGES/recovery-two-step.img, but to serve OTA
purpose only.

Test: `m dist` with a non-A/B target. Check the file in the generated
      target_files.zip.
Test: Create two-step package. Check that recovery-two-step.img is used.
Change-Id: Iec6a73c682e0f844cd8c0b758c9470fa35dd15d8
2019-07-26 13:59:11 -07:00
Tao Bao
706eb3c566 Merge "releasetools: Fix the use of StringIO."
am: 6b466c8f56

Change-Id: I5669a070c0d98239263557c005305002739d1c1d
2019-07-25 09:55:09 -07:00
Tao Bao
bb73388acf releasetools: Fix the use of StringIO.
Based on the actual semantics, it actually wants an in-memory _bytes_
buffer (io.BytesIO), especially when running with Python 3. This CL
fixes the issue and adds a unittest.

Bug: 131631303
Test: python -m unittest test_sign_target_files_apks
Test: python3 -m unittest test_sign_target_files_apks
Change-Id: I3fb067acc26713f1842e831225607779fd0d1b7e
2019-07-24 23:34:25 -07:00
Tom Cherry
6cf1b11ead Generate /etc/{passwd,group} for all partitions
Bug: 73062966
Test: can load AIDs from partitions other than vendor
Test: bionic-unit-tests
Merged-In: Ia85abbeefe5a945369970f2aef42692e07ab8c09
Change-Id: Ia85abbeefe5a945369970f2aef42692e07ab8c09
(cherry picked from commit 8a54ec8482)
2019-07-23 17:07:11 +00:00
Tom Cherry
6722de1277 Merge "Generate /etc/{passwd,group} for all partitions" 2019-07-23 16:40:01 +00:00
Tom Cherry
fb303a5903 Generate /etc/{passwd,group} for all partitions
Bug: 73062966
Test: can load AIDs from partitions other than vendor
Test: bionic-unit-tests
Merged-In: Ia85abbeefe5a945369970f2aef42692e07ab8c09
Change-Id: Ia85abbeefe5a945369970f2aef42692e07ab8c09
2019-07-22 21:41:17 +00:00
changho.shin
0f12536e6b Switch to aapt2 for getting minSdkVersion
aapt does not support parsing some referencing attributes.
Switch to latest tool, aapt2, which works better.

Test: aapt2 dump badging test.apk and returned 0.
Test: Run check_target_files_signatures.py with a target_files.zip.
Test: Run sign_target_files_apks.py with a target_files.zip.
Change-Id: Ib3a4740506f29ebab6930767f2aa8a0b5c4ba053
2019-07-19 20:26:42 +00:00
Daniel Norman
36f8163589 Merge "Clean up merge_target_files.py."
am: ff165e31fd

Change-Id: I4bb6e084b0beb4a57df38a9e083b7e4900320a63
2019-07-19 07:41:27 -07:00
Tao Bao
e8aa7dcf19 Merge "Build build_image and build_super_image as modules."
am: 25cf50f582

Change-Id: Idb34ce99b01fe432e0fe2f7824a87cc8fa3541bf
2019-07-19 07:40:39 -07:00
Elliott Hughes
b5c68a02ae Merge "Remove perfprofd reference."
am: a627e9056b

Change-Id: I9f59616bb617d1c590eb6c923a5d8453ab4c2577
2019-07-19 07:38:56 -07:00
Daniel Norman
ff165e31fd Merge "Clean up merge_target_files.py." 2019-07-19 03:15:29 +00:00
Daniel Norman
4cc9df660b Clean up merge_target_files.py.
- Removes functions that can be replaced with one-line external methods
- Moves read_config_list to common alongside similar methods LoadDictionaryFrom*
- Runs pyformat on merge_target_files.py

Bug: 137853921
Test: python -m unittest test_merge_target_files
Test: Using merge_target_files.py to create a merged build, & booting.
Change-Id: I833c1086db41e1374057cc7447fc50d1915734a7
2019-07-18 13:04:54 -07:00
Tao Bao
2bbb07c53c Build build_image and build_super_image as modules.
Bug: 63866463
Test: TreeHugger
Test: `atest releasetools_test`
Test: `atest releasetools_py3_test`
Change-Id: I2059a4ced709d1b2ee331a9aaaa5ca30db4ebf6b
2019-07-18 10:27:38 -07:00
Treehugger Robot
a627e9056b Merge "Remove perfprofd reference." 2019-07-18 17:22:43 +00:00
Tao Bao
b9c7ed6314 Merge "logging: set stdout and stderr to None in some cases"
am: e733ed7e6b

Change-Id: Ifa43d7a811b2c288de1a2ee3a10fee0efaa43661
2019-07-17 08:55:25 -07:00
Tao Bao
e733ed7e6b Merge "logging: set stdout and stderr to None in some cases" 2019-07-17 15:40:59 +00:00
Elliott Hughes
36d86c1670 Remove perfprofd reference.
Dead and gone.

Test: treehugger
Change-Id: I8575fd9efaa371d61fac5204ee4d982e985a9a56
2019-07-17 08:16:29 -07:00
Tianjie Xu
bcc3554b74 Merge "Factor out the image classes to break circular dependency"
am: dcb6644719

Change-Id: I16371818e6e022ee75187b9c9e104836a72dd983
2019-07-16 13:30:24 -07:00
Tianjie Xu
41976c725c Factor out the image classes to break circular dependency
This helps to break the circular dependency between common and
blockimgdiff.

Bug: 32379627
Test: unit tests pass
Change-Id: I90b5ff34782acbfac86f36265bd96c207d898bf6
2019-07-15 17:02:23 -07:00
Andrew Chant
1db284ff10 Merge "Catch unicode decode errors search for kernel ver"
am: 2b4828a564

Change-Id: Ieed4020a6490d44c4be1780eec08e501ed1e2219
2019-07-15 11:58:41 -07:00
Treehugger Robot
2b4828a564 Merge "Catch unicode decode errors search for kernel ver" 2019-07-15 18:33:57 +00:00
Elliott Hughes
7a45e0ccb9 Merge "Stop building unnecessary tarballs."
am: 9a4b84de56

Change-Id: I9465ce123bc1d5ab9662069af6181dd7382eff57
2019-07-12 08:10:05 -07:00
Tao Bao
0f6c1d2b47 Merge "releasetools: Ignore nonexistent APEX overrides."
am: d077c5b88e

Change-Id: Idc8e680e3140599f81c5f9cdc33e2a647a95a300
2019-07-12 08:00:29 -07:00
Elliott Hughes
9a4b84de56 Merge "Stop building unnecessary tarballs." 2019-07-12 14:52:53 +00:00
Tao Bao
3422309d6d releasetools: Ignore nonexistent APEX overrides.
This allows sharing the same signing config on different target_files
zips. Nonexistent APEX will be ignored with a warning.

Bug: 137249701
Test: Run sign_target_files_apks with APEX overrides.
Change-Id: I2bad0f5c00753ed36ec5ae3431c7dc2ff1fc3e9c
Merged-In: I2bad0f5c00753ed36ec5ae3431c7dc2ff1fc3e9c
(cherry picked from commit b369c7226a)
2019-07-12 00:25:41 -07:00
Elliott Hughes
440c99f1f2 Stop building unnecessary tarballs.
Test: treehugger
Change-Id: Iaba2aa6d09ad891f37171cc76778d08c78c78f93
2019-07-12 05:04:12 +00:00
Vic Yang
ea3740650c Merge "Implement range-based pin list"
am: f90ec83802

Change-Id: Ie5fcabc992569a05d4d12aa3689aa2bcad0d81d4
2019-07-11 15:44:27 -07:00
Treehugger Robot
f90ec83802 Merge "Implement range-based pin list" 2019-07-11 22:21:03 +00:00
Tao Bao
213210c4d0 Merge "releasetools: Replace iteritems with items."
am: 6623fdda5a

Change-Id: I35c9e3b39897d20463aa12252a6118b427e64dfc
2019-07-11 11:39:41 -07:00
Regnier, Philippe
2f7e11ef27 logging: set stdout and stderr to None in some cases
For very long processes, we might want to keep stdout and stderr
by default to None.
So no redirection will occur in the child process as explained in:
https://docs.python.org/2/library/subprocess.html
That will result in the child process stdin and stderr to be same
than in common.py and avoid to have the logs blocked during the
child process execution and flushed only when child process terminates.
Since the logs are continously displayed, it allows to easily confirm
that the process is not blocked.

Bug: 133380588
Test: generate iota & Check that the logs are not blocked.

Change-Id: I6d6cb56547bf3a4a4334dfa22b6b2b05d2c36a5e
Signed-off-by: Regnier, Philippe <philippe.regnier@intel.com>
2019-07-11 17:05:17 +08:00
Tao Bao
3888428f29 releasetools: Replace iteritems with items.
While in theory this could incur memory overhead for Python 2, the
impact is low for the existing use cases (plus we're moving away from
Python 2).

Bug: 131631303
Test: No additional occurrence of iteritems.
Test: Build with Python 3.
Change-Id: I0205c9edf25f46e3d85967c7dd2c1af035757741
2019-07-10 22:29:20 -07:00
Tao Bao
d2ff3b5dca Merge "Additionally run releasetools_test with Python 3."
am: 576ae9628b

Change-Id: I090d9f67524c636967400d57164f26158562d3a9
2019-07-10 13:41:57 -07:00
Tao Bao
576ae9628b Merge "Additionally run releasetools_test with Python 3." 2019-07-10 18:22:37 +00:00
Vic Yang
baa9f50cb2 Implement range-based pin list
This extends the original pin list generation to support specifying a
range within a file.  If any part of a file is pinned, its local file
header in the APK file is pinned as well.

Test: Build cuttlefish.
Bug: 136040313
Bug: 135953430
Change-Id: I6de8c2b4c2f35c6bf257f61414c6bce761233e51
2019-07-09 10:37:19 -07:00