Commit Graph

6498 Commits

Author SHA1 Message Date
Dan Willemsen
f02f1f16f0 Merge "Remove the PRODUCT-* goal"
am: a2d19cfd05

Change-Id: I8c83c177104117a88fa9822c48ec7fe45b69365c
2019-07-30 10:34:33 -07:00
Dan Willemsen
a2d19cfd05 Merge "Remove the PRODUCT-* goal" 2019-07-30 17:04:07 +00:00
Daniel Norman
3f9554b59a Merge "Adds new merge builds script for use in merging two non-dist builds."
am: a85f0ee8a2

Change-Id: Ib7b66d5ee82df1d9f31d521917490ac1bac4a4e2
2019-07-29 17:36:57 -07:00
Dan Willemsen
8a5d597bbf Remove the PRODUCT-* goal
I've switched all the build server configs to using TARGET_PRODUCT /
TARGET_BUILD_VARIANT and explicit goals instead.

Remove tools/check_builds.sh which relied on this, but hasn't been
touched in a long time.

Test: m PRODUCT-test
Test: treehugger
Change-Id: If5f8c714567b33aeb38223c176ca24ea649eb57d
2019-07-29 14:27:04 -07:00
Daniel Norman
bfc51efa97 Adds new merge builds script for use in merging two non-dist builds.
Bug: 137853921
Test: python -m unittest test_common
Test: python -m unittest test_merge_target_files
Test: Built two partial builds without dist. Ran out/host/linux-x86/bin/merge_builds.
Flashed using `fastboot flashall`. Device boots.
Change-Id: Iffd0a447cdf19a7775a813b4b896178aa6f861f3
2019-07-29 16:30:56 +00:00
Tao Bao
ac280723c2 Merge "releasetools: Move recovery-two-step.img to OTA/."
am: 98efd8290e

Change-Id: I21419dc4a13271c589b8bcb4a4be7da02ef1587d
2019-07-28 11:07:07 -07:00
Tao Bao
0480850f0b releasetools: Move recovery-two-step.img to OTA/.
It used to be packed at IMAGES/recovery-two-step.img, but to serve OTA
purpose only.

Test: `m dist` with a non-A/B target. Check the file in the generated
      target_files.zip.
Test: Create two-step package. Check that recovery-two-step.img is used.
Change-Id: Iec6a73c682e0f844cd8c0b758c9470fa35dd15d8
2019-07-26 13:59:11 -07:00
Tao Bao
706eb3c566 Merge "releasetools: Fix the use of StringIO."
am: 6b466c8f56

Change-Id: I5669a070c0d98239263557c005305002739d1c1d
2019-07-25 09:55:09 -07:00
Tao Bao
bb73388acf releasetools: Fix the use of StringIO.
Based on the actual semantics, it actually wants an in-memory _bytes_
buffer (io.BytesIO), especially when running with Python 3. This CL
fixes the issue and adds a unittest.

Bug: 131631303
Test: python -m unittest test_sign_target_files_apks
Test: python3 -m unittest test_sign_target_files_apks
Change-Id: I3fb067acc26713f1842e831225607779fd0d1b7e
2019-07-24 23:34:25 -07:00
Tom Cherry
6cf1b11ead Generate /etc/{passwd,group} for all partitions
Bug: 73062966
Test: can load AIDs from partitions other than vendor
Test: bionic-unit-tests
Merged-In: Ia85abbeefe5a945369970f2aef42692e07ab8c09
Change-Id: Ia85abbeefe5a945369970f2aef42692e07ab8c09
(cherry picked from commit 8a54ec8482)
2019-07-23 17:07:11 +00:00
Tom Cherry
6722de1277 Merge "Generate /etc/{passwd,group} for all partitions" 2019-07-23 16:40:01 +00:00
Tom Cherry
fb303a5903 Generate /etc/{passwd,group} for all partitions
Bug: 73062966
Test: can load AIDs from partitions other than vendor
Test: bionic-unit-tests
Merged-In: Ia85abbeefe5a945369970f2aef42692e07ab8c09
Change-Id: Ia85abbeefe5a945369970f2aef42692e07ab8c09
2019-07-22 21:41:17 +00:00
changho.shin
0f12536e6b Switch to aapt2 for getting minSdkVersion
aapt does not support parsing some referencing attributes.
Switch to latest tool, aapt2, which works better.

Test: aapt2 dump badging test.apk and returned 0.
Test: Run check_target_files_signatures.py with a target_files.zip.
Test: Run sign_target_files_apks.py with a target_files.zip.
Change-Id: Ib3a4740506f29ebab6930767f2aa8a0b5c4ba053
2019-07-19 20:26:42 +00:00
Daniel Norman
36f8163589 Merge "Clean up merge_target_files.py."
am: ff165e31fd

Change-Id: I4bb6e084b0beb4a57df38a9e083b7e4900320a63
2019-07-19 07:41:27 -07:00
Tao Bao
e8aa7dcf19 Merge "Build build_image and build_super_image as modules."
am: 25cf50f582

Change-Id: Idb34ce99b01fe432e0fe2f7824a87cc8fa3541bf
2019-07-19 07:40:39 -07:00
Elliott Hughes
b5c68a02ae Merge "Remove perfprofd reference."
am: a627e9056b

Change-Id: I9f59616bb617d1c590eb6c923a5d8453ab4c2577
2019-07-19 07:38:56 -07:00
Daniel Norman
ff165e31fd Merge "Clean up merge_target_files.py." 2019-07-19 03:15:29 +00:00
Daniel Norman
4cc9df660b Clean up merge_target_files.py.
- Removes functions that can be replaced with one-line external methods
- Moves read_config_list to common alongside similar methods LoadDictionaryFrom*
- Runs pyformat on merge_target_files.py

Bug: 137853921
Test: python -m unittest test_merge_target_files
Test: Using merge_target_files.py to create a merged build, & booting.
Change-Id: I833c1086db41e1374057cc7447fc50d1915734a7
2019-07-18 13:04:54 -07:00
Tao Bao
2bbb07c53c Build build_image and build_super_image as modules.
Bug: 63866463
Test: TreeHugger
Test: `atest releasetools_test`
Test: `atest releasetools_py3_test`
Change-Id: I2059a4ced709d1b2ee331a9aaaa5ca30db4ebf6b
2019-07-18 10:27:38 -07:00
Treehugger Robot
a627e9056b Merge "Remove perfprofd reference." 2019-07-18 17:22:43 +00:00
Tao Bao
b9c7ed6314 Merge "logging: set stdout and stderr to None in some cases"
am: e733ed7e6b

Change-Id: Ifa43d7a811b2c288de1a2ee3a10fee0efaa43661
2019-07-17 08:55:25 -07:00
Tao Bao
e733ed7e6b Merge "logging: set stdout and stderr to None in some cases" 2019-07-17 15:40:59 +00:00
Elliott Hughes
36d86c1670 Remove perfprofd reference.
Dead and gone.

Test: treehugger
Change-Id: I8575fd9efaa371d61fac5204ee4d982e985a9a56
2019-07-17 08:16:29 -07:00
Tianjie Xu
bcc3554b74 Merge "Factor out the image classes to break circular dependency"
am: dcb6644719

Change-Id: I16371818e6e022ee75187b9c9e104836a72dd983
2019-07-16 13:30:24 -07:00
Tianjie Xu
41976c725c Factor out the image classes to break circular dependency
This helps to break the circular dependency between common and
blockimgdiff.

Bug: 32379627
Test: unit tests pass
Change-Id: I90b5ff34782acbfac86f36265bd96c207d898bf6
2019-07-15 17:02:23 -07:00
Andrew Chant
1db284ff10 Merge "Catch unicode decode errors search for kernel ver"
am: 2b4828a564

Change-Id: Ieed4020a6490d44c4be1780eec08e501ed1e2219
2019-07-15 11:58:41 -07:00
Treehugger Robot
2b4828a564 Merge "Catch unicode decode errors search for kernel ver" 2019-07-15 18:33:57 +00:00
Elliott Hughes
7a45e0ccb9 Merge "Stop building unnecessary tarballs."
am: 9a4b84de56

Change-Id: I9465ce123bc1d5ab9662069af6181dd7382eff57
2019-07-12 08:10:05 -07:00
Tao Bao
0f6c1d2b47 Merge "releasetools: Ignore nonexistent APEX overrides."
am: d077c5b88e

Change-Id: Idc8e680e3140599f81c5f9cdc33e2a647a95a300
2019-07-12 08:00:29 -07:00
Elliott Hughes
9a4b84de56 Merge "Stop building unnecessary tarballs." 2019-07-12 14:52:53 +00:00
Tao Bao
3422309d6d releasetools: Ignore nonexistent APEX overrides.
This allows sharing the same signing config on different target_files
zips. Nonexistent APEX will be ignored with a warning.

Bug: 137249701
Test: Run sign_target_files_apks with APEX overrides.
Change-Id: I2bad0f5c00753ed36ec5ae3431c7dc2ff1fc3e9c
Merged-In: I2bad0f5c00753ed36ec5ae3431c7dc2ff1fc3e9c
(cherry picked from commit b369c7226a)
2019-07-12 00:25:41 -07:00
Elliott Hughes
440c99f1f2 Stop building unnecessary tarballs.
Test: treehugger
Change-Id: Iaba2aa6d09ad891f37171cc76778d08c78c78f93
2019-07-12 05:04:12 +00:00
Vic Yang
ea3740650c Merge "Implement range-based pin list"
am: f90ec83802

Change-Id: Ie5fcabc992569a05d4d12aa3689aa2bcad0d81d4
2019-07-11 15:44:27 -07:00
Treehugger Robot
f90ec83802 Merge "Implement range-based pin list" 2019-07-11 22:21:03 +00:00
Tao Bao
213210c4d0 Merge "releasetools: Replace iteritems with items."
am: 6623fdda5a

Change-Id: I35c9e3b39897d20463aa12252a6118b427e64dfc
2019-07-11 11:39:41 -07:00
Regnier, Philippe
2f7e11ef27 logging: set stdout and stderr to None in some cases
For very long processes, we might want to keep stdout and stderr
by default to None.
So no redirection will occur in the child process as explained in:
https://docs.python.org/2/library/subprocess.html
That will result in the child process stdin and stderr to be same
than in common.py and avoid to have the logs blocked during the
child process execution and flushed only when child process terminates.
Since the logs are continously displayed, it allows to easily confirm
that the process is not blocked.

Bug: 133380588
Test: generate iota & Check that the logs are not blocked.

Change-Id: I6d6cb56547bf3a4a4334dfa22b6b2b05d2c36a5e
Signed-off-by: Regnier, Philippe <philippe.regnier@intel.com>
2019-07-11 17:05:17 +08:00
Tao Bao
3888428f29 releasetools: Replace iteritems with items.
While in theory this could incur memory overhead for Python 2, the
impact is low for the existing use cases (plus we're moving away from
Python 2).

Bug: 131631303
Test: No additional occurrence of iteritems.
Test: Build with Python 3.
Change-Id: I0205c9edf25f46e3d85967c7dd2c1af035757741
2019-07-10 22:29:20 -07:00
Tao Bao
d2ff3b5dca Merge "Additionally run releasetools_test with Python 3."
am: 576ae9628b

Change-Id: I090d9f67524c636967400d57164f26158562d3a9
2019-07-10 13:41:57 -07:00
Tao Bao
576ae9628b Merge "Additionally run releasetools_test with Python 3." 2019-07-10 18:22:37 +00:00
Vic Yang
baa9f50cb2 Implement range-based pin list
This extends the original pin list generation to support specifying a
range within a file.  If any part of a file is pinned, its local file
header in the APK file is pinned as well.

Test: Build cuttlefish.
Bug: 136040313
Bug: 135953430
Change-Id: I6de8c2b4c2f35c6bf257f61414c6bce761233e51
2019-07-09 10:37:19 -07:00
Andrew Chant
5d323c1714 Catch unicode decode errors search for kernel ver
If a kernel happens to decode to gibberish (including
non-ascii bytes), catch the decoding error when
searching for the kernel version.

Bug: 137041171
Merged-In: Ic035b3a5c8c80025cb3cede7b0fdcf8a2e5a35fd
Change-Id: Ic035b3a5c8c80025cb3cede7b0fdcf8a2e5a35fd
2019-07-09 17:02:48 +00:00
Justin Yun
8f11ad5d40 Merge "Rename product_services to system_ext"
am: b7a50ece30

Change-Id: I7a56d76d3543b2bc4b1d0b24e54e8dd82ab7f15a
2019-07-09 02:14:55 -07:00
Justin Yun
6151e3f1ea Rename product_services to system_ext
Bug: 134359158
Test: build and check if system_ext.img is created
Change-Id: I67f2e95dd29eac6a28e07e24ea973d3a134c3bfc
2019-07-09 08:57:19 +00:00
Tao Bao
ca82fc90f7 Additionally run releasetools_test with Python 3.
Bug: 131631303
Test: TreeHugger
Test: `atest releasetools_test`
Test: `atest releasetools_py3_test`
Change-Id: Ie00f48835aae41bc69ab8554b856a62aa3b497cc
2019-07-09 05:14:31 +00:00
Tao Bao
ec2f3da2a2 Merge "releasetools: Make merge_target_files and its test Python 3 compatible."
am: 6e3ae4e849

Change-Id: I0ee3f8801e194f140f32a4123100aa0e4f449a70
2019-07-08 21:19:53 -07:00
Tao Bao
2ad4b82d45 releasetools: Make merge_target_files and its test Python 3 compatible.
And a few minor clean-ups to the styling.

Bug: 131631303
Test: python -m unittest test_merge_target_files
Test: python3 -m unittest test_merge_target_files
Test: Use `python merge_target_files` to merge two target_files zips.
Test: Use `python3 merge_target_files` to merge two target_files zips.
Change-Id: I8502dfb243408f658d022e8d5e5fbb60066e4ff0
2019-07-08 16:06:15 -07:00
Kiyoung Kim
dc6566f3b2 Merge "Split merge_target_files function"
am: 09736b4aaf

Change-Id: Ic077a4bf1276ead627c1d09f67636626623a45ab
2019-07-07 23:51:33 -07:00
Kiyoung Kim
7cbeda728d Split merge_target_files function
Split merge_target_files function into several steps, so we can increate
readability and add conditional flows for other *SSI mixed build with less effort

Test: m -j & atest passed
Change-Id: I558f9dd5bca31b132a09cb36d9dfcd30c92efbc9
2019-07-08 12:13:05 +09:00
TreeHugger Robot
218adad125 Merge "DO NOT MERGE - Merge qt-dev-plus-aosp-without-vendor (5699924) into stage-aosp-master" into stage-aosp-master 2019-07-02 07:40:58 +00:00
Tao Bao
48a2feb670 Change the condition for building super_empty.img.
This CL changes the condition for building super_empty.img from
PRODUCT_BUILD_SUPER_PARTITION to PRODUCT_USE_DYNAMIC_PARTITIONS, as a
follow-up to the change in [1].

With the CL in [1], it skips building super.img and super_empty.img both
when turning off PRODUCT_BUILD_SUPER_PARTITION. However, the latter
should be mandatory whenever dynamic partitions is enabled. Because
fastboot relies on this file to properly flash dynamic partitions. Plus,
the cost for building super_empty.img is much lower than the one for
super.img.

As part of the change, it'll write group info into target_files when
building with PRODUCT_BUILD_SUPER_PARTITION == false. It's the work for
target_files merging script to determine the values to be picked up. The
current logic in merge_target_files.py always uses the one from vendor
target_files. This CL adds a testcase to ensure the behavior.

[1] https://android-review.googlesource.com/c/platform/build/+/928756

Bug: 135752763
Test: `m dist` with a target that sets PRODUCT_BUILD_SUPER_PARTITION to
      false. Check the built artifacts contain super_empty.img. Verify
      that the build can be flashed properly.
Change-Id: I277f087eab45663a6c3b33333d16e9e576c1c25c
2019-06-28 14:23:53 -07:00