Commit Graph

6570 Commits

Author SHA1 Message Date
Tao Bao
28dd0c83a2 Merge "releasetools: Drop the support for creating image archive from dir." 2019-08-06 18:45:17 +00:00
Tao Bao
3810585592 Merge "releasetools: Don't write outputs if not in verbose mode."
am: a514c4ec22

Change-Id: I3cbc864609975cd38c90d68ef2c24caeede8ef4e
2019-08-06 11:08:15 -07:00
Tao Bao
93d52312d3 Merge "Build and use ota_from_target_files as a module."
am: b7957f112b

Change-Id: I2c2d6d17f0b52c83a591ebe28b31076741b3e967
2019-08-06 11:00:19 -07:00
Tao Bao
a514c4ec22 Merge "releasetools: Don't write outputs if not in verbose mode." 2019-08-06 17:33:22 +00:00
Tao Bao
005305adf8 releasetools: Build build_super_image as a lib.
The lib will be shared by more users (e.g. add_img_to_target_files) in
follow-up changes.

Test: TreeHugger
Test: atest releasetools_test releasetools_py3_test
Change-Id: Iaa48bfc8c1dada41362d974773c124a001471ef8
2019-08-05 13:02:26 -07:00
Tao Bao
8821d64481 Build and use ota_from_target_files as a module.
Bug: 63866463
Test: TreeHugger
Test: m -j otapackage
Test: `m -j otatools-package`. Check bin/ota_from_target_files is
      available. Build an OTA package with standalone otatools.
Change-Id: I533dee1c6997d233601e7f4eae637ff5743ed637
2019-08-05 12:57:41 -07:00
Daniel Norman
06975fa319 Merge "Adds support for optionally generating vbmeta.img in merge_builds."
am: 477d2894e4

Change-Id: I1af7d7ff9f4d55373868702d992e3db3ea1bfcb4
2019-08-02 16:46:35 -07:00
Tao Bao
2aac9c9d6d releasetools: Drop the support for creating image archive from dir.
Taking an extracted target_files dir input is only needed for
add_img_to_target_files.py, in particular for the call from build
system. For the case of img_from_target_files.py, it goes much faster
with zip2zip (3m vs 7s in my local test). This CL drops the unneeded
code path in preparation for the switch to zip2zip.

Test: `python build/make/tools/releasetools/img_from_target_files.py \
         -v aosp_taimen-target_files.zip aosp_taimen-img-test.zip`
Change-Id: I7f85d7d629d6fd1621a082f94a11cac7c3e6bbf2
2019-08-02 16:02:32 -07:00
Tao Bao
718faed3dc releasetools: Don't write outputs if not in verbose mode.
The change in [1] allows brillo_update_payload to write to stdout/stderr
directly without buffering, to ensure dumping progress to screen
continously. However, it now unconditionally writes to stdout/stderr
even if caller doesn't specify `-v`. The behavior upsets
PythonUnitTestResultParser in [2], which doesn't want any unexpected
output from stderr for result parsing.

This CL updates the logic to only give continous outputs in verbose
mode.

[1] commit 2f7e11ef27
[2] https://android.googlesource.com/platform/tools/tradefederation/+/refs/heads/master/test_framework/com/android/tradefed/testtype/PythonUnitTestResultParser.java

Test: Run ota_from_target_files.py with `-v`. Check there's continous
      output from brillo_update_payload.
Test: `python -m unittest -v test_ota_from_target_files > /dev/null`
      doesn't show outputs from brillo_update_payload.
Change-Id: I3b851203ef011a5b4d982948cf52793f6fb6ad2d
2019-08-02 13:36:49 -07:00
Daniel Norman
276f06275b Adds support for optionally generating vbmeta.img in merge_builds.
Bug: 137853921
Bug: 138671115
Test: python -m unittest test_common
Test: python -m unittest test_add_img_to_target_files
Test: Ran 'merge_builds --build_vbmeta' for two devices, one with the
vbmeta struct on system.img and another with vbmeta_system.img. Flashed
the regenerated vbmeta.img files on devices, devices boot.

Change-Id: I8d7585c7af468be3d242d8aceeed6d27e6fc6d96
2019-08-02 20:13:03 +00:00
Tao Bao
74dcbad8f2 Merge "releasetools: Add FORCE_RUN_RELEASETOOLS."
am: 1bc828c97e

Change-Id: I08cd4632611b8053c0457846b0a170488640fb80
2019-08-01 21:33:22 -07:00
Tao Bao
1bc828c97e Merge "releasetools: Add FORCE_RUN_RELEASETOOLS." 2019-08-02 03:57:39 +00:00
Dan Willemsen
58be946140 Merge changes from topic "rm_ONE_SHOT_MAKEFILE"
am: 8b8b9b6bf8

Change-Id: I5764b2ce848710584d2d837e9e33fb9fb71c8d33
2019-08-01 14:31:32 -07:00
Dan Willemsen
8b8b9b6bf8 Merge changes from topic "rm_ONE_SHOT_MAKEFILE"
* changes:
  Remove [jni_]link_type files
  Remove support for ONE_SHOT_MAKEFILE
2019-08-01 21:11:02 +00:00
Tao Bao
7d223c6ddd releasetools: Add FORCE_RUN_RELEASETOOLS.
This allows test runner to take care of the external tools setup, then
to start the test without a lunch'd environment. This is needed before
having supports like python_test_helper that packs dependents into
releasetools_test module.

Bug: 138791766
Test: Don't lunch any target. Use `FORCE_RUN_RELEASETOOLS=1 python -m
      unittest -v test_add_img_to_target_files` and check there's no
      skipped testcases.
Change-Id: Iafeaba54fc228d8e1fc5ff3c91f7ffebef18ad7f
2019-08-01 12:22:54 -07:00
Mitchell Wills
e2f0f13049 Merge "Add build script to create a self extracting archive with a click through license"
am: 9a1d60f5cb

Change-Id: I412c669ed7291288911f258232519c20fa136f8c
2019-07-31 19:43:27 -07:00
Treehugger Robot
9a1d60f5cb Merge "Add build script to create a self extracting archive with a click through license" 2019-08-01 02:17:33 +00:00
Dan Willemsen
0df0f6bbf1 Remove [jni_]link_type files
Now that mm/ONE_SHOT_MAKEFILE have been removed, we can expect to know
about all of our dependencies at the end of the build.

This removes 19k nodes from our build graph (aosp-master
aosp_arm64-eng), though in a default build, only 3k of those are used.

Test: ALLOW_MISSING_DEPENDENCIES=true, then trigger a missing dependency
Test: treehugger
Test: create link_type files, then apply CleanSpec.mk, ensure they're removed
Change-Id: I9506331e4a9911d2f26e59a2f72a97aef1644073
2019-07-31 18:09:22 -07:00
Dan Willemsen
93b7c1b547 Merge "Stop supporting wrapping the build with make"
am: d69e849c9b

Change-Id: If6e369f82348d9d7aebe37104072a6878f7281ea
2019-07-30 15:44:36 -07:00
Dan Willemsen
d69e849c9b Merge "Stop supporting wrapping the build with make" 2019-07-30 21:48:52 +00:00
Dan Willemsen
f02f1f16f0 Merge "Remove the PRODUCT-* goal"
am: a2d19cfd05

Change-Id: I8c83c177104117a88fa9822c48ec7fe45b69365c
2019-07-30 10:34:33 -07:00
Dan Willemsen
a2d19cfd05 Merge "Remove the PRODUCT-* goal" 2019-07-30 17:04:07 +00:00
Dan Willemsen
893948955d Stop supporting wrapping the build with make
The build servers are no longer using this. It's hard to search for
others using it -- I've cleaned them up as I've run across them, but
we'll just have to see who breaks once this goes in.

This was the last user of makeparallel, so we can remove it.

Test: make
Test: treehugger
Change-Id: If6df3f1a67d6a6df36afaa4b07cd88a48a364fe1
2019-07-29 22:43:35 -07:00
Daniel Norman
3f9554b59a Merge "Adds new merge builds script for use in merging two non-dist builds."
am: a85f0ee8a2

Change-Id: Ib7b66d5ee82df1d9f31d521917490ac1bac4a4e2
2019-07-29 17:36:57 -07:00
Dan Willemsen
8a5d597bbf Remove the PRODUCT-* goal
I've switched all the build server configs to using TARGET_PRODUCT /
TARGET_BUILD_VARIANT and explicit goals instead.

Remove tools/check_builds.sh which relied on this, but hasn't been
touched in a long time.

Test: m PRODUCT-test
Test: treehugger
Change-Id: If5f8c714567b33aeb38223c176ca24ea649eb57d
2019-07-29 14:27:04 -07:00
Mitchell Wills
1c790ca655 Add build script to create a self extracting archive with a click through license
Bug: 130257221
Test: generated an archive using the script and ran the result
Change-Id: If1147cd41fa939b0d9958196e627042b5731dc14
2019-07-29 11:04:57 -07:00
Daniel Norman
bfc51efa97 Adds new merge builds script for use in merging two non-dist builds.
Bug: 137853921
Test: python -m unittest test_common
Test: python -m unittest test_merge_target_files
Test: Built two partial builds without dist. Ran out/host/linux-x86/bin/merge_builds.
Flashed using `fastboot flashall`. Device boots.
Change-Id: Iffd0a447cdf19a7775a813b4b896178aa6f861f3
2019-07-29 16:30:56 +00:00
Tao Bao
ac280723c2 Merge "releasetools: Move recovery-two-step.img to OTA/."
am: 98efd8290e

Change-Id: I21419dc4a13271c589b8bcb4a4be7da02ef1587d
2019-07-28 11:07:07 -07:00
Tao Bao
0480850f0b releasetools: Move recovery-two-step.img to OTA/.
It used to be packed at IMAGES/recovery-two-step.img, but to serve OTA
purpose only.

Test: `m dist` with a non-A/B target. Check the file in the generated
      target_files.zip.
Test: Create two-step package. Check that recovery-two-step.img is used.
Change-Id: Iec6a73c682e0f844cd8c0b758c9470fa35dd15d8
2019-07-26 13:59:11 -07:00
Tao Bao
706eb3c566 Merge "releasetools: Fix the use of StringIO."
am: 6b466c8f56

Change-Id: I5669a070c0d98239263557c005305002739d1c1d
2019-07-25 09:55:09 -07:00
Tao Bao
bb73388acf releasetools: Fix the use of StringIO.
Based on the actual semantics, it actually wants an in-memory _bytes_
buffer (io.BytesIO), especially when running with Python 3. This CL
fixes the issue and adds a unittest.

Bug: 131631303
Test: python -m unittest test_sign_target_files_apks
Test: python3 -m unittest test_sign_target_files_apks
Change-Id: I3fb067acc26713f1842e831225607779fd0d1b7e
2019-07-24 23:34:25 -07:00
Tom Cherry
6cf1b11ead Generate /etc/{passwd,group} for all partitions
Bug: 73062966
Test: can load AIDs from partitions other than vendor
Test: bionic-unit-tests
Merged-In: Ia85abbeefe5a945369970f2aef42692e07ab8c09
Change-Id: Ia85abbeefe5a945369970f2aef42692e07ab8c09
(cherry picked from commit 8a54ec8482)
2019-07-23 17:07:11 +00:00
Tom Cherry
6722de1277 Merge "Generate /etc/{passwd,group} for all partitions" 2019-07-23 16:40:01 +00:00
Tom Cherry
fb303a5903 Generate /etc/{passwd,group} for all partitions
Bug: 73062966
Test: can load AIDs from partitions other than vendor
Test: bionic-unit-tests
Merged-In: Ia85abbeefe5a945369970f2aef42692e07ab8c09
Change-Id: Ia85abbeefe5a945369970f2aef42692e07ab8c09
2019-07-22 21:41:17 +00:00
changho.shin
0f12536e6b Switch to aapt2 for getting minSdkVersion
aapt does not support parsing some referencing attributes.
Switch to latest tool, aapt2, which works better.

Test: aapt2 dump badging test.apk and returned 0.
Test: Run check_target_files_signatures.py with a target_files.zip.
Test: Run sign_target_files_apks.py with a target_files.zip.
Change-Id: Ib3a4740506f29ebab6930767f2aa8a0b5c4ba053
2019-07-19 20:26:42 +00:00
Daniel Norman
36f8163589 Merge "Clean up merge_target_files.py."
am: ff165e31fd

Change-Id: I4bb6e084b0beb4a57df38a9e083b7e4900320a63
2019-07-19 07:41:27 -07:00
Tao Bao
e8aa7dcf19 Merge "Build build_image and build_super_image as modules."
am: 25cf50f582

Change-Id: Idb34ce99b01fe432e0fe2f7824a87cc8fa3541bf
2019-07-19 07:40:39 -07:00
Elliott Hughes
b5c68a02ae Merge "Remove perfprofd reference."
am: a627e9056b

Change-Id: I9f59616bb617d1c590eb6c923a5d8453ab4c2577
2019-07-19 07:38:56 -07:00
Daniel Norman
ff165e31fd Merge "Clean up merge_target_files.py." 2019-07-19 03:15:29 +00:00
Daniel Norman
4cc9df660b Clean up merge_target_files.py.
- Removes functions that can be replaced with one-line external methods
- Moves read_config_list to common alongside similar methods LoadDictionaryFrom*
- Runs pyformat on merge_target_files.py

Bug: 137853921
Test: python -m unittest test_merge_target_files
Test: Using merge_target_files.py to create a merged build, & booting.
Change-Id: I833c1086db41e1374057cc7447fc50d1915734a7
2019-07-18 13:04:54 -07:00
Tao Bao
2bbb07c53c Build build_image and build_super_image as modules.
Bug: 63866463
Test: TreeHugger
Test: `atest releasetools_test`
Test: `atest releasetools_py3_test`
Change-Id: I2059a4ced709d1b2ee331a9aaaa5ca30db4ebf6b
2019-07-18 10:27:38 -07:00
Treehugger Robot
a627e9056b Merge "Remove perfprofd reference." 2019-07-18 17:22:43 +00:00
Tao Bao
b9c7ed6314 Merge "logging: set stdout and stderr to None in some cases"
am: e733ed7e6b

Change-Id: Ifa43d7a811b2c288de1a2ee3a10fee0efaa43661
2019-07-17 08:55:25 -07:00
Tao Bao
e733ed7e6b Merge "logging: set stdout and stderr to None in some cases" 2019-07-17 15:40:59 +00:00
Elliott Hughes
36d86c1670 Remove perfprofd reference.
Dead and gone.

Test: treehugger
Change-Id: I8575fd9efaa371d61fac5204ee4d982e985a9a56
2019-07-17 08:16:29 -07:00
Tianjie Xu
bcc3554b74 Merge "Factor out the image classes to break circular dependency"
am: dcb6644719

Change-Id: I16371818e6e022ee75187b9c9e104836a72dd983
2019-07-16 13:30:24 -07:00
Tianjie Xu
41976c725c Factor out the image classes to break circular dependency
This helps to break the circular dependency between common and
blockimgdiff.

Bug: 32379627
Test: unit tests pass
Change-Id: I90b5ff34782acbfac86f36265bd96c207d898bf6
2019-07-15 17:02:23 -07:00
Andrew Chant
1db284ff10 Merge "Catch unicode decode errors search for kernel ver"
am: 2b4828a564

Change-Id: Ieed4020a6490d44c4be1780eec08e501ed1e2219
2019-07-15 11:58:41 -07:00
Treehugger Robot
2b4828a564 Merge "Catch unicode decode errors search for kernel ver" 2019-07-15 18:33:57 +00:00
Elliott Hughes
7a45e0ccb9 Merge "Stop building unnecessary tarballs."
am: 9a4b84de56

Change-Id: I9465ce123bc1d5ab9662069af6181dd7382eff57
2019-07-12 08:10:05 -07:00