Commit Graph

6708 Commits

Author SHA1 Message Date
Andrew Chant
5d323c1714 Catch unicode decode errors search for kernel ver
If a kernel happens to decode to gibberish (including
non-ascii bytes), catch the decoding error when
searching for the kernel version.

Bug: 137041171
Merged-In: Ic035b3a5c8c80025cb3cede7b0fdcf8a2e5a35fd
Change-Id: Ic035b3a5c8c80025cb3cede7b0fdcf8a2e5a35fd
2019-07-09 17:02:48 +00:00
Justin Yun
8f11ad5d40 Merge "Rename product_services to system_ext"
am: b7a50ece30

Change-Id: I7a56d76d3543b2bc4b1d0b24e54e8dd82ab7f15a
2019-07-09 02:14:55 -07:00
Justin Yun
6151e3f1ea Rename product_services to system_ext
Bug: 134359158
Test: build and check if system_ext.img is created
Change-Id: I67f2e95dd29eac6a28e07e24ea973d3a134c3bfc
2019-07-09 08:57:19 +00:00
Tao Bao
ca82fc90f7 Additionally run releasetools_test with Python 3.
Bug: 131631303
Test: TreeHugger
Test: `atest releasetools_test`
Test: `atest releasetools_py3_test`
Change-Id: Ie00f48835aae41bc69ab8554b856a62aa3b497cc
2019-07-09 05:14:31 +00:00
Tao Bao
ec2f3da2a2 Merge "releasetools: Make merge_target_files and its test Python 3 compatible."
am: 6e3ae4e849

Change-Id: I0ee3f8801e194f140f32a4123100aa0e4f449a70
2019-07-08 21:19:53 -07:00
Tao Bao
2ad4b82d45 releasetools: Make merge_target_files and its test Python 3 compatible.
And a few minor clean-ups to the styling.

Bug: 131631303
Test: python -m unittest test_merge_target_files
Test: python3 -m unittest test_merge_target_files
Test: Use `python merge_target_files` to merge two target_files zips.
Test: Use `python3 merge_target_files` to merge two target_files zips.
Change-Id: I8502dfb243408f658d022e8d5e5fbb60066e4ff0
2019-07-08 16:06:15 -07:00
Kiyoung Kim
dc6566f3b2 Merge "Split merge_target_files function"
am: 09736b4aaf

Change-Id: Ic077a4bf1276ead627c1d09f67636626623a45ab
2019-07-07 23:51:33 -07:00
Kiyoung Kim
7cbeda728d Split merge_target_files function
Split merge_target_files function into several steps, so we can increate
readability and add conditional flows for other *SSI mixed build with less effort

Test: m -j & atest passed
Change-Id: I558f9dd5bca31b132a09cb36d9dfcd30c92efbc9
2019-07-08 12:13:05 +09:00
TreeHugger Robot
218adad125 Merge "DO NOT MERGE - Merge qt-dev-plus-aosp-without-vendor (5699924) into stage-aosp-master" into stage-aosp-master 2019-07-02 07:40:58 +00:00
Tao Bao
48a2feb670 Change the condition for building super_empty.img.
This CL changes the condition for building super_empty.img from
PRODUCT_BUILD_SUPER_PARTITION to PRODUCT_USE_DYNAMIC_PARTITIONS, as a
follow-up to the change in [1].

With the CL in [1], it skips building super.img and super_empty.img both
when turning off PRODUCT_BUILD_SUPER_PARTITION. However, the latter
should be mandatory whenever dynamic partitions is enabled. Because
fastboot relies on this file to properly flash dynamic partitions. Plus,
the cost for building super_empty.img is much lower than the one for
super.img.

As part of the change, it'll write group info into target_files when
building with PRODUCT_BUILD_SUPER_PARTITION == false. It's the work for
target_files merging script to determine the values to be picked up. The
current logic in merge_target_files.py always uses the one from vendor
target_files. This CL adds a testcase to ensure the behavior.

[1] https://android-review.googlesource.com/c/platform/build/+/928756

Bug: 135752763
Test: `m dist` with a target that sets PRODUCT_BUILD_SUPER_PARTITION to
      false. Check the built artifacts contain super_empty.img. Verify
      that the build can be flashed properly.
Change-Id: I277f087eab45663a6c3b33333d16e9e576c1c25c
2019-06-28 14:23:53 -07:00
Tao Bao
c1c52e7b3b Merge "releasetools: Prefer the avbtool specified in target_files." am: 63cf1326da
am: d84a0531e9

Change-Id: Ieb93952f093a2d6e3d3c9796d61819c46cc4fcea
2019-06-27 19:04:51 -07:00
Tao Bao
63cf1326da Merge "releasetools: Prefer the avbtool specified in target_files." 2019-06-28 01:18:21 +00:00
Tao Bao
a1b63421d9 Merge "releasetools: Make additional modules Python 3 compatible." am: e4f07795e4
am: ed7923be4f

Change-Id: Idf708c33684e7c7b96286b363d15bf947422323d
2019-06-27 17:58:45 -07:00
Tao Bao
e4f07795e4 Merge "releasetools: Make additional modules Python 3 compatible." 2019-06-27 23:52:50 +00:00
Tao Bao
2cfd9b9077 Merge "releasetools: Clean up check_target_files_signatures.py." am: f6e8d46e2f
am: d82b810a77

Change-Id: Ie4cb3dff2fb4098714a25b4cf10643cd1951aae4
2019-06-27 16:08:24 -07:00
Tao Bao
f6e8d46e2f Merge "releasetools: Clean up check_target_files_signatures.py." 2019-06-27 21:36:14 +00:00
Tao Bao
e815cec8f3 Merge "releasetools: Add sanity test for sign_apex." am: 2a917796b5
am: 03dc3694be

Change-Id: I0d2ad60b1df2f660a2a0658eaef000a19538b762
2019-06-27 11:31:55 -07:00
Tao Bao
2a917796b5 Merge "releasetools: Add sanity test for sign_apex." 2019-06-27 17:35:29 +00:00
Tao Bao
a67e12d66f releasetools: Clean up check_target_files_signatures.py.
And make it Python 3 compatibile.

Bug: 131631303
Test: python check_target_files_signatures.py target-files.zip
Test: python3 check_target_files_signatures.py target-files.zip
Change-Id: I38e868c3124387f5543ad68c92c75917e40b7d8f
2019-06-27 10:24:10 -07:00
Tao Bao
a370545a2c releasetools: Make additional modules Python 3 compatible.
Bug: 131631303
Test: `python -m unittest test_sign_target_files_apks`
Test: `python3 -m unittest test_sign_target_files_apks`
Test: `python -m unittest test_add_img_to_target_files`
Test: `python3 -m unittest test_add_img_to_target_files`
Test: `python -m unittest test_ota_from_target_files`
Test: `python3 -m unittest test_ota_from_target_files`
Test: `python -m unittest test_validate_target_files`
Test: `python3 -m unittest test_validate_target_files`
Test: Run `python3 ota_from_target_files.py` to generate an OTA.
Test: Run `python3 sign_target_files_apks.py` to sign a target_files.
Change-Id: I56b45bbcbf7aa83e690785a9640c0212e45d12d8
2019-06-27 09:05:48 -07:00
Kiyoung Kim
c2544994a4 Merge "Create common.LoadDictionaryFromFile" am: fc195fee44
am: 486bc8d590

Change-Id: I9622dbde337937345b581fd11047c339c242aab7
2019-06-26 18:25:37 -07:00
Kiyoung Kim
fc195fee44 Merge "Create common.LoadDictionaryFromFile" 2019-06-27 00:27:49 +00:00
Tao Bao
1ac886e181 releasetools: Prefer the avbtool specified in target_files.
This allows a consistent logic in using the avbtool which could be
board-specific.

Test: `atest releasetools_test`
Test: Run sign_target_files_apks.py on a target_files.zip.
Change-Id: I8cd93b8e71146985734f85c31f4662f5e2e9534c
2019-06-26 17:18:48 -07:00
Tao Bao
c9c1b864ba releasetools: Add sanity test for sign_apex.
This ensures a matching interface between sign_apex and apex_utils.

The test apex `testdata/foo.apex` is generated by running
`system/apex/apexer/runtests.sh`.

Test: python -m unittest test_sign_apex
Test: atest releasetools_test
Change-Id: I7c14b1df2a3038ad206aa3e5aac084c47baaa00b
2019-06-26 17:17:09 -07:00
Tao Bao
c515aa03b8 Merge "releasetools: Clean up additional modules for Python 3 compatibility." am: 90f638d842
am: c82d8ceffc

Change-Id: I84e7866b06ff693240272b39119d0ab5cae7e72a
2019-06-26 12:05:13 -07:00
Tao Bao
59cf0c59de releasetools: Clean up additional modules for Python 3 compatibility.
It also addresses a few linter warnings (mostly regarding indentation).

Bug: 131631303
Test: TreeHugger
Test: `python -m unittest test_ota_from_target_files`
Test: `python3 -m unittest test_ota_from_target_files`
Test: `python build/make/tools/releasetools/check_ota_package_signature.py
           build/make/target/product/security/testkey.x509.pem
           test-keys-signed-ota.zip`
Test: `python3 build/make/tools/releasetools/check_ota_package_signature.py
           build/make/target/product/security/testkey.x509.pem
           test-keys-signed-ota.zip`
Change-Id: I11a745dfec052081ab0282bb028dc27debf71f57
2019-06-26 00:15:53 -07:00
Kiyoung Kim
ebe7c9c92f Create common.LoadDictionaryFromFile
There are some places defining same file open function and use
common.LoadDictionaryFromLines. This commit creates
LoadDictionaryFromFile to reduce some code redundancy.

Test: m -j & atest passed
Change-Id: I6a3fa48693095937f8c79ce6f3c110b6862a1967
2019-06-26 13:09:52 +09:00
Tao Bao
a54ec9bc55 Merge "releasetools: Make apex_utils Python 3 compatible." am: 58abe77060
am: da5b3b5178

Change-Id: I1c44ac55ea7709410f4fa3a91cf0e41d2df018a3
2019-06-25 16:07:39 -07:00
Elliott Hughes
0ebfea6351 Merge "Don't use tar's -p with -rf, where it makes no sense." am: 2fdfc4e439
am: 9b8d9396ea

Change-Id: Ic485440769e7463cb9b400fb4a717c5e0ffb1a45
2019-06-25 15:49:45 -07:00
Tao Bao
58abe77060 Merge "releasetools: Make apex_utils Python 3 compatible." 2019-06-25 22:34:39 +00:00
Elliott Hughes
2fdfc4e439 Merge "Don't use tar's -p with -rf, where it makes no sense." 2019-06-25 21:28:22 +00:00
Chih-hung Hsieh
be06ed28b5 Merge "Fix pylint warnings and now prefer python3" am: 7b3a8fb7ec
am: ebb825c42d

Change-Id: I31a58138b7a1e19ca4819c824bea89592ca50acc
2019-06-25 11:55:14 -07:00
Chih-hung Hsieh
7b3a8fb7ec Merge "Fix pylint warnings and now prefer python3" 2019-06-25 18:32:26 +00:00
Tao Bao
86b529a23e releasetools: Make apex_utils Python 3 compatible.
In order to get a Python 2 and 3 compatible re-raise behavior, this CL
removes the stack traceback for the lines within apex_util module (i.e.
sys.exc_info()[2]). It's not a big loss in practice, since we only have
one line within the try-except block (`common.RunAndCheckOutput()`)
that's no longer reported in the traceback.

Using `six` module could better solve this, but only after building
releasetools as python_binary_host modules where we can properly handle
the module dependency.

Bug: 131631303
Test: TreeHugger
Test: `python -m unittest test_apex_utils`
Test: `python3 -m unittest test_apex_utils`
Change-Id: I0c5a72ec9fad5ff9d8c9c94d29e813e433ec2921
2019-06-25 11:10:55 -07:00
Tao Bao
41e98f654c Merge changes Idc7776be,I409fe30a am: ec8e2b4e02
am: 70b5b96565

Change-Id: I33df35895fc75347dfda3cacdf717f75df7ba14d
2019-06-25 10:09:32 -07:00
Elliott Hughes
b82ef911f9 Don't use tar's -p with -rf, where it makes no sense.
-p only applies to tar invocations that cause extraction, but -rf is
just appending to a tar file.

toybox tar doesn't have -p, and it doesn't seem worth implementing if
this is our only "user".

Test: treehugger
Change-Id: I025c496b04cb57cb31ba81fd516fd40b158bc06b
2019-06-25 09:07:41 -07:00
Tao Bao
9e893c3cfd releasetools: Make verity_utils Python 3 compatible.
Bug: 131631303
Test: TreeHugger
Test: `python -m unittest test_verity_utils`
Test: `python3 -m unittest test_verity_utils`
Change-Id: Idc7776be2cf78fbdb0b3ba87231d05013da8d3ac
2019-06-24 21:07:19 -07:00
Tao Bao
da30cfae96 releasetools: Make common Python 3 compatible.
Bug: 131631303
Test: TreeHugger
Test: `python -m unittest test_common`
Test: `python3 -m unittest test_common`
Change-Id: I409fe30a5d71975c1d7b66e5e749843de530f1f9
2019-06-24 21:02:12 -07:00
Chih-Hung Hsieh
9018ea49d8 Fix pylint warnings and now prefer python3
* Read/write warning messages as utf-8 strings.
* No more embedded raw utf-8 code in file.

Test: ./warn.py --csvpath warnings.csv build.log > warnings.html
Change-Id: Ie51700721a32bede1d3b250f4a42fd8facd6af75
2019-06-24 15:24:39 -07:00
Tao Bao
cebd118790 Merge "releasetools: Remove some legacy paths for prop rewrites." am: eb7cb468e4
am: f589f9bb2d

Change-Id: I637506dfc84a4a4aee8f8ee744b4e3ff6d5d0642
2019-06-24 09:24:59 -07:00
Tao Bao
338c1b7e90 releasetools: Remove some legacy paths for prop rewrites.
sign_target_files_apks.py only needs to take care of the current
release. The legacy paths of ODM/build.prop, VENDOR/odm/build.prop, and
BOOT/RAMDISK/default.prop no longer exist in the target_files.zip from
master.

The other two cases of ROOT/default.prop and
RECOVERY/RAMDISK/default.prop are still kept in the code, as they will
still exist (as symlink or conditionally).

Test: Run sign_target_files_apks.py against
      aosp_taimen-target_files.zip. Check the rewritten prop files.
Test: `python -m unittest test_sign_target_files_apks`
Change-Id: I5e70bc2ccc0f3dcf0eace0718c59a3b0f89a9ff4
2019-06-21 10:26:15 -07:00
Tao Bao
add1f28ba4 Merge "releasetools: Make blockimgdiff Python 3 compatible." am: 4daa755fb2
am: 07a7471ddb

Change-Id: Ia91938e7efed299dc212dfa2c9116634e67dafd1
2019-06-21 08:25:58 -07:00
Tao Bao
8c9cc5fbc1 Merge "releasetools: Make rangelib Python 3 compatibile." am: 1f1aafa30d
am: 207f624f75

Change-Id: Iba18162ab1c05ec780b057f1391b1f9aaf0b4039
2019-06-20 18:17:23 -07:00
Tao Bao
b813120af1 releasetools: Make blockimgdiff Python 3 compatible.
Bug: 131631303
Test: `python -m unittest test_blockimgdiff`
Test: `python3 -m unittest test_blockimgdiff`
Change-Id: I8cf072ca4af6a525bc675cba0be998ff7be7f4f1
2019-06-20 14:30:30 -07:00
Tao Bao
1f1aafa30d Merge "releasetools: Make rangelib Python 3 compatibile." 2019-06-20 14:51:21 +00:00
Tao Bao
0f68744f1e Merge "releasetools: Make common.ZipWriteStr Python 3 compatible." am: 9521c802ec
am: 0812ce4e3d

Change-Id: Ia3009e10b5b9492b6ab50135dce4a77ffc978011
2019-06-19 15:56:45 -07:00
Tao Bao
d660c8d8f3 releasetools: Make rangelib Python 3 compatibile.
Bug: 131631303
Test: `python -m unittest test_rangelib`
Test: `python3 -m unittest test_rangelib`
Change-Id: Ia8e26623d5967f2eea003252ee56b861350d626b
2019-06-19 10:30:34 -07:00
Tao Bao
c1a1ec30df releasetools: Make common.ZipWriteStr Python 3 compatible.
Python 2 and 3 behave differently when calling ZipFile.writestr() with
zinfo.external_attr being 0. Python 3 uses `0o600 << 16` as the value
for such a case (since
18ee29d0b8),
which seems to make more sense. Otherwise the entry will end up with
0o000 as the permission bits. This CL updates common.ZipWriteStr to
follow the logic in Python 3, in order to get consistent behavior
between using the two versions.

Bug: 131631303
Test: `python -m unittest test_common.CommonZipTest`
Test: `python3 -m unittest test_common.CommonZipTest`
Change-Id: If8429855d922ef1ad76591f703215a0ce5089f0f
2019-06-18 23:53:53 -07:00
Tao Bao
ae64f09b06 Merge "releasetools: Update partitions in deterministic order." am: aadc9b9159
am: 269c12a5b9

Change-Id: Ic2bff4b76ccd4a0c5b3d3d71dedee0728fc16019
2019-06-18 16:23:38 -07:00
Tom Cherry
3156881708 Merge "Update fs_config documentation" am: 0405f780fc
am: 1e6a49a22a

Change-Id: Iec8d910916999ec35415bb295e88f3c8e88fd6e5
2019-06-18 16:21:45 -07:00