Commit Graph

5575 Commits

Author SHA1 Message Date
Tao Bao
ad5440d20e Merge "releasetools: Add NonAbOtaPropertyFiles for non-A/B OTA packages." am: 93a84dd9a6
am: f13591bbb3

Change-Id: I43114d7b8ce5900e706e9f234c0fe67cff7cf268
2018-03-19 20:05:28 +00:00
Tao Bao
c0746f4e94 releasetools: Add NonAbOtaPropertyFiles for non-A/B OTA packages.
This CL exposes ota-property-files flag for non-A/B OTA packages.
Currently the line only contains the info for the METADATA entry, for
example "ota-property-files=metadata:69:286". This allows system updater
to just download the METADATA entry, as opposed to downloading the
entire package, to learn about the info regarding the OTA package (e.g.
post-OTA build fingerprint). Note that this requires the OTA server-side
support to pass down the flag along with the update URL.

Bug: 74210298
Test: python -m unittest test_ota_from_target_files
Test: Generate a non-A/B package and check the property-files string.
Change-Id: I1482c587e18ea7101c8328777ea988c2d8ca06ac
2018-03-18 11:59:38 -07:00
Tao Bao
2462a1eb82 releasetools: Add AbOtaPropertyFiles.
am: 85f1698fa0

Change-Id: Ia829f4417659946fcc99b22b66bb392a0a75ae49
2018-03-16 05:08:10 +00:00
Tao Bao
85f1698fa0 releasetools: Add AbOtaPropertyFiles.
AbOtaPropertyFiles writes 'ota-property-files' into package metadata.
Comparing to StreamingPropertyFiles, AbOtaPropertyFiles additionally
writes a token of 'payload-metadata.bin' for a virtual entry that's head
of the 'payload.bin'. This entry can be used by the system updater to
verify if a payload is applicable to the underlying device.

AbOtaPropertyFiles intends to replace StreamingPropertyFiles, as it
covers all the info of the latter. We keep them both in P, and will
remove the latter in later release.

Bug: 74210298
Test: python -m unittest test_ota_from_target_files
Test: Generate full and incremental A/B OTAs. Check the property-files
      in the METADATA entry.
Change-Id: If5872c8ee0a97a8a95b07d4c69b463aa3fe1b9b9
(cherry picked from commit b63046750c)
2018-03-15 16:23:52 -07:00
Tao Bao
f8b2b15a9c Merge "releasetools: Create PropertyFiles class." am: febe7b0736
am: 624f900c01

Change-Id: I58fed3c6e450e3bed7af603d4ad0664618d4d319
2018-03-15 23:05:54 +00:00
Tao Bao
b63046750c releasetools: Add AbOtaPropertyFiles.
AbOtaPropertyFiles writes 'ota-property-files' into package metadata.
Comparing to StreamingPropertyFiles, AbOtaPropertyFiles additionally
writes a token of 'payload-metadata.bin' for a virtual entry that's head
of the 'payload.bin'. This entry can be used by the system updater to
verify if a payload is applicable to the underlying device.

AbOtaPropertyFiles intends to replace StreamingPropertyFiles, as it
covers all the info of the latter. We keep them both in P, and will
remove the latter in later release.

Bug: 74210298
Test: python -m unittest test_ota_from_target_files
Test: Generate full and incremental A/B OTAs. Check the property-files
      in the METADATA entry.
Change-Id: If5872c8ee0a97a8a95b07d4c69b463aa3fe1b9b9
2018-03-15 14:58:31 -07:00
Tao Bao
69203525e4 releasetools: Create PropertyFiles class.
And move StreamingPropertyFiles as its subclass. We will need similar
PropertyFiles instance for non-A/B OTA as well (to expose the
offset/size for the METADATA entry).

Bug: 74210298
Test: python -m unittest test_ota_from_target_files
Test: Generate an A/B OTA. Check the generated property-files string.
Test: pylint --rcfile=pylintrc \
          ota_from_target_files.py \
          test_ota_from_target_files.py
Change-Id: If90d97f0b330749fd8a6cde2ed9d0d6cd6ea60a8
2018-03-15 14:50:55 -07:00
Tao Bao
d3fc38a073 releasetools: Create PropertyFiles class.
And move StreamingPropertyFiles as its subclass. We will need similar
PropertyFiles instance for non-A/B OTA as well (to expose the
offset/size for the METADATA entry).

Bug: 74210298
Test: python -m unittest test_ota_from_target_files
Test: Generate an A/B OTA. Check the generated property-files string.
Test: pylint --rcfile=pylintrc \
          ota_from_target_files.py \
          test_ota_from_target_files.py
Change-Id: If90d97f0b330749fd8a6cde2ed9d0d6cd6ea60a8
(cherry picked from commit 432f374a1701909ca324f8b047666614684568c0)
2018-03-15 14:17:44 -07:00
Tao Bao
69cc0de37b Merge "releasetools: Support validating Verified Boot images." am: 1a6220f598
am: b5376ee331

Change-Id: I3db23dc63900c3e3bc92f1b02033dfa4ffeceefe
2018-03-12 17:49:01 +00:00
Tao Bao
ba557707d8 releasetools: Support validating Verified Boot images.
For a given (signed) target-files.zip, this CLs allows verifying the
Verified Boot related images. It works with both of VB 1.0 and VB 2.0
images.

As part of the CL, it also moves validate_target_files.py to argparse,
which is more flexible than the traditional getopt module.

Also add unittests for the VB 1.0 path. VB 2.0 tests will be added in
follow-up CL.

Example usage:

- Run the script on aosp_bullhead target-files.zip.

$ ./build/make/tools/releasetools/validate_target_files.py \
    --verity_key build/target/product/security/verity.x509.pem \
    --verity_key_mincrypt build/target/product/security/verity_key \
    aosp_bullhead-target_files-4522605.zip

- Run the script on aosp_walleye target-files.zip.

$ ./build/make/tools/releasetools/validate_target_files.py \
    --verity_key external/avb/test/data/testkey_rsa4096.pem \
    aosp_walleye-target_files-4627254.zip

Bug: 63706333
Bug: 65486807
Test: Run validate_target_files.py on target_files.zip files.
Test: PYTHONPATH=build/make/tools/releasetools python -m unittest \
          test_validate_target_files
Change-Id: I170f14d5828d15f3687d8af0a89a816968069057
2018-03-12 00:17:09 -07:00
Tao Bao
cc8e2666ac releasetools: Create StreamingPropertyFiles class.
This CL breaks down ComputeStreamingMetadata() into mutiple member
functions of StreamingPropertyFiles class, which correspond to the
two-pass logic when generating streaming property files (aka streaming
metadata).

StreamingPropertyFiles.Compute() does the work for the first pass, by
putting placeholders before doing initial signing. Finalize()
corresponds to the second pass, where the placeholders get replaced with
actual data. Verify() can be optionally called to assert the correctness
of the work.

The separation between Compute() and Finalize() is to allow having
multiple StreamingPropertyFiles instances (in coming up CLs). This way
we can call Compute() multiple times for each instance, followed by only
one call to SignOutput(). And similarly for Finalize().

Bug: 74210298
Test: Generate an A/B OTA package. Check the METADATA entry.
Test: python -m unittest test_ota_from_target_files
Change-Id: I45be0372a4863c4405e6d8e20bcb9ccdc29e7e11
(cherry picked from commit ae5e4c30fe)
2018-03-09 13:44:13 -08:00
Tao Bao
a49b0446d7 Merge "releasetools: Create StreamingPropertyFiles class." am: 32dfa4914d
am: 934e04d1a3

Change-Id: I2a1c65d09ffc1f4231cec45874557c3e3af74444
2018-03-09 20:13:30 +00:00
Tao Bao
ae5e4c30fe releasetools: Create StreamingPropertyFiles class.
This CL breaks down ComputeStreamingMetadata() into mutiple member
functions of StreamingPropertyFiles class, which correspond to the
two-pass logic when generating streaming property files (aka streaming
metadata).

StreamingPropertyFiles.Compute() does the work for the first pass, by
putting placeholders before doing initial signing. Finalize()
corresponds to the second pass, where the placeholders get replaced with
actual data. Verify() can be optionally called to assert the correctness
of the work.

The separation between Compute() and Finalize() is to allow having
multiple StreamingPropertyFiles instances (in coming up CLs). This way
we can call Compute() multiple times for each instance, followed by only
one call to SignOutput(). And similarly for Finalize().

Bug: 74210298
Test: Generate an A/B OTA package. Check the METADATA entry.
Test: python -m unittest test_ota_from_target_files
Change-Id: I45be0372a4863c4405e6d8e20bcb9ccdc29e7e11
2018-03-08 14:59:42 -08:00
Tao Bao
f6f13acf73 releasetools: Fix the broken validate_target_files.py.
The file was broken due to earlier touches:
 - Missing 'import zipfile';
 - Mismatching arguments when calling GetSparseImage().

Bug: 73996151
Test: Run validate_target_files.py with a walleye-target_files.zip.
Test: pylint --rcfile=pylintrc validate_target_files.py
Change-Id: I3692bd51fb27a3da698e06b75155e84502549f66
(cherry picked from commit c63626b4a3)
2018-03-08 08:10:28 -08:00
Baligh Uddin
8c7a2bb360 Merge "releasetools: Fix the broken validate_target_files.py." am: 1617db36af
am: 946752cb24

Change-Id: Id1f96e9c03a2154f707598ab0e9d13df1fe77136
2018-03-08 16:07:48 +00:00
Tao Bao
c63626b4a3 releasetools: Fix the broken validate_target_files.py.
The file was broken due to earlier touches:
 - Missing 'import zipfile';
 - Mismatching arguments when calling GetSparseImage().

Bug: 73996151
Test: Run validate_target_files.py with a walleye-target_files.zip.
Test: pylint --rcfile=pylintrc validate_target_files.py
Change-Id: I3692bd51fb27a3da698e06b75155e84502549f66
2018-03-07 21:45:40 -08:00
Tao Bao
b9ee298fc6 Merge "releasetools: Separate streaming metadata computation into functions." am: 737bc2b082
am: bfecf9b4cb

Change-Id: I47380d2479460fe9a18ff0b087a3a0a2523dc4c2
2018-03-07 04:40:32 +00:00
Tao Bao
fe5b69a4ae releasetools: Separate streaming metadata computation into functions.
And add unittests for ComputeStreamingMetadata().

This prepares for the changes that add additional property-files (for
both of A/B and non-A/B).

Bug: 74210298
Bug: 72751683
Test: python -m unittest test_ota_from_target_files
Test: Generate A/B OTA package. Check the ota-streaming-property-files
      in the METADATA entry.
Change-Id: Ib4b069f61c2c06c035c0cff73a55112f3936b969
(cherry picked from commit f5110498c0)
2018-03-06 20:26:38 -08:00
Tao Bao
f5110498c0 releasetools: Separate streaming metadata computation into functions.
And add unittests for ComputeStreamingMetadata().

This prepares for the changes that add additional property-files (for
both of A/B and non-A/B).

Bug: 74210298
Bug: 72751683
Test: python -m unittest test_ota_from_target_files
Test: Generate A/B OTA package. Check the ota-streaming-property-files
      in the METADATA entry.
Change-Id: Ib4b069f61c2c06c035c0cff73a55112f3936b969
2018-03-06 00:06:23 -08:00
Tao Bao
8e9155f236 Merge "releasetools: Fix a mismatching close." am: 395bf274b8 am: 06bc45b0c2
am: 6b93ed0c84

Change-Id: I74f9e797510bcecdb1f93824ce5881f92ff3b152
2018-03-03 00:10:31 +00:00
Tao Bao
6b93ed0c84 Merge "releasetools: Fix a mismatching close." am: 395bf274b8
am: 06bc45b0c2

Change-Id: I5e9a7d4a74e476b1ed86a9f7bc06e515998f81e2
2018-03-02 23:59:22 +00:00
Tao Bao
395bf274b8 Merge "releasetools: Fix a mismatching close." 2018-03-02 23:32:11 +00:00
Tao Bao
921c81f7c3 Merge "releasetools: Call Cleanup() in check_target_files_signatures.py." am: 903be914bb am: 5410d8a52a
am: 501b84236f

Change-Id: Ia4dfee212e3ff55fcdd9ce60faddab91c5e195c6
2018-03-02 23:05:20 +00:00
Tao Bao
501b84236f Merge "releasetools: Call Cleanup() in check_target_files_signatures.py." am: 903be914bb
am: 5410d8a52a

Change-Id: I3d015a9ffa1784c060d6a1d878feae9046090b5d
2018-03-02 22:55:11 +00:00
Tao Bao
a652c00c8e releasetools: Fix a mismatching close.
There's a mismatch in WriteABOTAPackageWithBrilloScript().

  temp_zip_file = tempfile.NamedTemporaryFile()
  ...
  common.ZipClose(temp_zip_file)

It's benign since common.ZipClose() happens to be calling
"temp_zip_file.close()". This CL moves the use of tempfile to
common.MakeTempFile(), so that the tempfile will be cleaned up
automatically as part of the call to common.Cleanup(). (Not fixing the
close() directly, since the nearby lines will be refactored into another
function shortly.)

Also remove one assert in the same function, which trivially holds in
the current code.

Test: Generate an A/B OTA.
Change-Id: I53b375d1150820de718dec0ead55abf5f4951071
2018-03-02 13:20:43 -08:00
Tao Bao
150708b012 Merge "releasetools: Don't return ZipFile from common.UnzipTemp()." am: 5c93279fbe am: 0db979c6be
am: abe02acc48

Change-Id: Ie8d1d55d134a7aade1be643634641ef0eb80ff65
2018-03-01 22:44:21 +00:00
Tao Bao
abe02acc48 Merge "releasetools: Don't return ZipFile from common.UnzipTemp()." am: 5c93279fbe
am: 0db979c6be

Change-Id: I9a4f2bc04851f048e168a11c039d781c2069b662
2018-03-01 22:38:46 +00:00
Tao Bao
767543a659 releasetools: Call Cleanup() in check_target_files_signatures.py.
Also minor cleanups to make it pylint clean.

Test: Run check_target_files_signatures.py with a target-files.zip.
Test: pylint --rcfile=pylintrc check_target_files_signatures.py
Change-Id: Ife3b54c7805c2f2562e87e91ab4b4de355782012
2018-03-01 10:12:39 -08:00
Tao Bao
dba59eeabd releasetools: Don't return ZipFile from common.UnzipTemp().
In addition to the unzipping work, common.UnzipTemp() kindly bundles an
open ZipFile object as part of the return value. It doesn't look very
helpful to the callers though. It also looks less obvious that the
caller needs to properly close the handle (missing the close here is
benign though). This CL just removes the ZipFile object out of the
return value, and leaves the work to callers.

Test: `m dist` on both of A/B and non-A/B target.
Test: python -m unittest test_add_img_to_target_files
Test: python -m unittest test_common
Test: python -m unittest test_ota_from_target_files
Test: Check the callers to common.UnzipTemp() in code search.
Change-Id: Id47da3fd42a0e76d6ae8851f05780db319ee48cf
2018-03-01 10:03:31 -08:00
Tao Bao
7a08313a5e Merge "releasetools: Move one case of mkdtemp() to common.MakeTempDir()." am: a7a5e15f72 am: fc0bb788ed
am: e9f3ee0f80

Change-Id: Ie5cf8533ae02aab3f52c3652b2afc55c729e16f2
2018-02-28 21:37:09 +00:00
Tao Bao
e9f3ee0f80 Merge "releasetools: Move one case of mkdtemp() to common.MakeTempDir()." am: a7a5e15f72
am: fc0bb788ed

Change-Id: Icb0ffb498e068417bdadabd4877731c63b776f37
2018-02-28 21:30:30 +00:00
Tao Bao
04bce3a696 releasetools: Move one case of mkdtemp() to common.MakeTempDir().
This should be the last case to be moved over.

Test: Generate an incremental BBOTA (which exercises the changed code).
Test: `rgrep mkdtemp` gives no more instance.
Change-Id: I76db069476201cdfaf3a2de9d9635dfe54507f7a
2018-02-28 11:13:29 -08:00
Tao Bao
9a90d8cae6 Merge "releasetools: Remove the unneeded 'prefix' parameters." am: 6e2a8ae094 am: 945e9c3e44
am: 5d74ac8574

Change-Id: I84554693ebd6be87eb0989d9b2d1df5f049c62f7
2018-02-27 22:36:18 +00:00
Tao Bao
5d74ac8574 Merge "releasetools: Remove the unneeded 'prefix' parameters." am: 6e2a8ae094
am: 945e9c3e44

Change-Id: Iee385b3b5d3a8f3848ccbfdd3773b7cf287f40cf
2018-02-27 22:31:24 +00:00
Tao Bao
6e2a8ae094 Merge "releasetools: Remove the unneeded 'prefix' parameters." 2018-02-27 22:13:06 +00:00
Tao Bao
886d883d61 releasetools: Remove the unneeded 'prefix' parameters.
... from the following functions in add_img_to_target_files.py.

  AddSystem
  AddSystemOther
  AddVendor
  AddProduct
  AddDtbo
  AddUserdata
  AddVBMeta
  AddPartitionTable
  AddCache

The last user of the parameter in img_from_target_files.py has been
removed in commit 2bb109709a (in O).

Test: pylint --rcfile=pylintrc add_img_to_target_files.py
Test: Check all the callers to the above functions.
Test: m dist
Change-Id: I551d1683def8f8535062fc90f68dafa0f4252822
2018-02-27 11:47:11 -08:00
Luis Hector Chavez
0fae5d0806 Merge "Make mksquashfs generate a fs_config-friendly root entry" am: 3d4fb4e335 am: 6128d7f30d
am: 21025515e4

Change-Id: I710cbaa8379c118e0c6a519f5d737a6395de2646
2018-02-27 16:41:46 +00:00
Luis Hector Chavez
21025515e4 Merge "Make mksquashfs generate a fs_config-friendly root entry" am: 3d4fb4e335
am: 6128d7f30d

Change-Id: I43a62ea7c4d6acb5adadfc488b94e05024499320
2018-02-27 16:36:39 +00:00
Treehugger Robot
3d4fb4e335 Merge "Make mksquashfs generate a fs_config-friendly root entry" 2018-02-27 16:24:21 +00:00
Dan Shi
9801d9b7b2 Merge "Default test to AndroidJUnitTest for auto-generated test config." am: ce1fa25d1b am: d91129eea9
am: 6dbd9e3df9

Change-Id: I84ee22fa20b08fb3db27d5e5d0e03f097a11463f
2018-02-26 23:26:06 +00:00
Dan Shi
6dbd9e3df9 Merge "Default test to AndroidJUnitTest for auto-generated test config." am: ce1fa25d1b
am: d91129eea9

Change-Id: Ib0e817e8d403ea130516dbc94201fd4a899c327f
2018-02-26 23:18:57 +00:00
Treehugger Robot
ce1fa25d1b Merge "Default test to AndroidJUnitTest for auto-generated test config." 2018-02-26 23:00:28 +00:00
Tao Bao
47db251db0 Merge "releasetools: Remove the parsing of META/imagesizes.txt." am: 3aee4dfac1 am: 72f3c44243
am: ed98b94326

Change-Id: Id4f84a66faf7c06b4bbacae799d36db4a498182b
2018-02-26 22:46:52 +00:00
Tao Bao
ed98b94326 Merge "releasetools: Remove the parsing of META/imagesizes.txt." am: 3aee4dfac1
am: 72f3c44243

Change-Id: I83af79b6d5a76099259a951585d92755b132af9e
2018-02-26 22:39:14 +00:00
Tao Bao
0835594a6f Merge "releasetools: Add unittests for GetCareMap() and AddCareMapTxtForAbOta()." am: 7998503ebe am: 5fed709d60
am: b8d0b5bd06

Change-Id: Iec88f7bdbf26047479d217d756d4847b1bac64f7
2018-02-26 22:32:18 +00:00
Tao Bao
b8d0b5bd06 Merge "releasetools: Add unittests for GetCareMap() and AddCareMapTxtForAbOta()." am: 7998503ebe
am: 5fed709d60

Change-Id: I117d452b1a4d63b7665c3593b5b2167fe57be954
2018-02-26 22:24:13 +00:00
Treehugger Robot
3aee4dfac1 Merge "releasetools: Remove the parsing of META/imagesizes.txt." 2018-02-26 22:18:31 +00:00
Tao Bao
7998503ebe Merge "releasetools: Add unittests for GetCareMap() and AddCareMapTxtForAbOta()." 2018-02-26 22:06:32 +00:00
Tao Bao
41fb7d27cf releasetools: Remove the parsing of META/imagesizes.txt.
The file has been removed from target-files.zip since commit
c19a8d5590 (Gingerbread), whose info has
been consolidated into META/misc_info.txt.

Test: `m dist`
Change-Id: Ic144457954f5742ea082dcd9ffbea71df4afe46e
2018-02-23 12:06:07 -08:00
Tao Bao
cf9c445ef9 Merge "releasetools: Allow skipping postinstall hooks when generating A/B OTAs." am: 6c0ddf2480 am: ff1288821d
am: a5baf56e6c

Change-Id: I59bf83ba415351ba07488086d8b812453b0ae61b
2018-02-22 20:49:01 +00:00