Commit Graph

6547 Commits

Author SHA1 Message Date
Dan Willemsen
86be07669f Merge "Remove package-stats.txt"
am: 1211130c92

Change-Id: I6be259692a3a25934f30595b7f23177c4f6b62bb
2019-08-21 23:51:21 -07:00
Dan Willemsen
d7994816c1 Remove package-stats.txt
I can't find any remaining users of this functionality.

Test: codesearch; treehugger
Change-Id: I63b079b381211b743baa293267c992d9b2eadcf9
2019-08-21 12:36:14 -07:00
Tianjie Xu
d4400ca15f Merge "Generate block based OTA for product partitions"
am: 2211352eed

Change-Id: Ib36f9dde2982d77fdd2a8a74d2e80d7a8b21326e
2019-08-20 13:39:52 -07:00
Tianjie Xu
f67dd8059d Generate block based OTA for product partitions
Currently only the system and vendor partitions are updated in the
generic block based OTA generation script. Since the product partition
is quite large and consist most of APK files, it would be beneficial to
update it similar to system. Handle the odm and system_ext partitions in
the same way as well.

Bug: 132683080
Test: Run unit tests, generate full and incremental OTA with product partitions.
Change-Id: I13478cf9bd32137c6729b8c9cb102080147093f2
2019-08-20 00:18:00 -07:00
Paul Trautrim
6c3aa71d1b Merge "Use --add_missing in merge_target_files.py"
am: cc875c1482

Change-Id: I0abe618a8e1900c57b67060361b46807531cddd7
2019-08-19 04:05:40 -07:00
Paul Trautrim
cc875c1482 Merge "Use --add_missing in merge_target_files.py" 2019-08-19 10:50:35 +00:00
Tao Bao
f4157c5859 Merge "releasetools: Add more module building rules."
am: 39369d08fb

Change-Id: Ic422023eceae40922a6251f32982cc1141f86aac
2019-08-14 09:08:50 -07:00
Tao Bao
39369d08fb Merge "releasetools: Add more module building rules." 2019-08-14 15:33:33 +00:00
Paul Trautrim
bc3600c030 Use --add_missing in merge_target_files.py
This is needed when including prebuilt images that do not need to be
regenerated. Otherwise the script will exit upon detecting an existing
IMAGES/ directory.

Test: test_merge_target_files
Bug: 139111475
Change-Id: I55d82bd7dc4fa5f27bc01e481362e79fecc200ff
2019-08-13 19:24:12 +09:00
Tao Bao
cd79edb331 Merge "releasetools: Fix the target files diffing."
am: 6a52b34473

Change-Id: I852a255f7ad99718a96bae0df92ce54ceb543c0a
2019-08-08 16:35:04 -07:00
Tao Bao
6a52b34473 Merge "releasetools: Fix the target files diffing." 2019-08-08 22:55:45 +00:00
Tao Bao
3c7b2a658c releasetools: Add more module building rules.
This CL adds the building rules for the following tools:
- add_img_to_target_files
- check_ota_package_signature
- check_target_files_signatures
- img_from_target_files
- merge_target_files
- ota_package_parser
- sign_apex
- sign_target_files_apks
- validate_target_files

Bug: 63866463
Test: mmma -j build/make/tools/releasetools
Test: Run each tool above to ensure there's no missing dependency (in
      the exercised code path).
Change-Id: Ide5b9775939ed02c822109cbcf4224000d5472df
2019-08-07 22:35:42 -07:00
Tao Bao
ced94b337c Merge "releasetools: ota_from_target_files uses target_files_diff."
am: 69340f6885

Change-Id: I822a3bfa774c79d2c4789a84fa084361753cb24b
2019-08-07 21:41:21 -07:00
Tao Bao
69340f6885 Merge "releasetools: ota_from_target_files uses target_files_diff." 2019-08-08 04:07:51 +00:00
Roland Levillain
77507f72fc Merge "Fix typo in error message in tools/fs_config/fs_config_generator.py."
am: c5cb45b7da

Change-Id: I5c1ca3414cc22113864c7ee4ad7d6c2dbc294018
2019-08-07 15:01:23 -07:00
Tao Bao
f0c4aa204c releasetools: Fix the target files diffing.
This CL refactors the non-A/B OTA generation code into a function,
so it allows adding common post-generation code for both flows.

It moves the code for target files diffing to common path, and fixes the
diffing functionality (broken due to the use of unzip pattern).

Test: Run ota_from_target_files with `--log_diff` flag to generate
      non-A/B and A/B OTAs respectively.
Change-Id: I92e5549a58344fc56a151a709c06534913d265d5
2019-08-07 14:53:08 -07:00
Treehugger Robot
c5cb45b7da Merge "Fix typo in error message in tools/fs_config/fs_config_generator.py." 2019-08-07 21:35:33 +00:00
Tao Bao
f882684d85 Merge "Move make_recovery_path and sparse_img to modules."
am: 4c9527b314

Change-Id: I13a67cfe58450696c37a0e7e4ecc3261d8bd7d5a
2019-08-07 13:15:15 -07:00
Tao Bao
496748378d releasetools: ota_from_target_files uses target_files_diff.
The dependency was overlooked as the module was conditionally imported.

Bug: 63866463
Test: `m -j ota_from_target_files`; Run ota_from_target_files with
      `--log_diff` flag (on a non-A/B target).
Change-Id: I71eff2e55cc28fd649f0ca1b38da53907dac6597
2019-08-07 13:01:27 -07:00
Tao Bao
4c9527b314 Merge "Move make_recovery_path and sparse_img to modules." 2019-08-07 18:49:20 +00:00
Roland Levillain
5ca32dfec5 Fix typo in error message in tools/fs_config/fs_config_generator.py.
Test: n/a
Change-Id: I46682483803e1c7c71fb07157d0ef9c990152d93
2019-08-07 15:51:01 +01:00
Tao Bao
e11a46000c Move make_recovery_path and sparse_img to modules.
These two tools are only used within build system, so not adding them
to otatools.zip.

Bug: 63866463
Test: TreeHugger
Test: Build a non-A/B target that calls make_recovery_patch.
Change-Id: Iaeddc98a2c4a27c6b49d91047500029e460db9e3
2019-08-06 23:27:48 -07:00
Tao Bao
7552bbdfb6 Merge "releasetools: merge_target_files sets verbose in main."
am: 3738f37e22

Change-Id: I0e193d444af6e8a05289e22bbc02fc084972750a
2019-08-06 19:47:51 -07:00
Tao Bao
abb806b3f0 releasetools: merge_target_files sets verbose in main.
The flag would otherwise be turned on unconditionally even when
importing merge_target_files as a module. As a result, `atest
releasetools_test` was broken / flaky prior to this change, as
PythonUnitTestResultParser doesn't like unexpected outputs in stderr.

Test: `atest releasetools_test releasetools_py3_test`
Change-Id: Ifd614c797d2cee3b33e4c2a7ad8e598850e9d60c
2019-08-06 14:13:19 -07:00
Tao Bao
e5f0839c4b Merge "releasetools: Build build_super_image as a lib."
am: 7edc2b1994

Change-Id: Ib9fb0144b79d183aacc08ed0f661a0c7356b5372
2019-08-06 12:18:31 -07:00
Tao Bao
7edc2b1994 Merge "releasetools: Build build_super_image as a lib." 2019-08-06 19:01:20 +00:00
Tao Bao
a4a236e318 Merge "releasetools: Drop the support for creating image archive from dir."
am: 28dd0c83a2

Change-Id: I5a7cb6fc33c2ebdc8eb6d44302f038b5c88540a4
2019-08-06 11:57:35 -07:00
Tao Bao
28dd0c83a2 Merge "releasetools: Drop the support for creating image archive from dir." 2019-08-06 18:45:17 +00:00
Tao Bao
3810585592 Merge "releasetools: Don't write outputs if not in verbose mode."
am: a514c4ec22

Change-Id: I3cbc864609975cd38c90d68ef2c24caeede8ef4e
2019-08-06 11:08:15 -07:00
Tao Bao
93d52312d3 Merge "Build and use ota_from_target_files as a module."
am: b7957f112b

Change-Id: I2c2d6d17f0b52c83a591ebe28b31076741b3e967
2019-08-06 11:00:19 -07:00
Tao Bao
a514c4ec22 Merge "releasetools: Don't write outputs if not in verbose mode." 2019-08-06 17:33:22 +00:00
Tao Bao
005305adf8 releasetools: Build build_super_image as a lib.
The lib will be shared by more users (e.g. add_img_to_target_files) in
follow-up changes.

Test: TreeHugger
Test: atest releasetools_test releasetools_py3_test
Change-Id: Iaa48bfc8c1dada41362d974773c124a001471ef8
2019-08-05 13:02:26 -07:00
Tao Bao
8821d64481 Build and use ota_from_target_files as a module.
Bug: 63866463
Test: TreeHugger
Test: m -j otapackage
Test: `m -j otatools-package`. Check bin/ota_from_target_files is
      available. Build an OTA package with standalone otatools.
Change-Id: I533dee1c6997d233601e7f4eae637ff5743ed637
2019-08-05 12:57:41 -07:00
Daniel Norman
06975fa319 Merge "Adds support for optionally generating vbmeta.img in merge_builds."
am: 477d2894e4

Change-Id: I1af7d7ff9f4d55373868702d992e3db3ea1bfcb4
2019-08-02 16:46:35 -07:00
Tao Bao
2aac9c9d6d releasetools: Drop the support for creating image archive from dir.
Taking an extracted target_files dir input is only needed for
add_img_to_target_files.py, in particular for the call from build
system. For the case of img_from_target_files.py, it goes much faster
with zip2zip (3m vs 7s in my local test). This CL drops the unneeded
code path in preparation for the switch to zip2zip.

Test: `python build/make/tools/releasetools/img_from_target_files.py \
         -v aosp_taimen-target_files.zip aosp_taimen-img-test.zip`
Change-Id: I7f85d7d629d6fd1621a082f94a11cac7c3e6bbf2
2019-08-02 16:02:32 -07:00
Tao Bao
718faed3dc releasetools: Don't write outputs if not in verbose mode.
The change in [1] allows brillo_update_payload to write to stdout/stderr
directly without buffering, to ensure dumping progress to screen
continously. However, it now unconditionally writes to stdout/stderr
even if caller doesn't specify `-v`. The behavior upsets
PythonUnitTestResultParser in [2], which doesn't want any unexpected
output from stderr for result parsing.

This CL updates the logic to only give continous outputs in verbose
mode.

[1] commit 2f7e11ef27
[2] https://android.googlesource.com/platform/tools/tradefederation/+/refs/heads/master/test_framework/com/android/tradefed/testtype/PythonUnitTestResultParser.java

Test: Run ota_from_target_files.py with `-v`. Check there's continous
      output from brillo_update_payload.
Test: `python -m unittest -v test_ota_from_target_files > /dev/null`
      doesn't show outputs from brillo_update_payload.
Change-Id: I3b851203ef011a5b4d982948cf52793f6fb6ad2d
2019-08-02 13:36:49 -07:00
Daniel Norman
276f06275b Adds support for optionally generating vbmeta.img in merge_builds.
Bug: 137853921
Bug: 138671115
Test: python -m unittest test_common
Test: python -m unittest test_add_img_to_target_files
Test: Ran 'merge_builds --build_vbmeta' for two devices, one with the
vbmeta struct on system.img and another with vbmeta_system.img. Flashed
the regenerated vbmeta.img files on devices, devices boot.

Change-Id: I8d7585c7af468be3d242d8aceeed6d27e6fc6d96
2019-08-02 20:13:03 +00:00
Tao Bao
74dcbad8f2 Merge "releasetools: Add FORCE_RUN_RELEASETOOLS."
am: 1bc828c97e

Change-Id: I08cd4632611b8053c0457846b0a170488640fb80
2019-08-01 21:33:22 -07:00
Tao Bao
1bc828c97e Merge "releasetools: Add FORCE_RUN_RELEASETOOLS." 2019-08-02 03:57:39 +00:00
Dan Willemsen
58be946140 Merge changes from topic "rm_ONE_SHOT_MAKEFILE"
am: 8b8b9b6bf8

Change-Id: I5764b2ce848710584d2d837e9e33fb9fb71c8d33
2019-08-01 14:31:32 -07:00
Dan Willemsen
8b8b9b6bf8 Merge changes from topic "rm_ONE_SHOT_MAKEFILE"
* changes:
  Remove [jni_]link_type files
  Remove support for ONE_SHOT_MAKEFILE
2019-08-01 21:11:02 +00:00
Tao Bao
7d223c6ddd releasetools: Add FORCE_RUN_RELEASETOOLS.
This allows test runner to take care of the external tools setup, then
to start the test without a lunch'd environment. This is needed before
having supports like python_test_helper that packs dependents into
releasetools_test module.

Bug: 138791766
Test: Don't lunch any target. Use `FORCE_RUN_RELEASETOOLS=1 python -m
      unittest -v test_add_img_to_target_files` and check there's no
      skipped testcases.
Change-Id: Iafeaba54fc228d8e1fc5ff3c91f7ffebef18ad7f
2019-08-01 12:22:54 -07:00
Mitchell Wills
e2f0f13049 Merge "Add build script to create a self extracting archive with a click through license"
am: 9a1d60f5cb

Change-Id: I412c669ed7291288911f258232519c20fa136f8c
2019-07-31 19:43:27 -07:00
Treehugger Robot
9a1d60f5cb Merge "Add build script to create a self extracting archive with a click through license" 2019-08-01 02:17:33 +00:00
Dan Willemsen
0df0f6bbf1 Remove [jni_]link_type files
Now that mm/ONE_SHOT_MAKEFILE have been removed, we can expect to know
about all of our dependencies at the end of the build.

This removes 19k nodes from our build graph (aosp-master
aosp_arm64-eng), though in a default build, only 3k of those are used.

Test: ALLOW_MISSING_DEPENDENCIES=true, then trigger a missing dependency
Test: treehugger
Test: create link_type files, then apply CleanSpec.mk, ensure they're removed
Change-Id: I9506331e4a9911d2f26e59a2f72a97aef1644073
2019-07-31 18:09:22 -07:00
Dan Willemsen
93b7c1b547 Merge "Stop supporting wrapping the build with make"
am: d69e849c9b

Change-Id: If6e369f82348d9d7aebe37104072a6878f7281ea
2019-07-30 15:44:36 -07:00
Dan Willemsen
d69e849c9b Merge "Stop supporting wrapping the build with make" 2019-07-30 21:48:52 +00:00
Dan Willemsen
f02f1f16f0 Merge "Remove the PRODUCT-* goal"
am: a2d19cfd05

Change-Id: I8c83c177104117a88fa9822c48ec7fe45b69365c
2019-07-30 10:34:33 -07:00
Dan Willemsen
a2d19cfd05 Merge "Remove the PRODUCT-* goal" 2019-07-30 17:04:07 +00:00
Dan Willemsen
893948955d Stop supporting wrapping the build with make
The build servers are no longer using this. It's hard to search for
others using it -- I've cleaned them up as I've run across them, but
we'll just have to see who breaks once this goes in.

This was the last user of makeparallel, so we can remove it.

Test: make
Test: treehugger
Change-Id: If6df3f1a67d6a6df36afaa4b07cd88a48a364fe1
2019-07-29 22:43:35 -07:00