Commit Graph

91 Commits

Author SHA1 Message Date
dianlujitao
ee95f242c8 extract_utils: Handle Brotli compressed block image
Change-Id: I1b3ef34ebba9b43fd684c6abaa1dbc77c8786da7
2020-04-25 16:21:32 +02:00
dianlujitao
fc48634419 extract_utils: Support odm/product/vendor partition for zip extract
Change-Id: Iec06835f1703e276f8882c5ee327ef241a2f8f23
2020-04-25 16:21:32 +02:00
dianlujitao
0b501d534a extract_utils: Generate deodexed apk/jar with consistent checksum
* zip stores timestamp for all included files. The timestamps of dex
   files are different across different runs, result in inconsistent
   checksum of output apk/jar.
 * Workaround the issue by using fixed timestamp for dex files.

Change-Id: I21f3a7e32cdfdb07c5f5c140df2e797efd4a8005
2020-04-06 13:10:36 +08:00
dianlujitao
db1caf4597 extract_utils: Fix doubled path separator
* $DST_FILE is already prefixed with "/", don't double it.

Change-Id: I2655313abd7720a54b20f541a95960ae9ec373ad
2020-04-06 12:43:17 +08:00
Han Wang
ae82c34dd5 extract_utils: Fix up 4a2b65fb60
Change-Id: I9e5ea799812ca932de95ec6cbdedb604cbde5736
2020-03-10 09:41:34 +02:00
XiNGRZ
4a2b65fb60 extract_utils: Fix bad substitution for zsh
Change-Id: I5f4fabc62b1ac3da159b36c3ae6080e145948530
2020-03-10 07:49:55 +01:00
dianlujitao
33ee59662f extract_utils: Drop string after semicolon when parsing destination
This fixes parsing when arguments contain colons, a typical usecase
would be:
-vendor/app/TimeService/TimeService.apk;:timeservice_app_cert

Change-Id: I7500ae09632632ddc10734d9b1df267e28286b67
2020-01-10 15:19:47 +01:00
Mohd Faraz
d1d72351b2 extract_utils: Added Suffix uses COMMON_SUFFIX
* COMMON_JAVA_PACKAGE_SUFFIX for jar

 * COMMON_ANDROID_PACKAGE_SUFFIX for apk

Change-Id: I812405dac12ef7183985c66a6e43b0ea5f85989c
Signed-off-by: Mohd Faraz <mohd.faraz.abc@gmail.com>
2019-10-27 11:24:28 +00:00
Rashed Abdel-Tawab
34b5cdc1e3 extract_files: Implement blueprint file generation
Switch to blueprint on:
 - shared objects
 - $partiton/etc/ files
 - JARs
 - executable binaries and scripts
 - APKs

Only /sbin binaries are still in Android.mk because blueprint
doesn't handle sbin installation yet

Change-Id: I1dfd7e8bb575367b2a7fa9e333c4c6fa3aa68180
2019-10-27 11:24:28 +00:00
Rashed Abdel-Tawab
42752d4d4e extract_files: Add initial blueprint creation
Start the copyrights for these at 2019 because the blueprints won't
have existed before then.

Change-Id: I5300109949785dd19555e35d17a1e80daef498b0
2019-10-27 11:24:28 +00:00
Rashed Abdel-Tawab
8aae50df3c extract_files: Match non-treble path prefixes
Some devices put stuff on /system, /system/vendor or even
/system/vendor/odm. Search for these paths too when generating
TARGET_COPY_OUT_$partition variables.

Change-Id: Ie2c087e57aaca02d5ea93f290d5fc50d1315a600
2019-10-27 11:24:28 +00:00
Rashed Abdel-Tawab
8de76b204b extract_files: Support system/ prefixes
With support for 4 independent partitions now, we seriously
need to start putting /system blobs in their own directory.
Add support for file lists with system/ prefixes while
maintaining support for old file lists without it.

Also, TARGET_COPY_OUT_SYSTEM is a thing now, and all devices,
regardless of treble or not, set TARGET_COPY_OUT_$partition
so let's get rid of the treble compat option and default it
to true.

Change-Id: I5b798d293768d7c1e16db3ba01e2de3e083088d7
2019-10-27 11:24:06 +00:00
PIPIPIG233666
17d7dcba43 extract_utils: Fix typo
* ./../../xiaomi/sm6150-common/../../../vendor/lineage/build/tools/extract_utils.sh: line 1: /#!/bin/bash: No such file or directory.

Signed-off-by: PIPIPIG233666 <2212848813@qq.com>
Change-Id: I178f745d4ecb818c38706ff100611df19221065d
2019-09-29 12:49:44 -04:00
Bruno Martins
8194b8e583 extract_utils: Source the prebuilts from the new repo
The prebuilt tools now live in prebuilts/tools-lineage.

Change-Id: I4022210c52e1002aae0c6e622a480e5c7e10c366
2019-09-29 03:06:49 +02:00
Rashed Abdel-Tawab
08e3a27135 extract_files: Add ODM partition blob support
Change-Id: Id94bbef879b6f4bcc20645c259981f9d163649c9
2019-09-27 06:46:44 -07:00
razorloves
b5c2c96a56 extract_utils: Support extracting from product partition
Change-Id: I2d0d01253fdf93d63a26556208d45cd69e76329f
(cherry picked from commit ea15be6e92e3be0c103948d220de64fd2e1d781f)
2019-09-10 11:28:05 +02:00
Rashed Abdel-Tawab
19c36cd7a6 extract_utils: Support multidex
Co-authored-by: Erfan Abdi <erfangplus@gmail.com>
Change-Id: I4881658d303b6c5c7f0b141beb50c1e9100af611
2019-07-18 07:31:36 +02:00
Rashed Abdel-Tawab
1c29c373dd extract_utils: Add functions to extract vendor blobs from vendor.img
Change-Id: I6f37adc955856fa78e719eb30249800abe80020c
2019-06-26 04:37:44 +02:00
Vladimir Oltean
4818c230b3 extract_utils: introduce support for executing blob fixups
* Traditionally, the task of hex-editing blobs has been approached in 2 ways:
  (1) Do it out-of-band, commit the modified blob, and record its edited
      sha1sum in proprietary-files.txt (aka pin it).
  (2) Do it in-band, by adding code to the device-level extract-files.sh
      (usually this performs patchelf or sed). This code runs after the
      extract_utils functions were invoked.
* Problems of approach (1):
    - It relies on verbal (basically commit message) documentation of
      the hex-editing that was done. Makes it more difficult to reproduce.
    - Each time blobs are updated, pinning needs to be temporarily removed,
      hex-editing done again manually and new hash put back.
* Problems of approach (2):
    - It is incompatible with the concept of pinning, which is useful
      for kanging blobs from another device. A pinned blob would either:
        - Match the hash, get hex-edited, then it won't match the hash
          next time around.
        - Not match the hash (because of, say, hex-editing), then the
          extraction script would use an unwanted blob version instead of the
          pinned one (either that, or say "!! file not found in source").

* In summary, this patch adds system-wide support for approach (2) in order
  to address the aforementioned shortcomings.

* At device level, users of extract_utils who wish to perform blob
  fixups can override a blob_fixup() Bash function in their
  extract-files.sh immediately after running "source ${HELPER}". The
  blob_fixup() function will be called by the common extract() function
  after extracting every individual blob, giving the user the
  opportunity to hook custom code after this operation takes place.

* In proprietary-files.txt, the line corresponding to this blob which
  needs fixups can look in one of 2 ways:

(a) vendor/lib64/vendor.qti.gnss@1.0_vendor.so

    Do this if you are taking the blob from the stock ROM. The fixup
    script will always run after the blob is extracted.

(b) vendor/lib64/vendor.qti.gnss@1.0_vendor.so|249c76153f8de014bf2dd2ab623ee3d87741fbc8|f7e9ee8e3804887a2f3939128e860767e6f27258

    Do this if you are kanging the blob from somebody else. The pinning
    logic now applies for both the pre- and the post-fixup hashes. The
    fixup script will only run if the blob doesn't match the hex-edited blob,
    although the fixup script should really be idempotent.

Change-Id: Ifdd73c885d995c645f6210597537693d1a2f903f
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2019-03-03 16:44:53 +02:00
Vladimir Oltean
5238ba8fe5 extract_utils: make --section match words anywhere in the line
Change-Id: Ia3fe2160f0f4bc7da1b5bd31a21080af51b41fab
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2019-03-03 16:42:24 +02:00
Vladimir Oltean
bec92ebaec extract_utils: introduce -k (kang mode) option
* The use case is easier updating of pinned blobs. When --kang is set,
  pinning is automatically ignored, and the script will print lines at
  its output that can be directly copied back into the
  proprietary-files.txt.
* Best served together with --section ${SECTION}, and proper grouping
  of the proprietary-files.txt.

Change-Id: I648fbcbd4580a4a002b00828bcfee18d1e265d7b
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2019-03-03 16:42:24 +02:00
Vladimir Oltean
c503446ec3 extract_utils: cleanup variable names and arguments passed to extract() function
* This also makes the --section argument non-positional, since otherwise
  it is not possible to easily support more than one optional positional
  argument. This is in preparation of one more optional argument to come
  in a follow-up patch: --kang.

Change-Id: Ieb142e0854319defb9a278ab68cd4aeefd0fbdd5
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2019-03-03 16:42:24 +02:00
Vladimir Oltean
78d690d12c extract_utils: be compatible with system-as-root layouts
* The use case is that if you have the following layout:
  $TOP --- system.img
       |
       +-- vendor.img
  you should be able (from $TOP) to:
    mkdir system; mount -o ro,loop system.img system
    mkdir vendor; mount -o ro,loop vendor.img vendor
  and then (from device tree)
    ./extract-files.sh $TOP

  But this doesn't work if system.img is SAR and contains another
  "system" dir inside. This patch makes sure it searches for a "system"
  dir in the provided path as well, if it couldn't find the blob
  anywhere else.

Change-Id: Ib49cd5b587b3a57478a66ff69cf840270c2b1403
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2019-03-03 16:42:24 +02:00
Vladimir Oltean
4fe79b11c1 extract_utils: drop the "/system/" prefix from prints of blob names
* This makes the printed output closer to the proprietary-files.txt syntax

Change-Id: I81b844bb6bb1d1a2f91a39151a892fbfc0bed20b
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2019-03-03 16:42:24 +02:00
Vladimir Oltean
b2c382193d extract_utils: fix mismatch with non-lowercase pinned sha1sum
Change-Id: I86421ca69007ad1992aa67deedf7d2e32bbe4dba
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2019-01-17 11:26:34 +01:00
Vladimir Oltean
b8084ec8a5 extract_utils: avoid doing stuff on files that don't exist
* If an apk/jar doesn't exist, the script would still try to deodex it.
* If an xml doesn't exist, the script would still try to "fix" it.
* Take it easier, man, it's not your fault.

Change-Id: I3061fb48b403da5121e3c17dd9ecdb6cd148bf97
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-12-24 01:32:53 +02:00
Luca Stefani
99a66bffdc extract_utils: Redirect vdexextractor log to /dev/null
Change-Id: I55028997c0396d57c024e6c42a53907f6d1510c8
2018-11-13 22:06:17 +01:00
Luca Stefani
f6096e989a Add CDEX support for boot JAR files
Change-Id: I0039256c883a58c5a5125fd9bff6e6ec3b98688d
2018-10-21 10:28:37 +02:00
TheStrix
5aad07422e extract_utils: Fix output dex file name generated by vdexExtractor
- Reflect the changes required by:
   ff58255706
 - Also fix JARVDEX file path for deodexing jars

Change-Id: Ief355b0bd9e69a67b552d0bd2060f43adb39fb5f
2018-10-07 12:17:35 +02:00
codeworkx
1c29bf66dd extract_utils: add CDEX support via compact_dex_converter
Source: https://github.com/anestisb/vdexExtractor#compact-dex-converter

Change-Id: I63664cae02e51a05e5a62c0c5c0595eead6c6513
2018-10-05 18:25:46 +02:00
Luca Stefani
a9a9cc730c extract_utils: Newline love for adb extraction
Change-Id: Ibbf7a03ed6920d556ed7658c277f37b02d8876b8
2018-07-12 21:28:21 +02:00
Vladimir Oltean
d577325ae2 extract_utils: make get_file() able to search paths with and w/o /system prefix
Change-Id: I75eba627646caa8ceb8a30575124c57e5ee9445c
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-07-08 20:14:12 +02:00
Vladimir Oltean
5f15e3e63c extract_utils: extract(): fix src not being searched in "src:dst;args" spec
* Root cause of the issue is improper naming of variables. Turns out,
  there was no variable to even denote where the blob should have been
  searched for, at "src".
* Previously there was one such variable, suggestively called "TARGET", that
  was desperately trying to serve as both, depending on who +2d hacks harder.
* One such example is "c982836 extract_utils: Fix makefile generation issues".
  That patch deliberately trimmed the "src:" from a spec (therefore
  obviously breaking the search at src) but enabling the searching at
  dst, via the good-for-all TARGET variable.
* This patch introduces the following variables:
  - SRC_FILE: absolute path corresponding to SPEC_SRC_FILE in the
    Android filesystem.
  - DST_FILE: absolute path corresponding to SPEC_DST_FILE in the
    Android filesystem. Somewhat analogous to the old TARGET variable,
    but actually contains the leading / as well (/system/bin/adsprpcd
    instead of system/bin/adsprpcd).
* Use existing common get_file() function (which previously was
  impossible to use, because it was impossible to determine calling
  arguments properly) to reduce complexity of handling adb and disk
  image as blob sources.
* Via the new SRC_FILE and DST_FILE variables, search for a blob in all
  possible locations (including paths stripped of "/system" which
  transforms an absolute path in the Android filesystem into a proper
  relative path to that file in a disk image).

Change-Id: Ic40fb4dc93541d8b3f33fde586b773199cf4ded2
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-07-08 20:14:12 +02:00
Vladimir Oltean
d674771760 extract_utils: extract(): rename DEST variable to VENDOR_REPO_FILE
* This denotes the path of the file that results from the extraction
  process, relative to the "proprietary" directory.
* This is a cleanup patch.

Change-Id: I38e759bb6ed697f0a31ca35a7aa9b9b92f8b6793
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-07-08 20:14:12 +02:00
Vladimir Oltean
d652a0675c extract_utils: extract(): rename ARGS variable to SPEC_ARGS
* This is a cleanup patch.

Change-Id: I2a4aac840c06c4f9b94b77b32ead33352ce395ff
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-07-08 20:14:12 +02:00
Vladimir Oltean
da3b64413d extract_utils: extract(): rename FILE variable to SPEC_SRC_FILE
* This is a cleanup patch.

Change-Id: I971b2b9eba5c1f303c0d5f902feaaad25b3b26ed
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-07-08 20:14:12 +02:00
Vladimir Oltean
411e0695f2 extract_utils: extract(): rename FROM variable to SPEC_DST_FILE
* This is a cleanup patch.

Change-Id: I308b0478484ccf9a594078825731606fce5f5507
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-07-08 20:14:12 +02:00
Vladimir Oltean
9564328417 extract_utils: create src_file() function to mirror behavior of target_file()
* This is a cleanup-only patch.

Change-Id: I144252de765686ac3469652994a04820ecc96d4d
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-07-08 20:14:12 +02:00
Vladimir Oltean
2654eaa7ef extract_utils: prefix_match(): do not strip target_args from its output
* The write_product_copy_files() and write_product_packages() functions
  rely on its undocumented behavior of keeping target_args in the
  returned list, because they are users of target_args (such as
  ";PRESIGNED" etc).
* Make the behavior documented.

Change-Id: If71595dca32abd40039706d4fed2d7f12e005365
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-07-08 20:14:12 +02:00
Vladimir Oltean
6a7946baeb extract_utils: target_file() and target_args() cleanup
* Strip target_args from target_file at callee instead of at caller
* WARNING! Changes (improperly documented) behavior of prefix_match()
  function, which is expected to not strip target_args(), and the root
  cause why stripping target_args was currently done at caller. Will be
  addressed in next patch.

Change-Id: I820d2350aa64ff41374809fcb22f812257132652
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-07-08 20:14:12 +02:00
Vladimir Oltean
59074e6832 Revert "extract_files: Add support for paths without system/"
This reverts commit 8b87b09ef7.
This will be addressed cleaner in the cleanup commit to follow.

Change-Id: Iec013bfafdbb334ab2657b10c433158002809b62
2018-07-08 20:14:12 +02:00
Vladimir Oltean
a8b4f887ea Revert "extract_utils: Fix makefile generation issues"
This reverts commit c982836ca6.
This breaks extracting from src in "src:dst", such as from a disk image.
Will be addressed in the cleanup commit that follows.

Change-Id: Iff84a926f0c3bf908320b43ba40235e0a89db644
2018-07-08 20:14:12 +02:00
Vladimir Oltean
a48b9fe9b6 write_makefiles: fix paths in BUILD_PREBUILT with destination
* proprietary-files.txt entries such as
   "-app/TimeService/TimeService.apk:priv-app/TimeService/TimeService.apk" should
   generate a "LOCAL_SRC_FILES := proprietary/priv-app/TimeService/TimeService.apk"
   in the Makefile definition.
 * However, currently, the prefix_match function is being called on the whole
   PRODUCT_PACKAGES_LIST entry (whole line, including ":"), and therefore,
   TimeService.apk would be included in the APPS list instead of PRIV_APPS.
 * Furthermore, because of the incorrect prefix_match, the generated
   LOCAL_SRC_FILES is "proprietary/app/priv-app/TimeService/TimeService.apk",
   which breaks the build because there is no file at that wrong path.
 * The fix is to make the extract function match up with write_product_packages
   by applying the target_file function on all BUILD_PREBUILT source files.

Change-Id: Ib4859b8854db0f2142bb3f28cce8dd25f7141732
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-04-09 12:55:31 -07:00
Joe Maples
9be579f8d7 extract_utils: Use vdexExtractor for extracting oreo classes.dex
vdexExtractor is a tool made by anestisb that is written in C++
with code taken directly from art. However, anestisb has also added
a quicken decompiler to oatdump that was merged upstream, so we only
need vdexExtractor for 8.1 and 8.0.

Change-Id: Ic2cf2dc627a1ad2fa4d500d02d9eac8b8a9577b5
Signed-off-by: Joe Maples <joe@frap129.org>
2018-03-30 20:46:07 +00:00
Michael Bestas
a3f97c75bf extract_utils: Fix vendor framework makefile generation
* Fix copy-pasta

Change-Id: I9b2f61702985cb59af34ad146fd61bedba043f81
2018-02-27 22:32:50 +02:00
Ethan Chen
5bc3c8426b extract-utils: Use LOCAL_VENDOR_FILE instead of LOCAL_PROPRIETARY_FILE
* Oreo expects VNDK compatible files to be listed as LOCAL_VENDOR_FILE,
  not LOCAL_PROPRIETARY_FILE.

Change-Id: Ia2384c4f3ab3a99b79df52c796c53dc25a0f4a88
2018-02-26 02:17:37 +00:00
Paul Keith
4e75f576f9 extract_utils: Fix regressions from 3f9b94cb67
Change-Id: I78fc47c63e6bfbd5ee98a56a1d80f763e96ac266
2018-01-29 23:33:57 +02:00
Michael Bestas
3f9b94cb67 extract_utils: Split optional args with semicolon instead of colon
* Fix optional flags like PRESIGNED
  after I1f5a1abd6929e4a7e0ccd6370d3a3dd986f94fed

Change-Id: I889f995d7f19b09908cfbaf7474442ce48bfabdb
2018-01-27 00:42:57 +00:00
Michael Bestas
c982836ca6 extract_utils: Fix makefile generation issues
* Fix makefile generation for packages that have set
  a different target destination

* Thanks to rashed and javelinanddart for their help
  in debugging and solving this issue.

Change-Id: I1f5a1abd6929e4a7e0ccd6370d3a3dd986f94fed
2018-01-27 00:42:46 +00:00
conbold
575c63567e extract_utils: Use temporary folder for system dump
* This way it's automatically deleted at the end of the extraction

Change-Id: I77b3357875b8070d370f04c5a245f9aa3ca1939c
2017-12-03 22:44:11 +00:00