build_soong/docs/native_code_coverage.md

Native Code Coverage for Android

Scope

These instructions are for Android developers to collect and inspect code coverage for C++ and Rust code on the Android platform.

Building with Native Code Coverage Instrumentation

Identify the paths where native code-coverage instrumentation should be enabled and set up the following environment variables.

    export CLANG_COVERAGE=true
    export NATIVE_COVERAGE_PATHS="<paths-to-instrument-for-coverage>"

NATIVE_COVERAGE_PATHS should be a list of paths. Any Android.bp module defined under these paths is instrumented for code-coverage. E.g:

export NATIVE_COVERAGE_PATHS="external/libcxx system/core/adb"

Additional Notes

• Native Code coverage is not supported for host modules or Android.mk modules.
• NATIVE_COVERAGE_PATHS="*" enables coverage instrumentation for all paths.
• Set native_coverage: false blueprint property to always disable code coverage instrumentation for a module. This is useful if this module has issues when building or running with coverage.
• NATIVE_COVERAGE_EXCLUDE_PATHS can be set to exclude subdirs under NATIVE_COVERAGE_PATHS from coverage instrumentation. E.g. NATIVE_COVERAGE_PATHS=frameworks/native NATIVE_COVERAGE_PATHS=frameworks/native/vulkan will instrument all native code under frameworks/native exceptframeworks/native/vulkan.

Running Tests

Collecting Profiles

When an instrumented program is run, the profiles are stored to the path and name specified in the LLVM_PROFILE_FILE environment variable. On Android coverage builds it is set to /data/misc/trace/clang-%p-%20m.profraw.

• %p is replaced by the pid of the process
• %m by the hash of the library/binary
• The 20 in%20m creates a pool of 20 profraw files and "online" profile merging is used to merge coverage to profiles onto this pool.

Reference:LLVM_PROFILE_FILE can include additional specifiers as described here.

For this and following steps, use the acov-llvm.py script: $ANDROID_BUILD_TOP/development/scripts/acov-llvm.py.

There may be profiles in /data/misc/trace collected before the test is run. Clear this data before running the test.

    # Clear any coverage that's already written to /data/misc/trace
    # and reset coverage for all daemons.
    <host>$ acov-llvm.py clean-device

    # Run the test.  The exact command depends on the nature of the test.
    <device>$ /data/local/tmp/$MY_TEST

For tests that exercise a daemon/service running in another process, write out the coverage for those processes as well.

    # Flush coverage of all daemons/processes running on the device.
    <host>$ acov-llvm.py flush

    # Flush coverage for a particular daemon, say adbd.
    <host>$ acov-llvm.py flush adbd

Viewing Coverage Data (acov-llvm.py)

To post-process and view coverage information we use the acov-llvm.py report command. It invokes two LLVM utilities llvm-profdata and llvm-cov. An advanced user can manually invoke these utilities for fine-grained control. This is discussed below.

To generate coverage report need the following parameters. These are dependent on the test/module:

1. One or more binaries and shared libraries from which coverage was collected. E.g.:

   1. ART mainline module contains a few libraries such as libart.so, libart-compiler.so.
   2. Bionic tests exercise code in libc.so and libm.so.

   We need the unstripped copies of these binaries. Source information included in the debuginfo is used to process the coverage data.

2. One or more source directories under $ANDROID_BUILD_TOP for which coverage needs to be reported.

Invoke the report subcommand of acov-llvm.py to produce a html coverage summary:

    $ acov-llvm.py report \
        -s <one-or-more-source-paths-in-$ANDROID_BUILD_TOP \
        -b <one-or-more-(unstripped)-binaries-in-$OUT>

E.g.:

    $ acov-llvm.py report \
        -s bionic \
        -b \
        $OUT/symbols/apex/com.android.runtime/lib/bionic/libc.so \
        $OUT/symbols/apex/com.android.runtime/lib/bionic/libm.so

The script will produce a report in a temporary directory under $ANDROID_BUILD_TOP. It'll produce a log as below:

    generating coverage report in covreport-xxxxxx

A html report would be generated under covreport-xxxxxx/html.

Viewing Coverage Data (manual)

acov-llvm.py report does a few operations under the hood which we can also manually invoke for flexibility.

Post-processing Coverage Files

Fetch coverage files from the device and post-process them to a .profdata file as follows:

    # Fetch the coverage data from the device.
    <host>$ cd coverage_data
    <host>$ adb pull /data/misc/trace/ $TRACE_DIR_HOST

    # Convert from .profraw format to the .profdata format.
    <host>$ llvm-profdata merge --output=$MY_TEST.profdata \
    $TRACE_DIR_HOST/clang-*.profraw

For added specificity, restrict the above command to just the s of the daemon or test processes of interest.

    <host>$ llvm-profdata merge --output=$MY_TEST.profdata \
    $MY_TEST.profraw \
    trace/clang-<pid1>.profraw trace/clang-<pid2>.profraw ...

Generating Coverage report

Documentation on Clang source-instrumentation-based coverage is available here. The llvm-cov utility is used to show coverage from a .profdata file. The documentation for commonly used llvm-cov command-line arguments is available here. (Try llvm-cov show --help for a complete list).

show subcommand

The show command displays the function and line coverage for each source file in the binary.

    <host>$ llvm-cov show \
        --show-region-summary=false
        --format=html --output-dir=coverage-html \
        --instr-profile=$MY_TEST.profdata \
        $MY_BIN \

• In the above command, $MY_BIN should be the unstripped binary (i.e. with debuginfo) since llvm-cov reads some debuginfo to process the coverage data.

  E.g.:

  ```
  <host>$ llvm-cov report \
      --instr-profile=adbd.profdata \
      $LOCATION_OF_UNSTRIPPED_ADBD/adbd \
      --show-region-summary=false
  ```
  

• The -ignore-filename-regex=<regex> option can be used to ignore files that are not of interest. E.g: -ignore-filename-regex="external/*"

• Use the --object=<BIN> argument to specify additional binaries and shared libraries whose coverage is included in this profdata. See the earlier section for examples where more than one binary may need to be used.

  E.g., the following command is used for bionic-unit-tests, which tests both libc.so and libm.so:

  ```
  <host>$ llvm-cov report \
      --instr-profile=bionic.profdata \
      $OUT/.../libc.so \
      --object=$OUT/.../libm.so
  ```
  

• llvm-cov also takes positional SOURCES argument to consider/display only particular paths of interest. E.g:

  ```
  <host>$ llvm-cov report \
      --instr-profile=adbd.profdata \
      $LOCATION_OF_ADBD/adbd \
      --show-region-summary=false \
      /proc/self/cwd/system/core/adb
  ```
  

Note that the paths for the sources need to be prepended with '/proc/self/cwd/'. This is because Android C/C++ compilations run with PWD=/proc/self/cwd and consequently the source names are recorded with that prefix. Alternatively, the --path-equivalence option to llvm-cov can be used.

report subcommand

The report subcommand summarizes the percentage of covered lines to the console. It takes options similar to the show subcommand.