WriteFileRule: Chunk long content and merge them to result

sbox.textproto files are created when handling genrule. The command
for the genrule and output files are written to this file in the
following format:

commands <
  command: "...."
  copy_after: <
    from: "out/asm-generic/auxvec.h"
    to: "out/soong/.intermediates/kernel/msm-4.19/qti_generate_kernel_headers_arm/gen/asm-generic/auxvec.h"
  >
  copy_after: <
    from: "out/asm-generic/bitsperlong.h"
     to:
     "out/soong/.intermediates/kernel/msm-4.19/qti_generate_kernel_headers_arm/gen/asm-generic/bitsperlong.h"
  >
  ....
>

This file grow by one copy_after entry for each output file.

When generating kenrnel headers where the number of output files are
~1000 we run into problems as the contents of sbox.textproto files
are written to disk by generating a shell script using the following
template: /bin/bash -c 'echo -e "$$0" > $out' $content
If $content is very long as in the case of generating kernel headers we
run into the issue where the command line is so long that the shell
script return E2BIG.

Fix this issue by chuking contents into smaller files and then merge
them as a final step.

Test: Build
Issue: 174444967
Change-Id: I1a74023b4222d19672e4df7edb19810a9cf2136f
This commit is contained in:
Hans Månsson
2020-11-27 12:37:28 +01:00
committed by John Eckerdal
parent 591833471f
commit d3f2bd79e8
3 changed files with 55 additions and 4 deletions

View File

@@ -401,6 +401,23 @@ func ShardPaths(paths Paths, shardSize int) []Paths {
return ret
}
// ShardString takes a string and returns a slice of strings where the length of each one is
// at most shardSize.
func ShardString(s string, shardSize int) []string {
if len(s) == 0 {
return nil
}
ret := make([]string, 0, (len(s)+shardSize-1)/shardSize)
for len(s) > shardSize {
ret = append(ret, s[0:shardSize])
s = s[shardSize:]
}
if len(s) > 0 {
ret = append(ret, s)
}
return ret
}
// ShardStrings takes a slice of strings, and returns a slice of slices of strings where each one has at most shardSize
// elements.
func ShardStrings(s []string, shardSize int) [][]string {