Commit Graph

4 Commits

Author SHA1 Message Date
Jeff Gaston
175f34c5c3 Revert "Revert "Refactor rate_limit.go for more clarify""
This reverts commit 526416b1e4.

Figured out a fix for the deadlocks; resubmitting the patch.
The first version was deadlocking because the switch statement in
zipWriter.write would choose a specific zip entry to start writing,
but the individual chunks may not have all necessarily been compressed
yet. When each individual chunk was made to require to request its own
allocations, the compression of the chunks of the file being currently
written could be blocked waiting for memory to be freed by chunks from
other files that hadn't yet started being written.

This patch is much like the original except it preallocates the memory
for the entire file upfront (and happens to use the total file size
rather than the compressed size, but I didn't observe that to cause any
performance differences).

Bug: 64536066
Test: m -j dist showcommands # which runs soong_zip to package everything

Change-Id: Id1d7ff415e54d3a6be71188abbdbbbab5a719fcf
2017-08-17 21:46:46 -07:00
Colin Cross
526416b1e4 Revert "Refactor rate_limit.go for more clarify"
This reverts commit 69f3b3e946.

Caused rare deadlocks.

Bug: 64536066
Bug: 64813447
Change-Id: Ieb1b931bb2c0afdd8bf8edbfc32c373df3c08d8d
2017-08-17 23:54:51 +00:00
Jeff Gaston
69f3b3e946 Refactor rate_limit.go for more clarify
This wasn't intended to change the behavior, but it does slightly.

  Previously any requests to acquire memory wouldn't block; only
  (subsequent) requests for execution would block (if another
  caller had already consumed the memory quota). Now the requests
  for memory can also also block.

It turns out in a brief test on my computer that soong_zip
runs about 10 to 20% faster with this change than without it.

  The final step involving soong_zip decreased from about
  3.6 sec to about 3.3 sec in tests on my computer.

  When testing the process of re-zipping the contents of
  angler-target_files*.zip , the time decreased from about
  6.3 sec to about 5.3 sec in tests on my computer, and the
  peak memory usage reported by `top` decreased from about
  1.5g to 1g

Bug: 64536066
Test: m -j dist showcommands # which runs soong_zip to package everything
Change-Id: I0422e4c363c675bb7a93309fac4616c768bfbe8f
2017-08-16 15:54:25 -07:00
Dan Willemsen
47ec28f3b1 Rename soong_jar to soong_zip
This is a general purpose tool that happens to contain some jar specific
features.

Change-Id: I05f4654d4517c245ad7a3c15492e0d2368bbf64f
2016-08-11 01:30:16 -07:00