samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Xinyu Liu <xinyuliu...@gmail.com>
Subject Review Request 51962: SAMZA-1021 Remove the redundent poll waiting inside AsyncRunLoop blockIfBusy
Date Fri, 16 Sep 2016 17:52:22 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51962/
-----------------------------------------------------------

Review request for samza, Boris Shkolnik, Jake Maes, and Yi Pan (Data Infrastructure).


Repository: samza


Description
-------

The waiting inside AsyncRunLoop is redundent given systemconsumers will block on polling for
empty partition. Empty partitions happens when chooser returns null envelope and at the same
time some task worker is avaliable to pick up messages. In this case we should rely on the
systemconsumers.poll() to block instead of blocking in scheduling.

The redundent waiting has been seen in ATC's ingraph where both choose_ns and block_ns are
high at the same time.


Diffs
-----

  samza-core/src/main/java/org/apache/samza/task/AsyncRunLoop.java a510bb0c5914c772438930d27f100b4d360c1296


Diff: https://reviews.apache.org/r/51962/diff/


Testing
-------

Tested by yarn job.


Thanks,

Xinyu Liu


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message