flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [flink] zhijiangW commented on issue #9905: [FLINK-14396][network] Implement rudimentary non-blocking network output
Date Tue, 22 Oct 2019 06:15:48 GMT
zhijiangW commented on issue #9905: [FLINK-14396][network] Implement rudimentary non-blocking
network output
URL: https://github.com/apache/flink/pull/9905#issuecomment-544820371
 
 
   Thanks for the further review and I updated the codes for new comments!
   
   Some further thoughts for the following work:
   
   1. I would create a separate ticket for refactoring the logic between local and global
pool. It might not be feasible to make the global request non-blocking completely. In other
words, we can not return to the mailbox directly when fails to get available buffer from global
pool, because we do not want to store the already emitted record in serialization stack. But
we could improve the current way a bit. It is no need to wait 2 seconds in `LocalBufferPool`
before next try to request global buffer. Instead it can be blocked there to monitor `isAvailable`
from global and be waked up immediately when the future is completed.
   
   2. Refactor the backpressure monitor in [FLINK-14472](https://issues.apache.org/jira/browse/FLINK-14472).
There are two cases for causing backpressure. One is via `BufferProvider#isAvailable` and
another is via judging whether it is waiting for global buffer available.  We could provide
a transparent interface method for giving the result of backpressure. In the LocalBufferPool
implementation it could consider above two cases together and the backpressure monitor does
not need to understand the details of `LocalBufferPool`. So it is no need to analyze the task
thread stack, and also solve the previous problem that backpressure is invalid if `requestBuffer`
is called by other threads.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message