flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [flink] pnowojski edited a comment on issue #8455: [FLINK-12284][Network, Metrics]Fix the incorrect inputBufferUsage metric in credit-based network mode
Date Mon, 27 May 2019 09:07:12 GMT
pnowojski edited a comment on issue #8455: [FLINK-12284][Network,Metrics]Fix the incorrect
inputBufferUsage metric in credit-based network mode
URL: https://github.com/apache/flink/pull/8455#issuecomment-496138201
 
 
   @Aitozi :
   
   I think your proposal makes sense. Having 3 metrics for credit based flow control sounds
as the best, the most explicit solution for me.
   
   > Hi @pnowojski , what is the reason for the second point ?
   
   Which second point did you mean @Aitozi ? Or did @zhijiangW answer your question?
   
   @zhijiangW :
   
   > Maybe my previous expression is not very clear. If floating buffers are used, I only
mean the exclusive buffers for one/some RemoteInputChannels are also expected to be used eventually,
but not indicate the exclusive buffers for all the channels are used.
   
   👍
   
   I meant that from the perspective of such data skew, just knowledge of `floating` buffers
usage is not enough for performance tuning. 
   
   If you see that `floating` buffers usage is constantly jumping, between empty & full,
it means that you should probably increase the either floating buffer pool or exclusive buffer
pool. Now depending if you have a data skew and mostly unused `exclusive` buffers, you would
be better off increasing `floating` buffer pool and potentially shrinking `exclusive`. While
if you do not have data skew (and mostly used `exclusive` buffer pool % a couple of idling
channels), you would be better off increasing `exclusive` buffer pool.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message