flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Luis Lázaro <lalaz...@keedio.com>
Subject Flink UI records received/sent chained-unchained operators
Date Wed, 24 May 2017 08:15:39 GMT
Hi everyone, 
my use case:

	• Flink 1.3.0-SNAPSHOT
	• DataStream API.
	• Cluster mode is standalone, 3 TM and a JM.
	• Parallelism is set to 1 on client level from UI.
	• Source is Kafka configured with one single partition.

 In Flink’s UI counter, I am getting a record count “sent" greater than the count of entries
“received".Despite this, total records is correct, ie, neither I losing nor am I getting
more records. Please check out screenshot#1: 



 My question is:  why the total number of records sent minus received is exactly the total
leaked? (250-223 = 27), i expected to see 27 "Records sent”
¿ Is Map->Filter-Filter, as a single operator a SplitStream ? 

.split(
case false => 223   
case true => 27
) 

If I run another job with the same execution plan but unchained operators in global operator
(MAP->FILTER->FILTER) , the account is exactly the same records sent to the account
filtered records. Please check out screenshot#2




PD: There is a similar question in  Flink UI and records sent <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-UI-and-records-sent-td12522.html>
, but I think they are not related.

Thanks in advance, any suggestion is well appreciated.

Best regards, Luis




Mime
View raw message