flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aneesha Kaushal <aneesha.kaus...@reflektion.com>
Subject Flink Batch job: All slots for groupReduce task scheduled on same machine
Date Tue, 20 Feb 2018 09:50:40 GMT
Hello, 

I have a fink batch job, where I am grouping dataset on some keys, and then using group reduce.
Parallelism is set to 16. 
The slots for the Map task is distributed across all the machines, but for GroupReduce all
the slots are being assigned to the same machine. Can you help me understand why/when this
can happen? 
Code looks something like: 
dataset.map(MapFunction())
  .groupBy(<keys to groupon>)
  .sortGroup(<key to sort on>, Order.DESCENDING)
  .reduceGroup(GroupReduceFunction()).name("Group reduce")
From flink dashboard: 




Thanks in advance
Aneesha





Mime
View raw message