flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chesnay Schepler <ches...@apache.org>
Subject Re: keyBy and parallelism
Date Thu, 12 Apr 2018 07:31:43 GMT
You will get 16 parallel executions since you specify a parallellism of 
16, however 8 of these will not get any data.

On 11.04.2018 23:29, Hao Sun wrote:
> From what I learnt, you have to control parallelism your self. You can 
> set parallelism on operator or set default one through flink-config.yaml.
> I might be wrong.
>
> On Wed, Apr 11, 2018 at 2:16 PM Christophe Jolif <cjolif@gmail.com 
> <mailto:cjolif@gmail.com>> wrote:
>
>     Hi all,
>
>     Imagine I have a default parallelism of 16 and I do something like
>
>     stream.keyBy("something").flatMap()
>
>     Now let's imagine I have less than 16 keys, maybe 8.
>
>     How many parallel executions of the flatMap function will I get? 8
>     because I have 8 keys, or 16 because I have default parallelism at 16?
>
>     (and I will have follow up questions depending on the answer I
>     suspect ;))
>
>     Thanks,
>     -- 
>     Christophe
>


Mime
View raw message