flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] xccui commented on issue #6003: [FLINK-9289][Dataset] Parallelism of generated operators should have max parallelism of input
Date Wed, 08 Aug 2018 15:58:07 GMT
xccui commented on issue #6003: [FLINK-9289][Dataset] Parallelism of generated operators should
have max parallelism of input
URL: https://github.com/apache/flink/pull/6003#issuecomment-411458158
 
 
   Thanks for the review @fhueske. I also agree that updating `KeyFunctions.appendKeyExtractor()`
can fix this issue, easily and effectively. However, it seems not to be a graceful, or say,
an untimate solution. 
   
   I think the main problem is whether we take the Union operator as a special one without
parallelism (your solution) or a common operator whose parallelism is forcibly set to the
larger one of its inputs (my solution). IMO, the drawback of the former solution is that we
may need to add more Union-specific logic in the future, while the drawback of the later one
is that I'm not sure setting the parallelism like that can really solve the problem once for
all. Anyway, we should forbid the users to set the parallelism for Union since it does not
take effect.
   
   What do you think?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message