flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hung Chang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-1017) Add setParallelism() to Java API documentation
Date Fri, 15 Aug 2014 04:15:18 GMT

    [ https://issues.apache.org/jira/browse/FLINK-1017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14098118#comment-14098118
] 

Hung Chang commented on FLINK-1017:
-----------------------------------

Thank you Fabian.

Could I ask about the general idea to distribute the parallelism at job level?
For example. After setting  
                int degreeOfParallelism = 3;
		ExecutionEnvironment.setDefaultLocalParallelism(degreeOfParallelism);
		final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
(My little sample https://github.com/HungUnicorn/Test/blob/master/ParallelWordCount  line
63)

It will retrieve the following result.
...
2> (action, 1)
2> (against, 1)
2> (and, 12)
1> (a, 5)
0> (of, 15)
....

Does it mean there are three threads 0,1,2 run DataSource -> Map -> Reduce -> DataSink?
Is it thread?

On the other hand, could I ask the idea to dispatch thread2 takes "action" & "against"
while thread1 takes "a", thread0 takes "of"?
Shall I read some documentation?

Thanks for your reading.

> Add setParallelism() to Java API documentation
> ----------------------------------------------
>
>                 Key: FLINK-1017
>                 URL: https://issues.apache.org/jira/browse/FLINK-1017
>             Project: Flink
>          Issue Type: Task
>          Components: Documentation
>    Affects Versions: 0.6-incubating, pre-apache-0.5
>            Reporter: Fabian Hueske
>            Assignee: Hung Chang
>            Priority: Minor
>              Labels: starter
>             Fix For: 0.6-incubating
>
>
> The Java API offers {{setParallelism()}} to control the degree of parallelism for each
operator. This feature is not documented and should be added.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message