flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-5473) setMaxParallelism() higher than 1 is possible on non-parallel operators
Date Fri, 20 Jan 2017 15:24:26 GMT

    [ https://issues.apache.org/jira/browse/FLINK-5473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15831939#comment-15831939

ASF GitHub Bot commented on FLINK-5473:

GitHub user StefanRRichter opened a pull request:


    [FLINK-5473] Limit MaxParallelism to 1 for non-parallel operators and improve choice of
max parallelism without explicit configuration

    This PR limits the maximum parallelism for non-parallel operator to 1.
    Furthermore, this improves the default behaviour if the user did not explicitly specify
a maximum parallelism. In particular, maximum parallelism can now be derived from savepoints,
allowing users that migrate from Flink 1.1 to Flink 1.2 to keep their job unchanged.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/StefanRRichter/flink recover-max-para

Alternatively you can review and apply these changes as the patch at:


To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #3182
commit 20d0a3fc88c85a71b692c5408fc7b2fd33da8ff2
Author: Stefan Richter <s.richter@data-artisans.com>
Date:   2017-01-16T13:31:22Z

    [FLINK-5473] Limit max parallelism to 1 for non-parallel operators

commit f6081f319b7f2ef8615557743d906bc4585445f7
Author: Stefan Richter <s.richter@data-artisans.com>
Date:   2017-01-16T17:41:37Z

    [FLINK-5473] Better default behaviours for unspecified maximum parallelism


> setMaxParallelism() higher than 1 is possible on non-parallel operators
> -----------------------------------------------------------------------
>                 Key: FLINK-5473
>                 URL: https://issues.apache.org/jira/browse/FLINK-5473
>             Project: Flink
>          Issue Type: Bug
>          Components: DataStream API
>    Affects Versions: 1.2.0
>            Reporter: Robert Metzger
>            Assignee: Stefan Richter
> While trying out Flink 1.2, I found out that you can set a maxParallelism higher than
1 on a non-parallel operator.
> I think we should have the same semantics as the setParallelism() method.
> Also, when setting a global maxParallelism in the execution environment, it will be set
as a default value for the non-parallel operator.
> When restoring a savepoint from 1.1, you have to set the maxParallelism to the parallelism
of the 1.1 job. Non-parallel operators will then also get the maxPar set to this value, leading
to an error on restore.
> So currently, users restoring from 1.1 to 1.2 have to manually set the maxParallelism
to 1 for all non-parallel operators.

This message was sent by Atlassian JIRA

View raw message