flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-4419) Batch improvement for supporting dfs as a ResultPartitionType
Date Fri, 21 Oct 2016 07:27:59 GMT

    [ https://issues.apache.org/jira/browse/FLINK-4419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15594339#comment-15594339
] 

ASF GitHub Bot commented on FLINK-4419:
---------------------------------------

GitHub user shuai-xu opened a pull request:

    https://github.com/apache/flink/pull/2674

    Jira 4444

    This pr add a DFSInputChannel and DFSSubpartition, which enable user ouput the results
of an operator to DFS and then downstream operators can read data from DFS.
    
    This is mainly used for batch jobs. This is party of the issue #[4419](https://issues.apache.org/jira/browse/FLINK-4419).

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/alibaba/flink jira-4444

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/2674.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2674
    
----
commit 0f9378fa48c344b37977b8e07b93adac762965ac
Author: shuai.xus <shuai.xus@alibaba-inc.com>
Date:   2016-10-17T09:54:28Z

    [FLINK-4444][runtime]Add a InputChannel and ResultSubPartition for DFS

commit 34d6e9006bd321d41d0b0736638099c76d93f2b3
Author: shuai.xus <shuai.xus@alibaba-inc.com>
Date:   2016-10-21T07:18:33Z

    [FLINK-4444][runtime]Add unit test for DFSInputChannel and DFSSubPartition

----


> Batch improvement for supporting dfs as a ResultPartitionType
> -------------------------------------------------------------
>
>                 Key: FLINK-4419
>                 URL: https://issues.apache.org/jira/browse/FLINK-4419
>             Project: Flink
>          Issue Type: Improvement
>          Components: Batch Connectors and Input/Output Formats
>            Reporter: shuai.xu
>            Assignee: shuai.xu
>
> This is the root issue to track a improvement for batch, which will enable dfs as a ResultPartitionType,
so that upstream node can exist totally after finished and need not be restarted if downstream
nodes fail.
> Full design is shown in (https://docs.google.com/document/d/15HtCtc9Gk8SyHsAezM7Od1opAHgnxLeHm3VX7A8fa-4/edit#).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message