beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-159) Support fixed number of shards in sinks
Date Sat, 25 Jun 2016 20:04:37 GMT

    [ https://issues.apache.org/jira/browse/BEAM-159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15349764#comment-15349764
] 

ASF GitHub Bot commented on BEAM-159:
-------------------------------------

GitHub user dhalperi opened a pull request:

    https://github.com/apache/incubator-beam/pull/534

    [BEAM-159][BEAM-92][BEAM-365] Write: add support for setting a fixed number of shards

    And remove special support in Dataflow and Direct runners for it.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/dhalperi/incubator-beam write-numshards

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-beam/pull/534.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #534
    
----
commit ccdcd40c249373a15f10dc70555276e7f3218f12
Author: Dan Halperin <dhalperi@google.com>
Date:   2016-06-14T21:03:41Z

    Write: add support for setting a fixed number of shards
    
    And remove special support in Dataflow and Direct runners for it.

----


> Support fixed number of shards in sinks
> ---------------------------------------
>
>                 Key: BEAM-159
>                 URL: https://issues.apache.org/jira/browse/BEAM-159
>             Project: Beam
>          Issue Type: Improvement
>          Components: beam-model, sdk-java-core
>            Reporter: Eugene Kirpichov
>            Assignee: Daniel Halperin
>
> TextIO supports .withNumShards, however custom sinks, in particular FileBasedSinks, provide
no support for controlling sharding. Some users want this, e.g. http://stackoverflow.com/questions/36316304/set-num-of-output-shard-in-write-tosink-in-dataflow



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message