beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Mills (JIRA)" <j...@apache.org>
Subject [jira] [Created] (BEAM-384) Streaming BigQueryIO should support user-provided IDs
Date Tue, 28 Jun 2016 20:07:57 GMT
Daniel Mills created BEAM-384:
---------------------------------

             Summary: Streaming BigQueryIO should support user-provided IDs
                 Key: BEAM-384
                 URL: https://issues.apache.org/jira/browse/BEAM-384
             Project: Beam
          Issue Type: Improvement
          Components: sdk-java-gcp
    Affects Versions: 0.1.0-incubating, 0.2.0-incubating
            Reporter: Daniel Mills
            Assignee: Daniel Halperin
            Priority: Minor


Currently, BigQueryIO always assigns IDs and does a shuffle to ensure that they are atomic.
 This incurs a noticeable cost and is unnecessary if the user already has deterministic IDs
that they can use.  The sink should be able to use these IDs to skip the shuffle.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message