beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Mills (JIRA)" <>
Subject [jira] [Created] (BEAM-384) Streaming BigQueryIO should support user-provided IDs
Date Tue, 28 Jun 2016 20:07:57 GMT
Daniel Mills created BEAM-384:

             Summary: Streaming BigQueryIO should support user-provided IDs
                 Key: BEAM-384
             Project: Beam
          Issue Type: Improvement
          Components: sdk-java-gcp
    Affects Versions: 0.1.0-incubating, 0.2.0-incubating
            Reporter: Daniel Mills
            Assignee: Daniel Halperin
            Priority: Minor

Currently, BigQueryIO always assigns IDs and does a shuffle to ensure that they are atomic.
 This incurs a noticeable cost and is unnecessary if the user already has deterministic IDs
that they can use.  The sink should be able to use these IDs to skip the shuffle.

This message was sent by Atlassian JIRA

View raw message