beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-3137) BigQueryIO.write() should better verify user schemas
Date Fri, 03 Nov 2017 21:00:18 GMT

    [ https://issues.apache.org/jira/browse/BEAM-3137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16238370#comment-16238370
] 

ASF GitHub Bot commented on BEAM-3137:
--------------------------------------

GitHub user jkff opened a pull request:

    https://github.com/apache/beam/pull/4075

    [BEAM-3137] Improved validation of DynamicDestinations results in BigQueryIO.write

    https://issues.apache.org/jira/browse/BEAM-3137
    
    R: @reuvenlax 

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/jkff/incubator-beam improve-bq-validation

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/beam/pull/4075.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #4075
    
----
commit 7da46f957ffe345c1ac4721e510f0b6c38a12174
Author: Eugene Kirpichov <ekirpichov@gmail.com>
Date:   2017-11-03T19:35:06Z

    Improved validation of DynamicDestinations results in BigQueryIO.write

----


> BigQueryIO.write() should better verify user schemas
> ----------------------------------------------------
>
>                 Key: BEAM-3137
>                 URL: https://issues.apache.org/jira/browse/BEAM-3137
>             Project: Beam
>          Issue Type: Bug
>          Components: sdk-java-gcp
>            Reporter: Eugene Kirpichov
>            Assignee: Reuven Lax
>             Fix For: 2.3.0
>
>
> BigQueryIO.write() currently misses the following schema-related validations which can
lead to confusing errors:
> - withSchemaFromView() doesn't check that the passed view is not null
> - When using withSchemaFromView(), there's no run-time check that the view actually contains
the TableDestination referenced by the tableFn - instead it may issue a load job with no schema
> There might be more validations missing.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message