beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Work logged] (BEAM-5151) Add EXTERNAL to CREATE TABLE statement
Date Thu, 06 Sep 2018 20:48:00 GMT

     [ https://issues.apache.org/jira/browse/BEAM-5151?focusedWorklogId=141931&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-141931
]

ASF GitHub Bot logged work on BEAM-5151:
----------------------------------------

                Author: ASF GitHub Bot
            Created on: 06/Sep/18 20:47
            Start Date: 06/Sep/18 20:47
    Worklog Time Spent: 10m 
      Work Description: amaliujia commented on issue #6252: [BEAM-5151][SQL] create external
table
URL: https://github.com/apache/beam/pull/6252#issuecomment-419235850
 
 
   Ping @XuMingmin @xumingming just in case this PR will have side effect on your side.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 141931)
    Time Spent: 3h 40m  (was: 3.5h)

> Add EXTERNAL to CREATE TABLE statement
> --------------------------------------
>
>                 Key: BEAM-5151
>                 URL: https://issues.apache.org/jira/browse/BEAM-5151
>             Project: Beam
>          Issue Type: Improvement
>          Components: dsl-sql
>            Reporter: Rui Wang
>            Assignee: Rui Wang
>            Priority: Major
>          Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> BeamSQL allows [CREATE TABLE|https://beam.apache.org/documentation/dsls/sql/create-table/] statements
to register virtual tables from external storage systems (e.g. BigQuery). 
>  
> BeamSQL is not a storage system, so any table registered by "CREATE TABLE" statement
is essentially equivalent to be registered by "CREATE EXTERNAL TABLE", which requires the
user to provide a LOCATION and BeamSQL will register the table outside of current execution
environment based on LOCATION.
>  
> So I propose to add EXTERNAL keyword to "CREATE TABLE" in BeamSQL to help users understand
they are registering tables, and BeamSQL does not create non existing tables by running CREATE
TABLE (at least on some storage systems, if not all). 
>  
> We can make the EXTERNAL keyword either required or optional.
>  
> If we make the EXTERNAL keyword required:
>  
> Pros:
> a. We can get rid of the registering table semantic on CREATE TABLE. 
> b, We keep the room that we could add CREATE TABLE back in the future if we want CREATE
TABLE to create, rather than not only register tables in BeamSQL. 
>  
> Cons:
> 1. CREATE TABLE syntax will not be supported so existing BeamSQL pipelines which has
CREATE TABLE require changes.
> 2. It's required to type tedious EXTERNAL keyword every time, especially in SQL Shell.
>  
> If we make the EXTERNAL keyword optional, we will have reversed pros and cons above.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message