hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (Jira)" <j...@apache.org>
Subject [jira] [Work logged] (HIVE-25405) Implement Connector Provider for Amazon Redshift
Date Fri, 30 Jul 2021 19:12:00 GMT

     [ https://issues.apache.org/jira/browse/HIVE-25405?focusedWorklogId=631849&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-631849
]

ASF GitHub Bot logged work on HIVE-25405:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 30/Jul/21 19:11
            Start Date: 30/Jul/21 19:11
    Worklog Time Spent: 10m 
      Work Description: dantongdong commented on a change in pull request #2546:
URL: https://github.com/apache/hive/pull/2546#discussion_r680161971



##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/dataconnector/jdbc/RedshiftConnectorProvider.java
##########
@@ -0,0 +1,101 @@
+package org.apache.hadoop.hive.metastore.dataconnector.jdbc;
+
+import org.apache.hadoop.hive.metastore.ColumnType;
+import org.apache.hadoop.hive.metastore.api.DataConnector;
+import org.apache.hadoop.hive.metastore.api.MetaException;
+import org.apache.hadoop.hive.metastore.api.Table;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.sql.ResultSet;
+import java.sql.SQLException;
+
+public class RedshiftConnectorProvider extends AbstractJDBCConnectorProvider {
+    private static Logger LOG = LoggerFactory.getLogger(RedshiftConnectorProvider.class);
+
+    private static final String DRIVER_CLASS = "com.amazon.redshift.jdbc42.Driver".intern();
+
+    public RedshiftConnectorProvider(String dbName, DataConnector dataConn) {
+        super(dbName, dataConn, DRIVER_CLASS);
+    }
+
+    @Override protected ResultSet fetchTableMetadata(String tableName) throws MetaException
{

Review comment:
       It seems like fetchTableMetadata and fetchTableNames method can be utilized across
all the database type we currently support(except for MySQL). Do you think may be better to
move the implementation of those two methods to AbstractJDBCConnectorProvider so that the
other database types can benefit from it? There is a helper method named fetchTablesViaDBMetaData
in AbstractJDBCConnectorProvider that may be helpful.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 631849)
    Time Spent: 20m  (was: 10m)

> Implement Connector Provider for Amazon Redshift
> ------------------------------------------------
>
>                 Key: HIVE-25405
>                 URL: https://issues.apache.org/jira/browse/HIVE-25405
>             Project: Hive
>          Issue Type: Sub-task
>            Reporter: Narayanan Venkateswaran
>            Assignee: Narayanan Venkateswaran
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 20m
>  Remaining Estimate: 0h
>
> h2. +DB Level Federation to Redshift+
> The work in this Jira helps setup federation to redshift databases at the DB level, in
the following way.
> {noformat}
> 0: jdbc:hive2://> describe connector redshift_test_7;
> 21/07/30 20:53:53 [343005b1-06f3-4891-81d1-0c6ad8612260 main]: WARN lazy.LazyStruct:
Extra bytes detected at the end of the row! Ignoring similar problems.
> +------------------+-----------+----------------------------------------------------+-------------+-------------+--------------------------+
> |       name       |   type    |                        url                         |
owner_name  | owner_type  |         comment          |
> +------------------+-----------+----------------------------------------------------+-------------+-------------+--------------------------+
> | redshift_test_7  | redshift  | jdbc:redshift://<url>                        
     | narayanan   | USER        | test redshift connector  |
> +------------------+-----------+----------------------------------------------------+-------------+-------------+--------------------------+
> 1 row selected (0.054 seconds) {noformat}
> {noformat}
> 0: jdbc:hive2://> describe database db_redshift_7;
> 21/07/30 20:53:38 [343005b1-06f3-4891-81d1-0c6ad8612260 main]: WARN lazy.LazyStruct:
Extra bytes detected at the end of the row! Ignoring similar problems.
> +----------------+----------+-----------+------------------+-------------+-------------+------------------+----------------+
> |    db_name     | comment  | location  | managedlocation  | owner_name  | owner_type
 |  connector_name  | remote_dbname  |
> +----------------+----------+-----------+------------------+-------------+-------------+------------------+----------------+
> | db_redshift_7  |          |           |                  | narayanan   | USER     
  | redshift_test_7  | dev            |
> +----------------+----------+-----------+------------------+-------------+-------------+------------------+----------------+
> 1 row selected (0.066 seconds) {noformat}
> {noformat}
> 0: jdbc:hive2://> use db_redshift_7;
> No rows affected (1.391 seconds) {noformat}
> {noformat}
> 0: jdbc:hive2://> show tables;
> +-----------------+
> |    tab_name     |
> +-----------------+
> | accommodations  |
> | category        |
> | date            |
> | event           |
> | listing         |
> | sales           |
> | sample          |
> | test_float_4    |
> | test_int_2      |
> | test_int_4      |
> | test_int_8      |
> | test_time       |
> | test_time_2     |
> | test_timestamp  |
> | users           |
> | venue           |
> | zipcode         |
> +-----------------+
> 17 rows selected (7.773 seconds) {noformat}
> {noformat}
> 0: jdbc:hive2://> select * from test_float_4;
> +-----------------+
> | test_float_4.i  |
> +-----------------+
> | 1.0             |
> +-----------------+
> 1 row selected (18.214 seconds) {noformat}
> {noformat}
> 0: jdbc:hive2://> select * from test_int_8;
> +---------------+
> | test_int_8.i  |
> +---------------+
> | 1             |
> +---------------+
> 1 row selected (12.626 seconds) {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message