sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Markus Kemper (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SQOOP-3076) Modify default support with (import + --hcatalog + --as-textfile) and (DATE,TIMESTAMP)
Date Sat, 10 Dec 2016 17:31:59 GMT
Markus Kemper created SQOOP-3076:
------------------------------------

             Summary: Modify default support with (import + --hcatalog + --as-textfile) and
(DATE,TIMESTAMP)
                 Key: SQOOP-3076
                 URL: https://issues.apache.org/jira/browse/SQOOP-3076
             Project: Sqoop
          Issue Type: Improvement
            Reporter: Markus Kemper


Please consider modifying the default behavior when RDBMS types are supported in Hive as seen
in the test case below for (DATE and TIMESTAMP) 

Test Case
{noformat}
sqoop eval --connect $MYCONN --username $MYUSER --password $MYPSWD --query "drop table t1_dates"
sqoop eval --connect $MYCONN --username $MYUSER --password $MYPSWD --query "create table t1_dates
(c1_int integer, c2_date date, c3_timestamp timestamp)"
sqoop eval --connect $MYCONN --username $MYUSER --password $MYPSWD --query "select dbms_metadata.get_ddl('TABLE',
'T1_DATES', 'SQOOP') from dual"
sqoop eval --connect $MYCONN --username $MYUSER --password $MYPSWD --query "insert into t1_dates
values (1, current_date, current_timestamp)"
sqoop eval --connect $MYCONN --username $MYUSER --password $MYPSWD --query "select * from
t1_dates"

Output:
------------------------
| DBMS_METADATA.GET_DDL('TABLE','T1_DATES','SQOOP') | 
------------------------
| 
  CREATE TABLE "SQOOP"."T1_DATES" 
   (	"C1_INT" NUMBER(*,0), 
	"C2_DATE" DATE, 
	"C3_TIMESTAMP" TIMESTAMP (6)
   ) SEGMENT CREATION DEFERRED 
  PCTFREE 10 PCTUSED 40 INITRANS 1 MAXTRANS 255 
 NOCOMPRESS LOGGING
  TABLESPACE "SQOOP"  | 
------------------------
---
------------------------------------------------
| C1_INT               | C2_DATE | C3_TIMESTAMP | 
------------------------------------------------
| 1                    | 2016-12-10 15:48:23.0 | 2016-12-10 15:48:23.707327 | 
------------------------------------------------

#################
# STEP 02 - Demonstrate Current Default Behavior
#################

sqoop import --connect $MYCONN --username $MYUSER --password $MYPSWD --table T1_DATES --hcatalog-database
default --hcatalog-table t1_dates_text --create-hcatalog-table --hcatalog-storage-stanza 'stored
as textfile' --num-mappers 1
beeline -u jdbc:hive2:// -e "use default; describe t1_dates_text; select * from t1_dates_text;"

+---------------+----------------+----------+--+
|   col_name    |   data_type    | comment  |
+---------------+----------------+----------+--+
| c1_int        | decimal(38,0)  |          |
| c2_date       | string         |          |
| c3_timestamp  | string         |          |
+---------------+----------------+----------+--+
---
+-----------------------+------------------------+-----------------------------+--+
| t1_dates_text.c1_int  | t1_dates_text.c2_date  | t1_dates_text.c3_timestamp  |
+-----------------------+------------------------+-----------------------------+--+
| 1                     | 2016-12-10 15:48:23.0  | 2016-12-10 15:48:23.707327  |
+-----------------------+------------------------+-----------------------------+--+

#################
# STEP 03 - Demonstrate Suggested Default Behavior
#################

beeline -u jdbc:hive2:// -e "use default; drop table t1_dates_text;"
sqoop import --connect $MYCONN --username $MYUSER --password $MYPSWD --table T1_DATES --hcatalog-database
default --hcatalog-table t1_dates_text --create-hcatalog-table --hcatalog-storage-stanza 'stored
as textfile' --num-mappers 1 --map-column-hive c2_date=date,c3_timestamp=timestamp
beeline -u jdbc:hive2:// -e "use default; describe t1_dates_text; select * from t1_dates_text;"

+---------------+----------------+----------+--+
|   col_name    |   data_type    | comment  |
+---------------+----------------+----------+--+
| c1_int        | decimal(38,0)  |          |
| c2_date       | date           |          |
| c3_timestamp  | timestamp      |          |
+---------------+----------------+----------+--+
--
+-----------------------+------------------------+-----------------------------+--+
| t1_dates_text.c1_int  | t1_dates_text.c2_date  | t1_dates_text.c3_timestamp  |
+-----------------------+------------------------+-----------------------------+--+
| 1                     | 2016-12-10             | 2016-12-10 15:48:23.707327  |
+-----------------------+------------------------+-----------------------------+--+
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message