sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hari Sekhon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-2840) Sqoop Hcat Int partition Error
Date Wed, 24 May 2017 13:05:05 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-2840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022855#comment-16022855
] 

Hari Sekhon commented on SQOOP-2840:
------------------------------------

Seeing this on Sqoop 1.4.6 on HDP 2.5 as well for date type partition.

> Sqoop Hcat Int partition Error
> ------------------------------
>
>                 Key: SQOOP-2840
>                 URL: https://issues.apache.org/jira/browse/SQOOP-2840
>             Project: Sqoop
>          Issue Type: Bug
>          Components: metastore
>    Affects Versions: 1.4.5
>            Reporter: Biswajit Nayak
>
> Hi All, 
> I am trying to do a SQOOP export from hive( integer type partition) to mysql and fail
with the following error. 
> Versions:-
> {code}
> Hadoop :-  2.7.1
> Hive      :-  1.2.0
> Sqoop   :-  1.4.5
> {code}
> Table in Hive :-
> {code}
> hive> use default;
> OK
> Time taken: 0.028 seconds
> hive> describe emp_details1;
> OK
> id                      int                                         
> name                    string                                      
> deg                     string                                      
> dept                    string                                      
> salary                  int                                         
> # Partition Information      
> # col_name              data_type               comment             
> salary                  int                                         
> Time taken: 0.125 seconds, Fetched: 10 row(s)
> hive> 
> hive> select * from emp_details1;
> OK
> 1201    gopal           50000
> 1202    manisha         50000
> 1203    kalil           50000
> 1204    prasanth        50000
> 1205    kranthi         50000
> 1206    satish          50000
> Time taken: 0.195 seconds, Fetched: 6 row(s)
> hive> 
> {code}
> Conf added to Hive megastore site.xml
> {code}
> [alti-test-01@hdpnightly271-ci-91-services ~]$ grep -A5 -B2 -i "hive.metastore.integral.jdo.pushdown"
/etc/hive-metastore/hive-site.xml 
>     </property>
>     <property>
>         <name>hive.metastore.integral.jdo.pushdown</name>
>         <value>TRUE</value>
>     </property>
> </configuration>
> [alti-test-01@hdpnightly271-ci-91-services ~]$ 
> {code}
> The issue remains same
> {code}
> [alti-test-01@hdpnightly271-ci-91-services ~]$ /opt/sqoop-1.4.5/bin/sqoop export --connect
jdbc:mysql://localhost:3306/test --username hive --password ********* --table employee --hcatalog-database
default --hcatalog-table emp_details1
> Warning: /opt/sqoop-1.4.5/bin/../../hbase does not exist! HBase imports will fail.
> Please set $HBASE_HOME to the root of your HBase installation.
> Warning: /opt/sqoop-1.4.5/bin/../../accumulo does not exist! Accumulo imports will fail.
> Please set $ACCUMULO_HOME to the root of your Accumulo installation.
> Warning: /opt/sqoop-1.4.5/bin/../../zookeeper does not exist! Accumulo imports will fail.
> Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
> 16/02/12 08:04:00 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5
> 16/02/12 08:04:00 WARN tool.BaseSqoopTool: Setting your password on the command-line
is insecure. Consider using -P instead.
> 16/02/12 08:04:00 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
> 16/02/12 08:04:00 INFO tool.CodeGenTool: Beginning code generation
> 16/02/12 08:04:01 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `employee`
AS t LIMIT 1
> 16/02/12 08:04:01 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `employee`
AS t LIMIT 1
> 16/02/12 08:04:01 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /opt/hadoop
> Note: /tmp/sqoop-alti-test-01/compile/1b0d4b1c30f167eb57ef488232ab49c8/employee.java
uses or overrides a deprecated API.
> Note: Recompile with -Xlint:deprecation for details.
> 16/02/12 08:04:07 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-alti-test-01/compile/1b0d4b1c30f167eb57ef488232ab49c8/employee.jar
> 16/02/12 08:04:07 INFO mapreduce.ExportJobBase: Beginning export of employee
> 16/02/12 08:04:08 INFO mapreduce.ExportJobBase: Configuring HCatalog for export job
> 16/02/12 08:04:08 INFO hcat.SqoopHCatUtilities: Configuring HCatalog specific details
for job
> 16/02/12 08:04:08 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `employee`
AS t LIMIT 1
> 16/02/12 08:04:08 INFO hcat.SqoopHCatUtilities: Database column names projected : [id,
name, deg, salary, dept]
> 16/02/12 08:04:08 INFO hcat.SqoopHCatUtilities: Database column name - info map :
>     id : [Type : 4,Precision : 11,Scale : 0]
>     name : [Type : 12,Precision : 20,Scale : 0]
>     deg : [Type : 12,Precision : 20,Scale : 0]
>     salary : [Type : 4,Precision : 11,Scale : 0]
>     dept : [Type : 12,Precision : 10,Scale : 0]
> 16/02/12 08:04:10 INFO hive.metastore: Trying to connect to metastore with URI thrift://hive-hdpnightly271-ci-91.test.altiscale.com:9083
> 16/02/12 08:04:10 INFO hive.metastore: Connected to metastore.
> 16/02/12 08:04:11 INFO hcat.SqoopHCatUtilities: HCatalog full table schema fields = [id,
name, deg, dept, salary]
> 16/02/12 08:04:12 ERROR tool.ExportTool: Encountered IOException running export job:
java.io.IOException: The table provided default.emp_details1 uses unsupported  partitioning
key type  for column salary : int.  Only string fields are allowed in partition columns in
Catalog
> {code}
> The issue has been discussed in [HIVE-2702] and suppose to be solved in hive 0.12 version.
But I am hitting this issue in Hive 1.2.0 version. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message