sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "tech.availkevin" <tech.availke...@gmail.com>
Subject Re: Exception in importing data into Hive
Date Sat, 28 Jun 2014 06:07:25 GMT
? 2014-6-28 0:01, Brenden Cobb ??:
> I think you're missing a few arguments to complete the operation. For 
> example:
>
> Sqoop import --connect jdbc:mysql:// -username username -password 
> password --table tableName  --hive-table tableName --create-hive-table 
> --hive-import --hive-home path/to/hive_home
>
> This is assuming the hive table does not already exist. If 
> the Hive table exists, drop the create-hive-table argument.
>
>
> From: "tech.availkevin" <tech.availkevin@gmail.com 
> <mailto:tech.availkevin@gmail.com>>
> Reply-To: "user@sqoop.apache.org <mailto:user@sqoop.apache.org>" 
> <user@sqoop.apache.org <mailto:user@sqoop.apache.org>>
> Date: Friday, June 27, 2014 8:48 AM
> To: "user@sqoop.apache.org <mailto:user@sqoop.apache.org>" 
> <user@sqoop.apache.org <mailto:user@sqoop.apache.org>>
> Subject: Exception in importing data into Hive
>
> Hi,can you help me?
>
> I'm trying to import mysql table into hive using the sqoop import
> command.it seemms that something wrong happened in which the sqoop
> import command tried to execute the hive script for writting data into
> hive tables,as follows:
>
> [root@localhost bin]# ./sqoop import --connect
> jdbc:mysql://192.168.16.2/serviceorderno --username root -P -m 2 --table
> productinfo --hive-import --hive-table default.productinfo -m 1
> --hive-home /root/hive/apache-hive-0.13.1-bin
> ......
> 14/06/27 14:04:52 INFO hive.HiveImport: Loading uploaded data into Hive
> 14/06/27 14:04:57 INFO hive.HiveImport: 14/06/27 14:04:57 WARN
> conf.HiveConf: DEPRECATED: hive.metastore.ds.retry.* no longer has any
> effect. Use hive.hmshandler.retry.* instead
> 14/06/27 14:04:58 INFO hive.HiveImport:
> 14/06/27 14:04:58 INFO hive.HiveImport: Logging initialized using
> configuration in
> jar:file:/root/hive/apache-hive-0.13.1-bin/lib/hive-common-0.13.1.jar!/hive-log4j.properties
> 14/06/27 14:05:09 INFO hive.HiveImport: Exception in thread "main"
> java.lang.RuntimeException: java.lang.RuntimeException: Unable to
> instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
> 14/06/27 14:05:09 INFO hive.HiveImport: at
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:346)
> 14/06/27 14:05:09 INFO hive.HiveImport: at
> org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
> 14/06/27 14:05:09 INFO hive.HiveImport: at
> org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
> 14/06/27 14:05:09 INFO hive.HiveImport: at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 14/06/27 14:05:09 INFO hive.HiveImport: at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> 14/06/27 14:05:09 INFO hive.HiveImport: at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ......
> 14/06/27 14:05:09 ERROR tool.ImportTool: Encountered IOException running
> import job: java.io.IOException: Hive exited with status 1
> at
> org.apache.sqoop.hive.HiveImport.executeExternalHiveScript(HiveImport.java:385)
> at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:335)
> at org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:239)
> at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:425)
>
> At first,I suspected it must be going wrong with the hive
> configuration.but it is normal when i separately run the hive cli to
> create table in hdfs,as follows:
>
> [root@localhost apache-hive-0.13.1-bin]# bin/hive --service cli
> 14/06/27 15:15:47 WARN conf.HiveConf: DEPRECATED:
> hive.metastore.ds.retry.* no longer has any effect. Use
> hive.hmshandler.retry.* instead
>
> Logging initialized using configuration in
> jar:file:/root/hive/apache-hive-0.13.1-bin/lib/hive-common-0.13.1.jar!/hive-log4j.properties
> hive> create database xxdd
>> ;
> OK
> Time taken: 1.053 seconds
> hive> use xxdd;
> OK
> Time taken: 0.056 seconds
> hive> create table aa (aa int);
> OK
> Time taken: 0.677 seconds
>
> Secondly,I view the hive script logging file dumping by which sqoop
> import command executing hive script.but there is nothing error or
> warning in it.as follows:
>
> 2014-06-27 15:27:37,058 INFO [main]: metastore.ObjectStore
> (ObjectStore.java:initialize(246)) - ObjectStore, initialize called
> 2014-06-27 15:27:41,670 WARN [main]: conf.HiveConf
> (HiveConf.java:initialize(1390)) - DEPRECATED: hive.metastore.ds.retry.*
> no longer has any effect. Use hive.hmshandler.retry.* instead
> 2014-06-27 15:27:41,694 INFO [main]: metastore.ObjectStore
> (ObjectStore.java:getPMF(315)) - Setting MetaStore object pin classes
> with
> hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"
> 2014-06-27 15:27:41,890 INFO [main]: metastore.MetaStoreDirectSql
> (MetaStoreDirectSql.java:<init>(110)) - MySQL check failed, assuming we
> are not on mysql: Lexical error at line 1, column 5. Encountered: "@"
> (64), after : "".
> 2014-06-27 15:27:46,220 INFO [main]: metastore.ObjectStore
> (ObjectStore.java:setConf(229)) - Initialized ObjectStore
> 2014-06-27 15:27:46,839 INFO [main]: metastore.HiveMetaStore
> (HiveMetaStore.java:createDefaultRoles(551)) - Added admin role in metastore
> 2014-06-27 15:27:46,842 INFO [main]: metastore.HiveMetaStore
> (HiveMetaStore.java:createDefaultRoles(560)) - Added public role in
> metastore
>
>
>
> Enclosing my computer enviroment:
> OS:
> Linux localhost 2.6.32-220.el6.x86_64 #1 SMP Tue Dec 6 19:48:22 GMT 2011
> x86_64 x86_64 x86_64 GNU/Linux
> java:
> java version "1.7.0_55"
> OpenJDK Runtime Environment (rhel-2.4.7.1.el6_5-x86_64 u55-b13)
> OpenJDK 64-Bit Server VM (build 24.51-b03, mixed mode)
> HADOOP:apache hadoop-1.2.1
> hive:apache-hive-0.13.1-bin
> sqoop:sqoop-1.4.4.bin__hadoop-0.20
>
>
> How to resolve the problem?
>
> THANKS IN ADVANCE!
>
>
>
Thank you very much!
Finally I  discovered the error caused by  the following:
hive.HiveImport: Exception in thread "main" java.lang.NoSuchMethodError: 
org.apache.thrift.EncodingUtils.setBit(BIZ)B

I replaced the libthrift-0.8.0.jar used in other components with the 
libthrift-0.9.0.jar used in hive,and everything goes ok.


Mime
View raw message