sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Souvik Bose (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-1334) Sqoop Hcatalog export failed with exception of Failed to load foster storage handler during map reduce
Date Fri, 06 Jun 2014 06:28:01 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14019615#comment-14019615
] 

Souvik Bose commented on SQOOP-1334:
------------------------------------

How can I work on the standard file format. I haven't mentioned any input or output format
explicitly. Also the files are simple JSON files residing on hdfs on which the query is build.
Also I have included all the homes and classpath explicitly for hcat before running the qeury.

Please note that my installation is pretty much standard out of the box cdh4.4 installed using
cloudera manager on a 4 node AWS EC2 cluster. So I am wondering what can go wrong.
Even if hcatalog doesn't work at all, Do you any ways for sqoop to handle direct json files
like :

{"hardware_status":"0000","software_version":"1","original_body":"663474007C3840E8","looqiId":"200","longitude":"51.44944444444444","latitude":"0.5205555555555557","battery_voltage":"29","time_stamp":"1400780444000","original_filename":"/home/ktspool/200_22052014194044.GPS"}

and export it into mysql with schema : 
CREATE TABLE IF NOT EXISTS `bd_sqoop1` (
  `time_stamp`
  `hardware_status` varchar(250) DEFAULT NULL,
  `original_body` varchar(250) DEFAULT NULL,
  `software_version` varchar(250) DEFAULT NULL,
  `looqiId` varchar(250) DEFAULT NULL,
  `longitude` varchar(250) DEFAULT NULL,
  `battery_voltage` varchar(250) DEFAULT NULL,
  `latitude` varchar(250) DEFAULT NULL,
  `original_filename` varchar(250) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;

I have the hive table perfectly working, so through may be the sqoop hcatalog export which
comes with cdh 4.4 is the best option for me. But it seems like it has got some problem by
itself.

> Sqoop Hcatalog export failed with exception of Failed to load foster storage handler
during map reduce
> ------------------------------------------------------------------------------------------------------
>
>                 Key: SQOOP-1334
>                 URL: https://issues.apache.org/jira/browse/SQOOP-1334
>             Project: Sqoop
>          Issue Type: Bug
>          Components: hive-integration
>    Affects Versions: 1.4.3
>         Environment: CDH4.4.0 with cloudera manager.
>            Reporter: Souvik Bose
>            Priority: Blocker
>              Labels: hcatalog, sqoop, storage-handler
>         Attachments: Sqoop_Hcatalog_Export_Log_verbose.txt
>
>
> I tried to export a hive external table to mysql using sqoop hcatalog export command.
> sqoop export --connect jdbc:mysql://example.com:3306/demo1 --username demo-password demo1234
bd_sqoop1 --hcatalog-table test_data
> I have set the HCAT_HOME, HADOOP_CLASSPATH, HIVE_HOME and paths of Jars. Now the job
is progressing upto the map reduce stage, but stops at map 0% reduce 0% with an exception
as below:
> 14/06/04 13:27:21 INFO mapred.JobClient: Task Id : attempt_201406021302_0043_m_000002_1,
Status : FAILED
> java.io.IOException: Error initializing HCat record reader
>         at org.apache.sqoop.mapreduce.hcat.SqoopHCatRecordReader.initNextRecordReader(SqoopHCatRecordReader.java:146)
>         at org.apache.sqoop.mapreduce.hcat.SqoopHCatRecordReader.<init>(SqoopHCatRecordReader.java:57)
>         at org.apache.sqoop.mapreduce.hcat.SqoopHCatExportFormat.createRecordReader(SqoopHCatExportFormat.java:128)
>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:644)
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
>         at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:396)
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
>         at org.apache.hadoop.mapred.Child.main(Child.java:262)
> Caused by: java.io.IOException: Failed to load foster storage handler
>         at org.apache.hcatalog.common.HCatUtil.getStorageHandler(HCatUtil.java:416)
>         at org.apache.hcatalog.common.HCatUtil.getStorageHandl



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message