sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Souvik Bose (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-1334) Sqoop Hcatalog export failed with exception of Failed to load foster storage handler during map reduce
Date Tue, 10 Jun 2014 15:18:02 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14026553#comment-14026553

Souvik Bose commented on SQOOP-1334:

As you can see above the input format is org.apache.hadoop.mapred.TextInputFormat
and output format is org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat. I was digging
through the logs in more details, and one thing I noticed which is unnatural if we compare
that with other logs. 

14/06/05 08:45:24 INFO mapred.FileInputFormat: Total input paths to process : 35244
14/06/05 08:45:43 DEBUG hcat.SqoopHCatExportFormat: Expected split count 4
14/06/05 08:45:43 DEBUG hcat.SqoopHCatExportFormat: HCatInputFormat provided split count 35244

In all of the successful cases I see that all the counts match upto 4. But those are pretty
obvious as the hcat imported tables using sqoops have 4 hdfs files generates as part0000-m
like that. But this is a case of external table which is referencing to a hdfs location with
lots of files.

> Sqoop Hcatalog export failed with exception of Failed to load foster storage handler
during map reduce
> ------------------------------------------------------------------------------------------------------
>                 Key: SQOOP-1334
>                 URL: https://issues.apache.org/jira/browse/SQOOP-1334
>             Project: Sqoop
>          Issue Type: Bug
>          Components: hive-integration
>    Affects Versions: 1.4.3
>         Environment: CDH4.4.0 with cloudera manager.
>            Reporter: Souvik Bose
>            Priority: Blocker
>              Labels: hcatalog, sqoop, storage-handler
>         Attachments: Sqoop_Hcatalog_Export_Log_verbose.txt
> I tried to export a hive external table to mysql using sqoop hcatalog export command.
> sqoop export --connect jdbc:mysql://example.com:3306/demo1 --username demo-password demo1234
bd_sqoop1 --hcatalog-table test_data
> I have set the HCAT_HOME, HADOOP_CLASSPATH, HIVE_HOME and paths of Jars. Now the job
is progressing upto the map reduce stage, but stops at map 0% reduce 0% with an exception
as below:
> 14/06/04 13:27:21 INFO mapred.JobClient: Task Id : attempt_201406021302_0043_m_000002_1,
Status : FAILED
> java.io.IOException: Error initializing HCat record reader
>         at org.apache.sqoop.mapreduce.hcat.SqoopHCatRecordReader.initNextRecordReader(SqoopHCatRecordReader.java:146)
>         at org.apache.sqoop.mapreduce.hcat.SqoopHCatRecordReader.<init>(SqoopHCatRecordReader.java:57)
>         at org.apache.sqoop.mapreduce.hcat.SqoopHCatExportFormat.createRecordReader(SqoopHCatExportFormat.java:128)
>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:644)
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
>         at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:396)
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
>         at org.apache.hadoop.mapred.Child.main(Child.java:262)
> Caused by: java.io.IOException: Failed to load foster storage handler
>         at org.apache.hcatalog.common.HCatUtil.getStorageHandler(HCatUtil.java:416)
>         at org.apache.hcatalog.common.HCatUtil.getStorageHandl

This message was sent by Atlassian JIRA

View raw message