sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "VISHNU S NAIR (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-2846) Sqoop Export with update-key failing for avro data file
Date Tue, 01 Mar 2016 10:30:18 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-2846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173552#comment-15173552
] 

VISHNU S NAIR commented on SQOOP-2846:
--------------------------------------

After analyzing the issue, it is found that export with update key is not implemneted properly
in case of avro and parqute file formats. 

In order to correct the issue [SQOOP-2846], export with update option is implemented for the
mentioned file formats. For this appropriate
mapper, input format and output format classes were used in the implementation.

Updated test cases were executed successfully and the same is also incorporated in the patch.

> Sqoop Export with update-key failing for avro data file
> -------------------------------------------------------
>
>                 Key: SQOOP-2846
>                 URL: https://issues.apache.org/jira/browse/SQOOP-2846
>             Project: Sqoop
>          Issue Type: Bug
>          Components: connectors/oracle
>    Affects Versions: 1.4.6
>            Reporter: Abhinav Meghmala
>            Assignee: VISHNU S NAIR
>
> Hi, I am trying to export an AVRO data file from HDFS to Oracle database (11gR2) using
an update-key in sqoop command Ex: 
> sqoop export \ 
> --connect jdbc:oracle:thin:@X.X.X.X:1521/xe \ 
> --username ***** \ 
> --password ***** \ 
> --table XXX \ 
> --update-key <primary-key-column-name> \ 
> --update-mode allowinsert \ 
> --export-dir <hdfs-path> 
> However, the same export is successful without using the "update-key" in sqoop command.

> Failure Logs - 
> Error: java.io.IOException: Can't export data, please check failed map task logs 
> at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:112) 
> at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:39) 
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145) 
> at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64) 
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) 
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) 
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) 
> at java.security.AccessController.doPrivileged(Native Method) 
> at javax.security.auth.Subject.doAs(Subject.java:415) 
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)

> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) 
> Caused by: java.lang.RuntimeException: Can't parse input data: 'Objavro.schema�'

> at EMP.__loadFromFields(EMP.java:526) 
> at EMP.parse(EMP.java:439) 
> at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:83) 
> ... 10 more 
> Caused by: java.lang.NumberFormatException 
> at java.math.BigDecimal.<init>(BigDecimal.java:470) 
> at java.math.BigDecimal.<init>(BigDecimal.java:739) 
> at EMP.__loadFromFields(EMP.java:488) 
> ... 12 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message