sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "VISHNU S NAIR (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SQOOP-2846) Sqoop Export with update-key failing for avro data file
Date Thu, 03 Mar 2016 05:54:18 GMT

     [ https://issues.apache.org/jira/browse/SQOOP-2846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

VISHNU S NAIR updated SQOOP-2846:
---------------------------------
    Attachment: 0001-SQOOP-2846.patch

After analyzing the issue, it is found that export with update key is
not implemneted properly in case of avro and parqute file formats.

In order to correct the issue [SQOOP-2846], export with update option is
implemented for the mentioned file formats. For this appropriate
mapper, input format and output format classes were used in the
implementation.

Updated test cases were executed successfully and the same is also
incorporated in the patch.

> Sqoop Export with update-key failing for avro data file
> -------------------------------------------------------
>
>                 Key: SQOOP-2846
>                 URL: https://issues.apache.org/jira/browse/SQOOP-2846
>             Project: Sqoop
>          Issue Type: Bug
>          Components: connectors/oracle
>    Affects Versions: 1.4.6
>            Reporter: Abhinav Meghmala
>            Assignee: VISHNU S NAIR
>             Fix For: 1.4.6
>
>         Attachments: 0001-SQOOP-2846.patch
>
>
> Hi, I am trying to export an AVRO data file from HDFS to Oracle database (11gR2) using
an update-key in sqoop command Ex: 
> sqoop export \ 
> --connect jdbc:oracle:thin:@X.X.X.X:1521/xe \ 
> --username ***** \ 
> --password ***** \ 
> --table XXX \ 
> --update-key <primary-key-column-name> \ 
> --update-mode allowinsert \ 
> --export-dir <hdfs-path> 
> However, the same export is successful without using the "update-key" in sqoop command.

> Failure Logs - 
> Error: java.io.IOException: Can't export data, please check failed map task logs 
> at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:112) 
> at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:39) 
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145) 
> at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64) 
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) 
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) 
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) 
> at java.security.AccessController.doPrivileged(Native Method) 
> at javax.security.auth.Subject.doAs(Subject.java:415) 
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)

> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) 
> Caused by: java.lang.RuntimeException: Can't parse input data: 'Objavro.schema�'

> at EMP.__loadFromFields(EMP.java:526) 
> at EMP.parse(EMP.java:439) 
> at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:83) 
> ... 10 more 
> Caused by: java.lang.NumberFormatException 
> at java.math.BigDecimal.<init>(BigDecimal.java:470) 
> at java.math.BigDecimal.<init>(BigDecimal.java:739) 
> at EMP.__loadFromFields(EMP.java:488) 
> ... 12 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message