sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Frédéric ESCANDELL (JIRA) <j...@apache.org>
Subject [jira] [Updated] (SQOOP-3339) Netezza export doesn't work on ORC tables
Date Tue, 26 Jun 2018 17:39:00 GMT

     [ https://issues.apache.org/jira/browse/SQOOP-3339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Frédéric ESCANDELL updated SQOOP-3339:
--------------------------------------
    Description: 
While executing sqoop export on a ORC table, the exception followed is launched : 
{code:java}
Error: java.lang.ClassCastException: org.apache.hadoop.io.NullWritable cannot be cast to org.apache.hadoop.io.LongWritable
        at org.apache.sqoop.mapreduce.db.netezza.NetezzaExternalTableHCatExportMapper.map(NetezzaExternalTableHCatExportMapper.java:34)
        at org.apache.sqoop.mapreduce.db.netezza.NetezzaExternalTableExportMapper.run(NetezzaExternalTableExportMapper.java:233)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
        at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:170)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:164)
{code}
If the exported is stored as TextFile, the mapper class receive LongWritable as key but if
it's an ORC table, the mapper class receive NullWritable.

The patch in attachment propose to modify the signature of map function :
{code:java}
public void map(LongWritable key, HCatRecord hcr, Context context){code}
to
{code:java}
public void map(Object key, HCatRecord hcr, Context context){code}
 

 

 

 

  was:
While executing sqoop export on a ORC table, the exception followed is launched : 

 
{code:java}
Error: java.lang.ClassCastException: org.apache.hadoop.io.NullWritable cannot be cast to org.apache.hadoop.io.LongWritable
        at org.apache.sqoop.mapreduce.db.netezza.NetezzaExternalTableHCatExportMapper.map(NetezzaExternalTableHCatExportMapper.java:34)
        at org.apache.sqoop.mapreduce.db.netezza.NetezzaExternalTableExportMapper.run(NetezzaExternalTableExportMapper.java:233)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
        at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:170)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:164)
{code}
If the exported is stored as TextFile, the mapper class receive LongWritable as key but if
it's an ORC table, the mapper class receive NullWritable.

The patch in attachment propose to modify the signature of map function :

 
{code:java}
public void map(LongWritable key, HCatRecord hcr, Context context){code}
 

to 

 
{code:java}
public void map(Object key, HCatRecord hcr, Context context){code}
 

 

 

 


> Netezza export doesn't work on ORC tables
> -----------------------------------------
>
>                 Key: SQOOP-3339
>                 URL: https://issues.apache.org/jira/browse/SQOOP-3339
>             Project: Sqoop
>          Issue Type: Bug
>    Affects Versions: 1.4.7
>            Reporter: Frédéric ESCANDELL
>            Priority: Blocker
>
> While executing sqoop export on a ORC table, the exception followed is launched : 
> {code:java}
> Error: java.lang.ClassCastException: org.apache.hadoop.io.NullWritable cannot be cast
to org.apache.hadoop.io.LongWritable
>         at org.apache.sqoop.mapreduce.db.netezza.NetezzaExternalTableHCatExportMapper.map(NetezzaExternalTableHCatExportMapper.java:34)
>         at org.apache.sqoop.mapreduce.db.netezza.NetezzaExternalTableExportMapper.run(NetezzaExternalTableExportMapper.java:233)
>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:170)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:422)
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:164)
> {code}
> If the exported is stored as TextFile, the mapper class receive LongWritable as key but
if it's an ORC table, the mapper class receive NullWritable.
> The patch in attachment propose to modify the signature of map function :
> {code:java}
> public void map(LongWritable key, HCatRecord hcr, Context context){code}
> to
> {code:java}
> public void map(Object key, HCatRecord hcr, Context context){code}
>  
>  
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message