sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Abraham Elmahrek (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SQOOP-2229) Sqoop2: Kite connector should handle doubles
Date Thu, 02 Apr 2015 19:50:53 GMT

     [ https://issues.apache.org/jira/browse/SQOOP-2229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Abraham Elmahrek updated SQOOP-2229:
------------------------------------
    Fix Version/s:     (was: 1.99.7)
                   1.99.6

> Sqoop2: Kite connector should handle doubles
> --------------------------------------------
>
>                 Key: SQOOP-2229
>                 URL: https://issues.apache.org/jira/browse/SQOOP-2229
>             Project: Sqoop
>          Issue Type: Bug
>          Components: sqoop2-kite-connector
>            Reporter: Abraham Elmahrek
>            Assignee: Abraham Elmahrek
>             Fix For: 1.99.6
>
>         Attachments: SQOOP-2229.001.patch, SQOOP-2229.002.patch
>
>
> {code}
> Caused by: org.apache.avro.UnresolvedUnionException: Not in union ["double","null"]:
0.0
> 	at org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:561)
> 	at org.apache.avro.generic.GenericDatumWriter.resolveUnion(GenericDatumWriter.java:144)
> 	at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:71)
> 	at org.apache.avro.reflect.ReflectDatumWriter.write(ReflectDatumWriter.java:104)
> 	at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:106)
> 	at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:66)
> 	at org.apache.avro.reflect.ReflectDatumWriter.write(ReflectDatumWriter.java:104)
> 	at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:58)
> 	at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:257)
> 	... 12 more
> 2015-03-18 13:53:13,158 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running
child : org.apache.sqoop.common.SqoopException: MAPRED_EXEC_0018:Error occurs during loader
run
> 	at org.apache.sqoop.job.mr.SqoopOutputFormatLoadExecutor$ConsumerThread.run(SqoopOutputFormatLoadExecutor.java:261)
> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> 	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 	at java.lang.Thread.run(Thread.java:724)
> Caused by: org.kitesdk.data.DatasetRecordException: Failed to append record
> 	at org.kitesdk.data.spi.filesystem.AvroAppender.append(AvroAppender.java:74)
> 	at org.kitesdk.data.spi.filesystem.FileSystemWriter.write(FileSystemWriter.java:158)
> 	at org.apache.sqoop.connector.kite.KiteDatasetExecutor.writeRecord(KiteDatasetExecutor.java:85)
> 	at org.apache.sqoop.connector.kite.KiteLoader.load(KiteLoader.java:72)
> 	at org.apache.sqoop.connector.kite.KiteLoader.load(KiteLoader.java:36)
> 	at org.apache.sqoop.job.mr.SqoopOutputFormatLoadExecutor$ConsumerThread.run(SqoopOutputFormatLoadExecutor.java:250)
> 	... 6 more
> Caused by: org.apache.avro.file.DataFileWriter$AppendWriteException: org.apache.avro.UnresolvedUnionException:
Not in union ["double","null"]: 0.0
> 	at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:263)
> 	at org.kitesdk.data.spi.filesystem.AvroAppender.append(AvroAppender.java:72)
> 	... 11 more
> Caused by: org.apache.avro.UnresolvedUnionException: Not in union ["double","null"]:
0.0
> 	at org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:561)
> 	at org.apache.avro.generic.GenericDatumWriter.resolveUnion(GenericDatumWriter.java:144)
> 	at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:71)
> 	at org.apache.avro.reflect.ReflectDatumWriter.write(ReflectDatumWriter.java:104)
> 	at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:106)
> 	at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:66)
> 	at org.apache.avro.reflect.ReflectDatumWriter.write(ReflectDatumWriter.java:104)
> 	at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:58)
> 	at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:257)
> 	... 12 more
> {code}
> Right now Sqoop just makes every thing a double, when there are floats and doubles in
Kite. Sqoop should choose the data type correctly based on the byte size in Floating Point.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message