sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jarek Jarcec Cecho (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-2489) Sqoop2: Hive with Parquet in Kite Connector
Date Tue, 01 Sep 2015 11:40:46 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-2489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14725244#comment-14725244
] 

Jarek Jarcec Cecho commented on SQOOP-2489:
-------------------------------------------

Can you add integration tests that will try Parquet import as part of the patch [~sleefd]?
I'm afraid that without it we can always regress here. I think that adding the test shouldn't
be that difficult as we already have test importing to CSV [here|https://github.com/apache/sqoop/blob/sqoop2/test/src/test/java/org/apache/sqoop/integration/connector/kite/FromRDBMSToKiteTest.java].
Perhaps let's just make it run with data provider and try all various file formats?

> Sqoop2: Hive with Parquet in Kite Connector
> -------------------------------------------
>
>                 Key: SQOOP-2489
>                 URL: https://issues.apache.org/jira/browse/SQOOP-2489
>             Project: Sqoop
>          Issue Type: Bug
>          Components: sqoop2-kite-connector
>    Affects Versions: 1.99.6
>            Reporter: Abraham Elmahrek
>             Fix For: 1.99.7
>
>         Attachments: 0001-fix-jira-SQOOP-2489.patch
>
>
> {code}
> java.lang.NoClassDefFoundError: org/apache/hadoop/hive/ql/io/HiveOutputFormat
> 	at java.lang.ClassLoader.defineClass1(Native Method)
> 	at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
> 	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> 	at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
> 	at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
> 	at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
> 	at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> 	at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> 	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> 	at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> 	at java.lang.Class.forName0(Native Method)
> 	at java.lang.Class.forName(Class.java:190)
> 	at org.kitesdk.data.spi.hive.HiveUtils.getHiveParquetOutputFormat(HiveUtils.java:446)
> 	at org.kitesdk.data.spi.hive.HiveUtils.<clinit>(HiveUtils.java:91)
> 	at org.kitesdk.data.spi.hive.HiveManagedMetadataProvider.create(HiveManagedMetadataProvider.java:83)
> 	at org.kitesdk.data.spi.hive.HiveManagedDatasetRepository.create(HiveManagedDatasetRepository.java:77)
> 	at org.kitesdk.data.Datasets.create(Datasets.java:239)
> 	at org.kitesdk.data.Datasets.create(Datasets.java:307)
> 	at org.kitesdk.data.Datasets.create(Datasets.java:335)
> 	at org.apache.sqoop.connector.kite.KiteDatasetExecutor.createDataset(KiteDatasetExecutor.java:70)
> 	at org.apache.sqoop.connector.kite.KiteLoader.getExecutor(KiteLoader.java:52)
> 	at org.apache.sqoop.connector.kite.KiteLoader.load(KiteLoader.java:62)
> 	at org.apache.sqoop.connector.kite.KiteLoader.load(KiteLoader.java:36)
> 	at org.apache.sqoop.job.mr.SqoopOutputFormatLoadExecutor$ConsumerThread.run(SqoopOutputFormatLoadExecutor.java:250)
> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 	at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.ql.io.HiveOutputFormat
> 	at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> 	at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> 	at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> 	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> 	at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> 	... 31 more
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message