sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Silva João <joao.si...@altran.com>
Subject Re: kite-data-mapreduce missing?
Date Wed, 06 Apr 2016 09:08:47 GMT
Hi Debabrata,

I did that, although in directory /user/hdfs/sqoop/kite-data-mapreduce.jar.

In the logs, I see this jar being loaded but the result is the same.

João M. S. Silva

On 06/04/2016 01:45, Debabrata Pani wrote:
> Hi Joao,
> Should we not add this jar too oozie application lib subdirectory ?
> Generally the workflow.xml is in a folder on HDFS. We can have a lib
> subdirectory under the folder and add jars to it. Can we try doing that ?
>
> On Apr 5, 2016 9:55 PM, "Silva João" <joao.silva@altran.com
> <mailto:joao.silva@altran.com>> wrote:
>
>     I just realized that a file
>     /usr/hdp/2.4.0.0-169/sqoop/lib/kite-data-mapreduce-1.0.0.jar exists. I
>     copied it to HDFS and added it to the workflow.xml, e.g.:
>
>         <file>/user/hdfs/sqoop/kite-data-mapreduce.jar</file>
>
>     I see it being used in the logs, but it does not change the result.
>
>     João M. S. Silva
>
>     On 05/04/2016 16:47, Silva João wrote:
>      > Hi,
>      >
>      > I'm running an oozie coordinator to periodically run an oozie
>     workflow
>      > which runs an *incremental* sqoop import from Postgres to Parquet
>     files.
>      >
>      > The coordinator was running OK, when no new data was added to the
>      > origin. But after I inserted a new line in the Postgres DB,
>     oozie/sqoop
>      > failed with the following error:
>      >
>      > Failing Oozie Launcher, Main class
>      > [org.apache.oozie.action.hadoop.SqoopMain], main() threw exception,
>      > org/kitesdk/data/mapreduce/DatasetKeyOutputFormat
>      > java.lang.NoClassDefFoundError:
>      > org/kitesdk/data/mapreduce/DatasetKeyOutputFormat
>      > at
>      >
>     org.apache.sqoop.mapreduce.DataDrivenImportJob.getOutputFormatClass(DataDrivenImportJob.java:190)
>      >
>      > I've read that the jar for kite-data-mapreduce is missing? Is this
>      > correct? How do I install such a package?
>      >
>      > Thanks.
>      >
>

Mime
View raw message