sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amit Sela <am...@infolinks.com>
Subject Re: Programmatic (java) Sqoop import from MySQL to HBase Could not find hadoop core jar
Date Sun, 03 Nov 2013 23:18:09 GMT
It points to $HADOOP_HOME/bin/.. Meaning $HADOOP_HOME.
I know it looks stupid to write it that way but I wanted it to be a
complete match to the definition as running from command line.
I'll give a try the normal way.
On Nov 3, 2013 11:05 PM, "Jarek Jarcec Cecho" <jarcec@apache.org> wrote:

> Hi Amit,
> the parameter --hadoop-home should be pointing to $HADOOP_HOME, not to a
> $HADOOP_HOME/bin.
>
> Jarcec
>
> On Sun, Nov 03, 2013 at 03:06:38PM +0200, Amit Sela wrote:
> > I'm running Hadoop 1.0.4 (and HBAse 0.94.2) and the directory I specify
> for
> > --hadoop-home is: "$HADOOP_HOME/bin/..".
> > That's the exact definition in our system.
> >
> >
> > On Sat, Nov 2, 2013 at 3:29 AM, Jarek Jarcec Cecho <jarcec@apache.org
> >wrote:
> >
> > > Hi Amit,
> > > what Hadoop version are you using? Would you mind sharing with us
> listing
> > > of the directory that you specified in the --hadoop-home argument?
> > >
> > > Please note that Sqoop 1 do not have officially supported client API
> and
> > > thus directly using Sqoop.runTool() is not supported and you might run
> into
> > > a lot of issues such as this one.
> > >
> > > Jarcec
> > >
> > > On Thu, Oct 31, 2013 at 11:09:05AM +0200, Amit Sela wrote:
> > > > That's the stack trace for the exception I got:
> > > > Encountered IOException running import job: java.io.IOException:
> Could
> > > not
> > > > find hadoop core jar!
> > > > at
> > > >
> > >
> org.apache.sqoop.orm.CompilationManager.compile(CompilationManager.java:146)
> > > > at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:97)
> > > > at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:396)
> > > > at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:502)
> > > > at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
> > > > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> > > > at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
> > > > at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
> > > > at
> > > >
> > >
> com.infolinks.redmap.services.impl.utils.Sqooper.importMysqlToHBase(Sqooper.java:35)
> > > > at
> > > >
> > >
> com.infolinks.redmap.services.impl.submitters.SqoopSubmitter$1.call(SqoopSubmitter.java:79)
> > > > at
> > > >
> > >
> com.infolinks.redmap.services.impl.submitters.SqoopSubmitter$1.call(SqoopSubmitter.java:76)
> > > > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> > > > at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> > > > at
> > > >
> > >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
> > > > at
> > > >
> > >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
> > > > at
> > > >
> > >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> > > > at
> > > >
> > >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> > > > at java.lang.Thread.run(Thread.java:722)
> > > >
> > > > My infrastructure allows me to execute MR jobs and to access HBase
> so I
> > > > don't understand why executing Sqoop results with "Could not find
> hadoop
> > > > core jar!" unless Sqoop is actually looking for the jar to deploy it
> for
> > > > some reason...
> > > >
> > > >
> > > >
> > > > On Wed, Oct 30, 2013 at 11:14 PM, Jarek Jarcec Cecho <
> jarcec@apache.org
> > > >wrote:
> > > >
> > > > > Hi Amit,
> > > > > would you mind sharing the entire stack trace that you are getting?
> > > > >
> > > > > Please note that Sqoop 1 do not have officially supported client
> API
> > > and
> > > > > thus directly using Sqoop.runTool() is not supported and you might
> run
> > > into
> > > > > a lot of issues such as this one.
> > > > >
> > > > > Jarcec
> > > > >
> > > > > On Sun, Oct 27, 2013 at 05:52:30PM +0200, Amit Sela wrote:
> > > > > > Which jar ?
> > > > > >
> > > > > > Hadoop jars are in "HADOOP_HOME/lib" and also sqoop jar.
> > > > > > I'm trying to execute the import from an OSGi running client,
> with a
> > > > > maven
> > > > > > dependency and embedding on sqoop.
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Sun, Oct 27, 2013 at 5:15 PM, Krishnan K <
> kkrishnan88@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > > > Can you try adding the jar to HADOOP_CLASSPATH / CLASSPATH
?
> > > > > > >
> > > > > > >
> > > > > > > On Sun, Oct 27, 2013 at 7:34 PM, Amit Sela <
> amits@infolinks.com>
> > > > > wrote:
> > > > > > >
> > > > > > >> Hi all,
> > > > > > >>
> > > > > > >> I'm new to Sqoop and I'm trying to execute (from Java
code) a
> > > sqoop
> > > > > > >> import from MySQL to HBase.
> > > > > > >>
> > > > > > >> I'm using Sqoop.runTool() with a String[] containing
all the
> > > necessary
> > > > > > >> parameters, I even added "--hadoop-home" with the value
of my
> > > > > $HADOOP_HOME.
> > > > > > >>
> > > > > > >> Anyone had this problem before ? any ideas ?
> > > > > > >>
> > > > > > >> Thanks,
> > > > > > >>
> > > > > > >> Amit.
> > > > > > >>
> > > > > > >
> > > > > > >
> > > > >
> > >
>

Mime
View raw message