sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sebastian Gäde <s116...@hft-leipzig.de>
Subject Re: using Sqoop 1.4.4 with Hadoop 2.4.0
Date Sun, 15 Jun 2014 07:42:39 GMT
Hi,

Thanks Abe.

Yes, Sqoop and Hadoop were downloaded from a tarball.

I set HADOOP_COMMON_HOME to /opt/hadoop-2.4.0 and HADOOP_MAPRED_HOME to 
/opt/hadoop-2.4.0/share/hadoop/yarn using the conf/sqoop-env.sh. I also 
tried /opt/hadoop-2.4.0/share/hadoop/mapreduce for HADOOP_MAPRED_HOME as 
I did not know which JAR I need. However, the error was the same.

Is there a way to enable DEBUG messages? This sometimes helped me in the 
passed with other Hadoop issues.

Cheers
Seb.

Am 14.06.2014 06:37, schrieb Abraham Elmahrek:
> Hey there,
>
> The exception is indicative of a missing jar from the class path. It
> seems like the hadoop-mapreduce-client-core.jar file is missing. Have
> you set HADOOP_COMMON_HOME and HADOOP_MAPRED_HOME?
>
> How did you install Sqoop and Hadoop? If you've downloaded a tarball,
> you may need to set the environment variables specified above.
>
> For more info see
> http://sqoop.apache.org/docs/1.4.4/SqoopUserGuide.html#_controlling_the_hadoop_installation.
>
> -Abe
>
>
> On Fri, Jun 13, 2014 at 4:15 AM, Sebastian Gäde <s116102@hft-leipzig.de
> <mailto:s116102@hft-leipzig.de>> wrote:
>
>     Hi,
>
>     I’d like to export MR result (text) files from HDFS to MySQL. My
>     cluster is
>     running Hadoop 2.4.0.
>
>     When I’m running
>     ./sqoop export –connect jdbc:mysql://mysqlserver/sqoop –username user
>     –password pass –table test1 –export-dir /output
>     I get the below exception. I assume, sqoop is using the old MR API?
>     Can I
>     change this somehow to use the new API, then I'd be able to point
>     $HADOOP_MAPRED_HOME to the right position?!
>
>     14/06/13 13:06:44 WARN tool.BaseSqoopTool: Setting your password on the
>     command-line is insecure. Consider using -P instead.
>     14/06/13 13:06:44 INFO manager.MySQLManager: Preparing to use a MySQL
>     streaming resultset.
>     14/06/13 13:06:44 INFO tool.CodeGenTool: Beginning code generation
>     14/06/13 13:06:50 INFO manager.SqlManager: Executing SQL statement:
>     SELECT
>     t.* FROM `test1` AS t LIMIT 1
>     14/06/13 13:06:50 INFO manager.SqlManager: Executing SQL statement:
>     SELECT
>     t.* FROM `test1` AS t LIMIT 1
>     14/06/13 13:06:50 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is
>     /opt/hadoop-2.4.0/share/hadoop/yarn
>     Exception in thread "main" java.lang.NoClassDefFoundError:
>     org/apache/hadoop/mapred/JobConf
>              at
>     org.apache.sqoop.orm.CompilationManager.findHadoopCoreJar(CompilationManager
>     .java:109)
>              at
>     org.apache.sqoop.orm.CompilationManager.compile(CompilationManager.java:137)
>              at
>     org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:97)
>              at
>     org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:64)
>              at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:100)
>              at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
>              at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>              at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
>              at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
>              at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
>              at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
>
>     Thanks very much!
>     Sebastian
>
>
>

Mime
View raw message