sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abraham Elmahrek <...@cloudera.com>
Subject Re: using Sqoop 1.4.4 with Hadoop 2.4.0
Date Sun, 15 Jun 2014 21:54:23 GMT
You can add --verbose to the beginning of your command (right after the
word "export") and you'll get some more info. Could you copy/paste the
output of your sqoop command with this flag? There should also be logs in
the MapReduce job/task logs. In this case, I think the application and
container logs may have more information. They are usually located under
'/var/log/hadoop-yarn/'. Have you considered installing these components
using a vendor package or BigTop?


On Sun, Jun 15, 2014 at 12:42 AM, Sebastian Gäde <s116102@hft-leipzig.de>
wrote:

> Hi,
>
> Thanks Abe.
>
> Yes, Sqoop and Hadoop were downloaded from a tarball.
>
> I set HADOOP_COMMON_HOME to /opt/hadoop-2.4.0 and HADOOP_MAPRED_HOME to
> /opt/hadoop-2.4.0/share/hadoop/yarn using the conf/sqoop-env.sh. I also
> tried /opt/hadoop-2.4.0/share/hadoop/mapreduce for HADOOP_MAPRED_HOME as
> I did not know which JAR I need. However, the error was the same.
>
> Is there a way to enable DEBUG messages? This sometimes helped me in the
> passed with other Hadoop issues.
>
> Cheers
> Seb.
>
> Am 14.06.2014 06:37, schrieb Abraham Elmahrek:
>
>> Hey there,
>>
>> The exception is indicative of a missing jar from the class path. It
>> seems like the hadoop-mapreduce-client-core.jar file is missing. Have
>> you set HADOOP_COMMON_HOME and HADOOP_MAPRED_HOME?
>>
>> How did you install Sqoop and Hadoop? If you've downloaded a tarball,
>> you may need to set the environment variables specified above.
>>
>> For more info see
>> http://sqoop.apache.org/docs/1.4.4/SqoopUserGuide.html#_
>> controlling_the_hadoop_installation.
>>
>> -Abe
>>
>>
>> On Fri, Jun 13, 2014 at 4:15 AM, Sebastian Gäde <s116102@hft-leipzig.de
>> <mailto:s116102@hft-leipzig.de>> wrote:
>>
>>     Hi,
>>
>>     I’d like to export MR result (text) files from HDFS to MySQL. My
>>     cluster is
>>     running Hadoop 2.4.0.
>>
>>     When I’m running
>>     ./sqoop export –connect jdbc:mysql://mysqlserver/sqoop –username user
>>     –password pass –table test1 –export-dir /output
>>     I get the below exception. I assume, sqoop is using the old MR API?
>>     Can I
>>     change this somehow to use the new API, then I'd be able to point
>>     $HADOOP_MAPRED_HOME to the right position?!
>>
>>     14/06/13 13:06:44 WARN tool.BaseSqoopTool: Setting your password on
>> the
>>     command-line is insecure. Consider using -P instead.
>>     14/06/13 13:06:44 INFO manager.MySQLManager: Preparing to use a MySQL
>>     streaming resultset.
>>     14/06/13 13:06:44 INFO tool.CodeGenTool: Beginning code generation
>>     14/06/13 13:06:50 INFO manager.SqlManager: Executing SQL statement:
>>     SELECT
>>     t.* FROM `test1` AS t LIMIT 1
>>     14/06/13 13:06:50 INFO manager.SqlManager: Executing SQL statement:
>>     SELECT
>>     t.* FROM `test1` AS t LIMIT 1
>>     14/06/13 13:06:50 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is
>>     /opt/hadoop-2.4.0/share/hadoop/yarn
>>     Exception in thread "main" java.lang.NoClassDefFoundError:
>>     org/apache/hadoop/mapred/JobConf
>>              at
>>     org.apache.sqoop.orm.CompilationManager.findHadoopCoreJar(
>> CompilationManager
>>     .java:109)
>>              at
>>     org.apache.sqoop.orm.CompilationManager.compile(
>> CompilationManager.java:137)
>>              at
>>     org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:97)
>>              at
>>     org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:64)
>>              at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:100)
>>              at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
>>              at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>              at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
>>              at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
>>              at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
>>              at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
>>
>>     Thanks very much!
>>     Sebastian
>>
>>
>>
>>

Mime
View raw message