sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abraham Elmahrek <...@cloudera.com>
Subject Re: Sqoop 1.4.4 targeting Hadoop 0.20.203.0 Fails
Date Mon, 06 Oct 2014 18:46:02 GMT
Hey there,

A couple of things stand out:
1. you need to provide "hadoopversion" for your changes to take effect.
e.g. ant -Dhadoopversion=20 tar.
2. Why not use a 0.20 release from
http://www.apache.org/dyn/closer.cgi/sqoop/1.4.5 ?

-Abe

On Mon, Oct 6, 2014 at 11:35 AM, Josef Kafka <clojurian123@gmail.com> wrote:

> Hi,
>
> We are trying to find a version of Sqoop that runs against Hadoop
> 0.20.203.0. I discovered the following on StackOverflow:
>
> "Sqoop have currently two main branches. Sqoop 1 is older fully
> functional and mature project supporting Hadoop 0.20, 1.x, 0.23 and
> 2.0.x You can download the bits from here. Please make sure that you
> download file ending with "_hadoop-0.20", otherwise you will be
> getting weird exceptions."
>
>
>
> What we did is to download the 1.4.4 branch of Sqoop from here:
> https://github.com/apache/sqoop/tree/branch-1.4.4
>
>
>
> and change the build.xml file of Sqoop to target 0.20.203.0 before "ant
> mvn-install":
>
>
>   <!-- Set default Hadoop version if not set -->
>   <if>
>     <isset property="hadoopversion" />
>     <then>
>       <echo message="Use Hadoop ${hadoopversion}" />
>     </then>
>     <else>
>       <echo message="Use Hadoop 2.x by default" />
>       <property name="hadoopversion" value="*20*" />
>     </else>
>   </if>
>
>     <!-- Set variables per target hadoop version -->
>     <if>
>       <equals arg1="${hadoopversion}" arg2="*20*" />
>       <then>
>             <property name="hadoop.version" value="*0.20.203.0*" />
>             <property name="hbase.version" value="0.94.2" />
>             <property name="zookeeper.version" value="3.4.2" />
>             <property name="hadoop.version.full" value="0.20.203.0" />
>             <property name="hcatalog.version" value="0.11.0" />
>       </then>
>
>     <elseif>
>
>
>
>
> But when we try to run the job, we get the following error:
>
>
> Exception in thread "main" java.lang.IncompatibleClassChangeError: Found
> class org.apache.hadoop.mapreduce.JobContext, but interface was expected
>         at
>
> org.apache.sqoop.config.ConfigurationHelper.getJobNumMaps(ConfigurationHelper.java:53)
>         at
>
> com.cloudera.sqoop.config.ConfigurationHelper.getJobNumMaps(ConfigurationHelper.java:36)
>         at
>
> org.apache.sqoop.mapreduce.db.DataDrivenDBInputFormat.getSplits(DataDrivenDBInputFormat.java:121)
>         at
> org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:902)
>         at
> org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:919)
>         at
> org.apache.hadoop.mapred.JobClient.access$500(JobClient.java:170)
>         at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:838)
>         at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:791)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>         at
>
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
>         at
> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:791)
>         at org.apache.hadoop.mapreduce.Job.submit(Job.java:465)
>         at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:494)
>         at
>
> org.apache.sqoop.mapreduce.ImportJobBase.doSubmitJob(ImportJobBase.java:186)
>         at
> org.apache.sqoop.mapreduce.ImportJobBase.runJob(ImportJobBase.java:159)
>         at
> org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:239)
>         at
> org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:600)
>         at
> org.apache.sqoop.manager.OracleManager.importTable(OracleManager.java:384)
>         at
> org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:413)
>         at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:502)
>         at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>         at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
>         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
>         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
>         at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:53)
>         at
>
> org.troposphere.mapreduce.bft.mrjobs.SqoopHDFSImportJob.runJob(SqoopHDFSImportJob.java:58)
>         at
>
> org.troposphere.mapreduce.commons.AbstractRunnable.run(AbstractRunnable.java:29)
>         at
> org.troposphere.mapreduce.runner.JobRunner.run(JobRunner.java:62)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>         at
> org.troposphere.mapreduce.runner.JobRunner.main(JobRunner.java:41)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message