sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sam liu <liuqiyun2...@gmail.com>
Subject Re: Why sqoop picks up hsqldb driver jar when connect to mysql db?
Date Mon, 19 Aug 2013 09:27:24 GMT
Hi Jarek, I am using Hadoop-2.0.4-alpha.


2013/8/12 Jarek Jarcec Cecho <jarcec@apache.org>

> Hi Sam,
> Sqoop uses HSQLDB version 1.8, not 2.0. There was period of time when
> Hadoop was forcing HSQLDB 2.0 into downstream component classpath, but that
> was resolved. Please check out SQOOP-571 for more background.
>
> I've tried both 1.4.3 and 1.4.3 releases on BigTop 0.6 (that is based on
> Hadoop 2.0.5-alpha) and both were working for me without any issues, so I'm
> wondering what Hadoop version are you using?
>
> Jarcec
>
> On Wed, Aug 07, 2013 at 02:56:47PM +0800, sam liu wrote:
> > Hi Abe,
> >
> > I tried with --driver param, but it does not work for me, and returned
> the
> > same exception.
> >
> > My command is 'bin/sqoop import --driver com.mysql.jdbc.Driver --connect
> > jdbc:mysql://hostname:3306/test  --username user --password pwd --table
> > testtbl -m 1 --target-dir /tmp/test6'.
> >
> > Thanks!
> >
> >
> > 2013/8/7 Abraham Elmahrek <abe@cloudera.com>
> >
> > > Sam,
> > >
> > > Did you try the --driver parameter? It allows you to choose the JDBC
> > > driver class.
> > >
> > > -Abe
> > >
> > >
> > > On Tue, Aug 6, 2013 at 7:54 PM, sam liu <liuqiyun2011@gmail.com>
> wrote:
> > >
> > >> I think this is a defect of Sqoop, and opened a jira for it:
> > >> https://issues.apache.org/jira/browse/SQOOP-1164. I set the priority
> as
> > >> Minor, because it has the workaround: only leave the target driver
> jar file
> > >> in the sqoop/lib folder, but remove other driver jar files from the
> folder.
> > >> However, it should be mentioned that when we have many dirver jar
> files
> > >> under sqoop/lib folder for multiple kinds of databases, or a script
> will
> > >> connect to different databases, this workaround can not work well.
> > >>
> > >> Thanks!
> > >>
> > >>
> > >> 2013/8/6 sam liu <liuqiyun2011@gmail.com>
> > >>
> > >>> Hi Experts,
> > >>>
> > >>> Under SQOOP/lib, I have both hsqldb driver jar file and mysql driver
> jar
> > >>> file:
> > >>> - lib/hsqldb-1.8.0.10.jar
> > >>> - lib/mysql-connector-java-5.1.20-bin.jar
> > >>>
> > >>> But, when I try to connect to mysql db using sqoop import tool, sqoop
> > >>> will pick up hsqldb driver jar and returned following exception.
> Even if I
> > >>> add '--connection-manager org.apache.sqoop.manager.MySQLManager'
> into the
> > >>> command, this issue still exists. But, after I removed the hsqldb
> jar file
> > >>> from lib folder, this issue does not exist any more.
> > >>>
> > >>> Why? What's the correct method to invoke import tool to avoid such
> > >>> issue? Thanks!
> > >>>
> > >>> > bin/sqoop import --connect jdbc:mysql://hostname:3306/test
> > >>> --connection-manager org.apache.sqoop.manager.MySQLManager
> --username user
> > >>> --password pwd --table testtbl -m 1 --target-dir /tmp/test5
> > >>> 13/08/05 18:44:03 WARN tool.BaseSqoopTool: Setting your password on
> the
> > >>> command-line is insecure. Consider using -P instead.
> > >>> 13/08/05 18:44:03 INFO manager.MySQLManager: Preparing to use a MySQL
> > >>> streaming resultset.
> > >>> 13/08/05 18:44:03 INFO tool.CodeGenTool: Beginning code generation
> > >>> Exception in thread "main" java.lang.NoSuchMethodError:
> > >>>
> org/hsqldb/DatabaseURL.parseURL(Ljava/lang/String;ZZ)Lorg/hsqldb/persist/HsqlProperties;
> > >>>         at org.hsqldb.jdbc.JDBCDriver.getConnection(Unknown Source)
> > >>>         at org.hsqldb.jdbc.JDBCDriver.connect(Unknown Source)
> > >>>         at
> java.sql.DriverManager.getConnection(DriverManager.java:402)
> > >>>         at
> java.sql.DriverManager.getConnection(DriverManager.java:450)
> > >>>         at
> > >>>
> org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:745)
> > >>>         at
> > >>>
> org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
> > >>>         at
> > >>> org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:605)
> > >>>         at
> > >>> org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:628)
> > >>>         at
> > >>>
> org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:235)
> > >>>         at
> > >>>
> org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:219)
> > >>>         at
> > >>>
> org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:283)
> > >>>         at
> > >>>
> org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1255)
> > >>>         at
> > >>> org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1072)
> > >>>         at
> > >>> org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:82)
> > >>>         at
> > >>> org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:390)
> > >>>         at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476)
> > >>>         at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
> > >>>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> > >>>         at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
> > >>>         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
> > >>>         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
> > >>>         at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
> > >>>
> > >>> --
> > >>>
> > >>> Sam Liu
> > >>>
> > >>
> > >>
> > >>
> > >> --
> > >>
> > >> Sam Liu
> > >>
> > >
> > >
> >
> >
> > --
> >
> > Sam Liu
>



-- 

Sam Liu

Mime
View raw message