sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cheolsoo Park (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (SQOOP-524) hsqldb jar update regression
Date Fri, 27 Jul 2012 21:40:33 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13424166#comment-13424166
] 

Cheolsoo Park edited comment on SQOOP-524 at 7/27/12 9:39 PM:
--------------------------------------------------------------

Hi Abhijeet, thanks for your patch!

{quote}
two hsqldb JARs of different versions are getting copied in SQOOP_CLASSPATH
{quote}

To answer your question, it is a bug too. hsqldb-1.8.0.10.jar is being pulled by some dependency
library, so we must exclude it in ivy.xml. In fact, I excluded it when hadoopversion=23, but
it should be excluded globally regardless of the hadoopversion. The good thing is that hqsldb-2.2.x
has a different org attribute from hsqldb-1.8.x, so excluding hsqldb-1.8.x does not prevent
us from downloading hsqldb-2.2.x.

Can you please add the following change to your patch? This fixes the problem.

{code}
--- ivy.xml	(revision 1366535)
+++ ivy.xml	(working copy)
@@ -74,7 +74,6 @@
       rev="${hadoop.version}" conf="hadoop23->default">
       <artifact name="hadoop-common" type="jar" />
       <artifact name="hadoop-common" type="jar" m:classifier="tests"/>
-      <exclude org="hsqldb" module="hsqldb"/>
     </dependency>
     <dependency org="org.apache.hadoop" name="hadoop-hdfs"
       rev="${hadoop.version}" conf="hadoop23->default">
@@ -150,6 +149,7 @@
       <exclude org="com.cloudera.cdh" module="zookeeper-ant" />
     </dependency>
 
+    <exclude org="hsqldb" module="hsqldb"/>
     <exclude org="org.apache.hadoop" module="avro"/>
     <exclude org="commons-daemon" module="commons-daemon" />
{code}

                
      was (Author: cheolsoo):
    Hi Abhijeet, thanks for your patch!

{quote}
two hsqldb JARs of different versions are getting copied in SQOOP_CLASSPATH
{quote}

To answer your question, it is a bug too. hsqldb-1.8.0.10.jar must be pulled by some dependency
library, so we must exclude it in ivy.xml. In fact, I excluded it when hadoopversion=23, but
it should be excluded globally regardless of the hadoopversion. The good thing is that hqsldb-2.2.x
has a different org attribute from hsql-1.8.x, so excluding hsqldb does not prevent us from
downloading hsqldb-2.2.x.

Can you please add the following change to your patch? This fixes the problem.

{code}
--- ivy.xml	(revision 1366535)
+++ ivy.xml	(working copy)
@@ -74,7 +74,6 @@
       rev="${hadoop.version}" conf="hadoop23->default">
       <artifact name="hadoop-common" type="jar" />
       <artifact name="hadoop-common" type="jar" m:classifier="tests"/>
-      <exclude org="hsqldb" module="hsqldb"/>
     </dependency>
     <dependency org="org.apache.hadoop" name="hadoop-hdfs"
       rev="${hadoop.version}" conf="hadoop23->default">
@@ -150,6 +149,7 @@
       <exclude org="com.cloudera.cdh" module="zookeeper-ant" />
     </dependency>
 
+    <exclude org="hsqldb" module="hsqldb"/>
     <exclude org="org.apache.hadoop" module="avro"/>
     <exclude org="commons-daemon" module="commons-daemon" />
{code}

                  
> hsqldb jar update regression
> ----------------------------
>
>                 Key: SQOOP-524
>                 URL: https://issues.apache.org/jira/browse/SQOOP-524
>             Project: Sqoop
>          Issue Type: Bug
>            Reporter: Abhijeet Gaikwad
>            Assignee: Abhijeet Gaikwad
>            Priority: Blocker
>             Fix For: 1.4.2
>
>         Attachments: SQOOP-524.patch
>
>
> This is due to recent update of hsqldb jar version. Updating the jar does not break the
build but it breaks the execution of every SQOOP job, as Hadoop's classpath is prepended to
Sqoop's classpath and hadoop contains previous version of hsqldb JAR.
> This is a class loading issue. hsqldb-1.8.0.10.jar from hadoop is loaded first and we
have hsqldb-2.2.8.jar specific code in sqoop.
> Steps to reproduce:
> 1. Checkout latest revision from SVN/GIT
> 2. Build - "ant -Dhadoopversion <as_per_hadoop_version_you_have>" 
> 3. Try to run sqoop with any DBMS. You get -
> Exception in thread "main" java.lang.NoSuchMethodError: org.hsqldb.DatabaseURL.parseURL(Ljava/lang/String;ZZ)Lorg/hsqldb/persist/HsqlProperties;
>         at org.hsqldb.jdbc.JDBCDriver.getConnection(Unknown Source)
>         at org.hsqldb.jdbc.JDBCDriver.connect(Unknown Source)
>         at java.sql.DriverManager.getConnection(DriverManager.java:582)
>         at java.sql.DriverManager.getConnection(DriverManager.java:185)
>         at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:665)
>         at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
>         at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:525)
>         at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:548)
>         at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:191)
>         at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:175)
>         at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:262)
>         at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1235)
>         at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1060)
>         at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:82)
>         at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:390)
>         at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476)
>         at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>         at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
>         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
>         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
>         at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
>         at com.cloudera.sqoop.Sqoop.main(Sqoop.java:57)
> Unless, you set this on command-line:
> export HADOOP_USER_CLASSPATH_FIRST=true
> Haven't tried ahead but it should also fail when Sqoop internally triggers a Hadoop JOB.
This is just a speculation, will confirm this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message