sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sam liu <liuqiyun2...@gmail.com>
Subject Re: Failed to use sqoop-1.4.3 job tool on hadoop-2.1.0-beta
Date Wed, 20 Nov 2013 07:41:29 GMT
I found a solution to resolve this issue: replace the content of
˜/.sqoop/metastore.db.script with following codes. However, I have a
questions: what code generated ˜/.sqoop/metastore.db.script?

Codes for replacement:
SET DATABASE UNIQUE NAME HSQLDB427399082E
SET DATABASE GC 0
SET DATABASE DEFAULT RESULT MEMORY ROWS 0
SET DATABASE EVENT LOG LEVEL 0
SET DATABASE SQL REFERENCES FALSE
SET DATABASE SQL SIZE TRUE
SET DATABASE SQL NAMES FALSE
SET DATABASE TRANSACTION CONTROL LOCKS
SET DATABASE DEFAULT ISOLATION LEVEL READ COMMITTED
SET FILES WRITE DELAY 10
SET FILES BACKUP INCREMENT TRUE
SET FILES CACHE SIZE 10000
SET FILES CACHE ROWS 50000
SET FILES SCALE 1
SET FILES LOB SCALE 32
SET FILES DEFRAG 20
SET FILES NIO TRUE
SET FILES LOG TRUE
SET FILES LOG SIZE 200
SET DATABASE TEXT TABLE DEFAULTS ''
CREATE USER SA PASSWORD ''
CREATE SCHEMA PUBLIC AUTHORIZATION DBA
SET SCHEMA PUBLIC
CREATE MEMORY TABLE PUBLIC.SQOOP_ROOT(VERSION INTEGER,PROPNAME VARCHAR(128)
NOT NULL,PROPVAL VARCHAR(256),CONSTRAINT SQOOP_ROOT_UNQ
UNIQUE(VERSION,PROPNAME)
)
CREATE MEMORY TABLE PUBLIC.SQOOP_SESSIONS(JOB_NAME VARCHAR(64) NOT
NULL,PROPNAME VARCHAR(128) NOT NULL,PROPVAL VARCHAR(1024),PROPCLASS
VARCHAR(32) NOT NULL,
CONSTRAINT SQOOP_SESSIONS_UNQ UNIQUE(JOB_NAME,PROPNAME,PROPCLASS))
ALTER SEQUENCE SYSTEM_LOBS.LOB_ID RESTART WITH 1
SET DATABASE DEFAULT INITIAL SCHEMA PUBLIC
GRANT USAGE ON DOMAIN INFORMATION_SCHEMA.YES_OR_NO TO PUBLIC
GRANT USAGE ON DOMAIN INFORMATION_SCHEMA.TIME_STAMP TO PUBLIC
GRANT USAGE ON DOMAIN INFORMATION_SCHEMA.CARDINAL_NUMBER TO PUBLIC
GRANT USAGE ON DOMAIN INFORMATION_SCHEMA.CHARACTER_DATA TO PUBLIC
GRANT USAGE ON DOMAIN INFORMATION_SCHEMA.SQL_IDENTIFIER TO PUBLIC
GRANT DBA TO SA
SET SCHEMA PUBLIC
INSERT INTO SQOOP_ROOT VALUES(NULL,'sqoop.hsqldb.job.storage.version','0')
INSERT INTO SQOOP_ROOT
VALUES(0,'sqoop.hsqldb.job.info.table','SQOOP_SESSIONS')


2013/11/20 sam liu <liuqiyun2011@gmail.com>

> I am confusing these exceptions... For using 'sqoop job --list' command,
> sqoop will connect to hsqldb, and need hsqldb has a database for storing
> sqoop job metadata in advance, right?
>
>
> 2013/11/19 sam liu <liuqiyun2011@gmail.com>
>
>> Updated the exception info:
>>
>>
>> Now I failed to run sqoop-1.4.3 job tool($SQOOP_HOME/bin/sqoop job
>> --list) agains hadoop-2.1.0-beta:
>> 1) If I use hsqldb-2.0.0.jar in my env, the exception will be:13/11/18
>> 21:52:47 WARN hsqldb.HsqldbJobStorage: Could not interpret as a number: null
>> 13/11/18 21:52:47 ERROR hsqldb.HsqldbJobStorage: Can not interpret
>> metadata schema
>> 13/11/18 21:52:47 ERROR hsqldb.HsqldbJobStorage: The metadata schema
>> version is null
>> 13/11/18 21:52:47 ERROR hsqldb.HsqldbJobStorage: The highest version
>> supported is 0
>> 13/11/18 21:52:47 ERROR hsqldb.HsqldbJobStorage: To use this version of
>> Sqoop, you must downgrade your metadata schema.
>> 13/11/18 21:52:47 ERROR tool.JobTool: I/O error performing job operation:
>> java.io.IOException: Invalid metadata version.
>>         at
>> org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.init(HsqldbJobStorage.java:202)
>>
>>         at
>> org.apache.sqoop.metastore.hsqldb.AutoHsqldbStorage.open(AutoHsqldbStorage.java:112)
>>         at org.apache.sqoop.tool.JobTool.run(JobTool.java:274)
>>         at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
>>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>         at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
>>         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
>>         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
>>         at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
>>
>>
>> 2) If I use hsqldb-1.8.0.10.jar in my env, the exception will be:
>> Exception in thread "main" java.lang.NoSuchMethodError:
>> org/hsqldb/DatabaseURL.
>> parseURL(Ljava/lang/String;ZZ)Lorg/hsqldb/persist/HsqlProperties;
>>         at org.hsqldb.jdbc.JDBCDriver.getConnection(Unknown Source)
>>         at org.hsqldb.jdbc.JDBCDriver.connect(Unknown Source)
>>         at java.sql.DriverManager.getConnection(DriverManager.java:419)
>>         at java.sql.DriverManager.getConnection(DriverManager.java:467)
>>         at
>> org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.init(HsqldbJobStorage.java:176)
>>         at
>> org.apache.sqoop.metastore.hsqldb.AutoHsqldbStorage.open(AutoHsqldbStorage.java:112)
>>         at org.apache.sqoop.tool.JobTool.run(JobTool.java:274)
>>         at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
>>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>         at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
>>         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
>>         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
>>         at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
>>
>>
>>
>> 2013/11/19 sam liu <liuqiyun2011@gmail.com>
>>
>>> Hi Experts,
>>>
>>> My cluster is hadoop-2.1.0-beta and it requires hsqldb-2.0.0.jar. At the
>>> same time, my sqoop is 1.4.3, and it requires hsqldb-1.8.0.10.jar.
>>>
>>> Now I failed to run sqoop-1.4.3 job tool($SQOOP_HOME/bin/sqoop job
>>> --list) agains hadoop-2.1.0-beta:
>>> 1) If I use hsqldb-2.0.0.jar in my env, the exception will be:
>>> Exception in thread "main" java.lang.NoSuchMethodError:
>>> org/hsqldb/DatabaseURL.parseURL(Ljava/lang/String;ZZ)Lorg/hsqldb/persist/HsqlProperties;
>>>         at org.hsqldb.jdbc.JDBCDriver.getConnection(Unknown Source)
>>>         at org.hsqldb.jdbc.JDBCDriver.connect(Unknown Source)
>>>         at java.sql.DriverManager.getConnection(DriverManager.java:419)
>>>         at java.sql.DriverManager.getConnection(DriverManager.java:467)
>>>         at
>>> org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.init(HsqldbJobStorage.java:176)
>>>         at
>>> org.apache.sqoop.metastore.hsqldb.AutoHsqldbStorage.open(AutoHsqldbStorage.java:112)
>>>         at org.apache.sqoop.tool.JobTool.run(JobTool.java:274)
>>>         at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
>>>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>>         at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
>>>         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
>>>         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
>>>         at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
>>>
>>> 2) If I use hsqldb-1.8.0.10.jar in my env, the exception will be:
>>> Exception in thread "main" java.lang.NoSuchMethodError:
>>> org/hsqldb/DatabaseURL.parseURL(Ljava/lang/String;ZZ)Lorg/hsqldb/persist/HsqlProperties;
>>>         at org.hsqldb.jdbc.JDBCDriver.getConnection(Unknown Source)
>>>         at org.hsqldb.jdbc.JDBCDriver.connect(Unknown Source)
>>>         at java.sql.DriverManager.getConnection(DriverManager.java:419)
>>>         at java.sql.DriverManager.getConnection(DriverManager.java:467)
>>>         at
>>> org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.init(HsqldbJobStorage.java:176)
>>>         at
>>> org.apache.sqoop.metastore.hsqldb.AutoHsqldbStorage.open(AutoHsqldbStorage.java:112)
>>>         at org.apache.sqoop.tool.JobTool.run(JobTool.java:274)
>>>         at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
>>>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>>         at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
>>>         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
>>>         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
>>>         at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
>>>
>>> Is it a defect of Sqoop?
>>>
>>> Thanks!
>>>
>>> --
>>>
>>> Sam Liu
>>>
>>
>>
>>
>> --
>>
>> Sam Liu
>>
>
>
>
> --
>
> Sam Liu
>



-- 

Sam Liu

Mime
View raw message