sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jarek Jarcec Cecho <jar...@apache.org>
Subject Re: Importing from OracleDB
Date Fri, 01 Feb 2013 17:26:32 GMT
Hi James,
I'm afraid that using synonyms is not supported by Oracle driver. The problem is that Oracle
driver is issuing following query to get columns for imported table:

  SELECT COLUMN_NAME FROM ALL_TAB_COLUMNS WHERE OWNER = ? AND TABLE_NAME = ? ORDER BY COLUMN_ID

This query returns no rows for synonym and thus import fails.

As a workaround I would recommend to try out Generic JDBC Connector that is not using this
query to fetch table metadata. You can force Sqoop to use Generic JDBC Connector by passing
"--driver" parameter with value oracle.jdbc.OracleDriver, e.g.

  sqoop import --connect ... --driver oracle.jdbc.OracleDriver

Another solution that might possibly help you is to install OraOop connector, but I'm not
sure whether it supports synonyms or not.

Jarcec

On Fri, Feb 01, 2013 at 01:37:26PM +0000, James Hogarth wrote:
> Hitting an interesting test case here using
> sqoop-1.4.1+54-1.cdh4.1.2.p0.21.el6.noarch ...
> 
> I have an OracleDB where I need to do a daily grab of a particular table -
> ultimately it needs to end in hive...
> 
> The OracleDB is running 11.2.0.2 and I'm using the 11.2.0.3 (ie current)
> jdbc thin (java6 of course) driver.
> 
> I've tried quite the variety of combination of username and table name
> capitalization options (as per that known issue) both quoting and not
> quoting. I should note that for security reasons on this particular
> database I have access to a synonym only and not a standard table.
> 
> If I don't specify the columns to import (so attempt to let sqoop
> automatically get the list of columns for the table) I get:
> 
> [me@client ~]$ sqoop import --connect jdbc:oracle:thin:@database:1525:SID
> --username user --password pass --table table1  --num-mappers 1
> --target-dir table1_temp  --verbose
> 
> 13/02/01 11:38:23 DEBUG tool.BaseSqoopTool: Enabled debug logging.
> 13/02/01 11:38:23 WARN tool.BaseSqoopTool: Setting your password on the
> command-line is insecure. Consider using -P instead.
> 13/02/01 11:38:23 DEBUG sqoop.ConnFactory: Loaded manager factory:
> com.cloudera.sqoop.manager.DefaultManagerFactory
> 13/02/01 11:38:23 DEBUG sqoop.ConnFactory: Trying ManagerFactory:
> com.cloudera.sqoop.manager.DefaultManagerFactory
> 13/02/01 11:38:23 DEBUG manager.DefaultManagerFactory: Trying with scheme:
> jdbc:oracle:thin:@database:1525
> 13/02/01 11:38:23 DEBUG manager.OracleManager$ConnCache: Instantiated new
> connection cache.
> 13/02/01 11:38:23 INFO manager.SqlManager: Using default fetchSize of 1000
> 13/02/01 11:38:23 DEBUG sqoop.ConnFactory: Instantiated ConnManager
> org.apache.sqoop.manager.OracleManager@16bdb503
> 13/02/01 11:38:23 INFO tool.CodeGenTool: Beginning code generation
> 13/02/01 11:38:23 DEBUG manager.OracleManager: Using column names query:
> SELECT t.* FROM table1 t WHERE 1=0
> 13/02/01 11:38:23 DEBUG manager.OracleManager: Creating a new connection
> for jdbc:oracle:thin:@database:1525:SID, using username: user
> 13/02/01 11:38:23 DEBUG manager.OracleManager: No connection paramenters
> specified. Using regular API for making connection.
> 13/02/01 11:38:24 INFO manager.OracleManager: Time zone has been set to GMT
> 13/02/01 11:38:24 DEBUG manager.SqlManager: Using fetchSize for next query:
> 1000
> 13/02/01 11:38:24 INFO manager.SqlManager: Executing SQL statement: SELECT
> t.* FROM table1 t WHERE 1=0
> 13/02/01 11:38:24 DEBUG manager.OracleManager$ConnCache: Caching released
> connection for jdbc:oracle:thin:@database:1525:SID/username
> 13/02/01 11:38:24 DEBUG orm.ClassWriter: selected columns:
> 13/02/01 11:38:24 ERROR tool.ImportTool: Imported Failed: Attempted to
> generate class with no columns!
> 
> Specifying the columns writes to HDFS okay:
> 
> [me@client ~]$ sqoop import --connect jdbc:oracle:thin:@database:1525:SID
> --username user --password pass --table table1 --columns
> col1,col2,col3,col4,col5,col6,col7,col8,col9,col10,col11,col12
> --num-mappers 1 --target-dir table1_temp  --verbose
> 13/02/01 12:54:22 DEBUG tool.BaseSqoopTool: Enabled debug logging.
> 13/02/01 12:54:22 WARN tool.BaseSqoopTool: Setting your password on the
> command-line is insecure. Consider using -P instead.
> 13/02/01 12:54:22 DEBUG sqoop.ConnFactory: Loaded manager factory:
> com.cloudera.sqoop.manager.DefaultManagerFactory
> 13/02/01 12:54:22 DEBUG sqoop.ConnFactory: Trying ManagerFactory:
> com.cloudera.sqoop.manager.DefaultManagerFactory
> 13/02/01 12:54:22 DEBUG manager.DefaultManagerFactory: Trying with scheme:
> jdbc:oracle:thin:@database:1525
> 13/02/01 12:54:22 DEBUG manager.OracleManager$ConnCache: Instantiated new
> connection cache.
> 13/02/01 12:54:22 INFO manager.SqlManager: Using default fetchSize of 1000
> 13/02/01 12:54:22 DEBUG sqoop.ConnFactory: Instantiated ConnManager
> org.apache.sqoop.manager.OracleManager@6719dc16
> 13/02/01 12:54:22 INFO tool.CodeGenTool: Beginning code generation
> 13/02/01 12:54:22 DEBUG manager.OracleManager: Using column names query:
> SELECT t.* FROM table1 t WHERE 1=0
> 13/02/01 12:54:22 DEBUG manager.OracleManager: Creating a new connection
> for jdbc:oracle:thin:@database:1525:SID, using username: user
> 13/02/01 12:54:22 DEBUG manager.OracleManager: No connection paramenters
> specified. Using regular API for making connection.
> 13/02/01 12:54:23 INFO manager.OracleManager: Time zone has been set to GMT
> 13/02/01 12:54:23 DEBUG manager.SqlManager: Using fetchSize for next query:
> 1000
> 13/02/01 12:54:23 INFO manager.SqlManager: Executing SQL statement: SELECT
> t.* FROM table1 t WHERE 1=0
> 13/02/01 12:54:23 DEBUG orm.ClassWriter: selected columns:
> 13/02/01 12:54:23 DEBUG orm.ClassWriter:   col1
> 13/02/01 12:54:23 DEBUG orm.ClassWriter:   col2
> 13/02/01 12:54:23 DEBUG orm.ClassWriter:   col3
> 13/02/01 12:54:23 DEBUG orm.ClassWriter:   col4
> 13/02/01 12:54:23 DEBUG orm.ClassWriter:   col5
> 13/02/01 12:54:23 DEBUG orm.ClassWriter:   col6
> 13/02/01 12:54:23 DEBUG orm.ClassWriter:   col7
> 13/02/01 12:54:23 DEBUG orm.ClassWriter:   col8
> 13/02/01 12:54:23 DEBUG orm.ClassWriter:   col9
> 13/02/01 12:54:23 DEBUG orm.ClassWriter:   col10
> 13/02/01 12:54:23 DEBUG orm.ClassWriter:   col11
> 13/02/01 12:54:23 DEBUG orm.ClassWriter:   col12
> 13/02/01 12:54:23 DEBUG orm.ClassWriter: Writing source file:
> /tmp/sqoop-user/compile/485bb295736ca89d6d471d61c3690220/table1.java
> 13/02/01 12:54:23 DEBUG orm.ClassWriter: Table name: table1
> 13/02/01 12:54:23 DEBUG orm.ClassWriter: Columns: col1:93, col2:2, col3:12,
> col4:12, col5:12, col6:12, col7:12, col8:12, col9:12, col10:2, col11:12,
> col12:2,
> 13/02/01 12:54:23 DEBUG orm.ClassWriter: sourceFilename is table1.java
> 13/02/01 12:54:23 DEBUG orm.CompilationManager: Found existing
> /tmp/sqoop-user/compile/485bb295736ca89d6d471d61c3690220/
> 13/02/01 12:54:23 INFO orm.CompilationManager: HADOOP_HOME is
> /usr/lib/hadoop
> 13/02/01 12:54:23 DEBUG orm.CompilationManager: Adding source file:
> /tmp/sqoop-user/compile/485bb295736ca89d6d471d61c3690220/table1.java
> 13/02/01 12:54:23 DEBUG orm.CompilationManager: Invoking javac with args:
> 13/02/01 12:54:23 DEBUG orm.CompilationManager:   -sourcepath
> 13/02/01 12:54:23 DEBUG orm.CompilationManager:
> /tmp/sqoop-user/compile/485bb295736ca89d6d471d61c3690220/
> 13/02/01 12:54:23 DEBUG orm.CompilationManager:   -d
> 13/02/01 12:54:23 DEBUG orm.CompilationManager:
> /tmp/sqoop-user/compile/485bb295736ca89d6d471d61c3690220/
> 13/02/01 12:54:23 DEBUG orm.CompilationManager:   -classpath
> 13/02/01 12:54:23 DEBUG orm.CompilationManager:
> /etc/hadoop/conf:/usr/lib/hadoop/lib/jersey-server-1.8.jar:/usr/lib/hadoop/lib/xmlenc-0.52.jar:/usr/lib/hadoop/lib/commons-beanutils-1.7.0.jar:/usr/lib/hadoop/lib/servlet-api-2.5.jar:/usr/lib/hadoop/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop/lib/jackson-jaxrs-1.8.8.jar:/usr/lib/hadoop/lib/commons-logging-1.1.1.jar:/usr/lib/hadoop/lib/mockito-all-1.8.5.jar:/usr/lib/hadoop/lib/stax-api-1.0.1.jar:/usr/lib/hadoop/lib/jettison-1.1.jar:/usr/lib/hadoop/lib/commons-cli-1.2.jar:/usr/lib/hadoop/lib/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop/lib/guava-11.0.2.jar:/usr/lib/hadoop/lib/jline-0.9.94.jar:/usr/lib/hadoop/lib/avro-1.7.1.cloudera.2.jar:/usr/lib/hadoop/lib/commons-math-2.1.jar:/usr/lib/hadoop/lib/jasper-runtime-5.5.23.jar:/usr/lib/hadoop/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop/lib/kfs-0.3.jar:/usr/lib/hadoop/lib/commons-collections-3.2.1.jar:/usr/lib/hadoop/lib/activation-1.1.jar:/usr/lib/hadoop/lib/jasper-compiler-5.5.23.jar:/usr/lib/hadoop/lib/jsp-api-2.1.jar:/usr/lib/hadoop/lib/log4j-1.2.17.jar:/usr/lib/hadoop/lib/jets3t-0.6.1.jar:/usr/lib/hadoop/lib/jetty-6.1.26.cloudera.2.jar:/usr/lib/hadoop/lib/jetty-util-6.1.26.cloudera.2.jar:/usr/lib/hadoop/lib/commons-httpclient-3.1.jar:/usr/lib/hadoop/lib/jersey-json-1.8.jar:/usr/lib/hadoop/lib/jersey-core-1.8.jar:/usr/lib/hadoop/lib/jsch-0.1.42.jar:/usr/lib/hadoop/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop/lib/jackson-xc-1.8.8.jar:/usr/lib/hadoop/lib/slf4j-api-1.6.1.jar:/usr/lib/hadoop/lib/jsr305-1.3.9.jar:/usr/lib/hadoop/lib/junit-4.8.2.jar:/usr/lib/hadoop/lib/hue-plugins-2.1.0-cdh4.1.2.jar:/usr/lib/hadoop/lib/zookeeper-3.4.3-cdh4.1.2.jar:/usr/lib/hadoop/lib/slf4j-log4j12-1.6.1.jar:/usr/lib/hadoop/lib/commons-net-3.1.jar:/usr/lib/hadoop/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop/lib/commons-codec-1.4.jar:/usr/lib/hadoop/lib/commons-io-2.1.jar:/usr/lib/hadoop/lib/commons-el-1.0.jar:/usr/lib/hadoop/lib/commons-lang-2.5.jar:/usr/lib/hadoop/lib/paranamer-2.3.jar:/usr/lib/hadoop/lib/commons-configuration-1.6.jar:/usr/lib/hadoop/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop/lib/commons-digester-1.8.jar:/usr/lib/hadoop/lib/asm-3.2.jar:/usr/lib/hadoop/.//hadoop-common-2.0.0-cdh4.1.2-tests.jar:/usr/lib/hadoop/.//hadoop-auth.jar:/usr/lib/hadoop/.//hadoop-annotations-2.0.0-cdh4.1.2.jar:/usr/lib/hadoop/.//hadoop-annotations.jar:/usr/lib/hadoop/.//hadoop-common.jar:/usr/lib/hadoop/.//hadoop-common-2.0.0-cdh4.1.2.jar:/usr/lib/hadoop/.//hadoop-auth-2.0.0-cdh4.1.2.jar:/usr/lib/sqoop/conf:/etc/zookeeper::/usr/lib/sqoop/lib/ant-contrib-1.0b3.jar:/usr/lib/sqoop/lib/ant-eclipse-1.0-jvm1.2.jar:/usr/lib/sqoop/lib/avro-1.7.1.cloudera.2.jar:/usr/lib/sqoop/lib/avro-ipc-1.7.1.cloudera.2.jar:/usr/lib/sqoop/lib/avro-mapred-1.7.1.cloudera.2.jar:/usr/lib/sqoop/lib/commons-io-1.4.jar:/usr/lib/sqoop/lib/hsqldb-1.8.0.10.jar:/usr/lib/sqoop/lib/ifxjdbc.jar:/usr/lib/sqoop/lib/jackson-core-asl-1.8.8.jar:/usr/lib/sqoop/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/sqoop/lib/jetty-util-6.1.26.jar:/usr/lib/sqoop/lib/netty-3.4.0.Final.jar:/usr/lib/sqoop/lib/ojdbc6.jar:/usr/lib/sqoop/lib/paranamer-2.3.jar:/usr/lib/sqoop/lib/servlet-api-2.5-20081211.jar:/usr/lib/sqoop/lib/snappy-java-1.0.4.1.jar:/usr/lib/hbase/bin/../conf:/usr/java/jdk1.6.0_31/lib/tools.jar:/usr/lib/hbase/bin/..:/usr/lib/hbase/bin/../hbase-0.92.1-cdh4.1.2-security.jar:/usr/lib/hbase/bin/../hbase-0.92.1-cdh4.1.2-security-tests.jar:/usr/lib/hbase/bin/../hbase.jar:/usr/lib/hbase/bin/../lib/activation-1.1.jar:/usr/lib/hbase/bin/../lib/aopalliance-1.0.jar:/usr/lib/hbase/bin/../lib/asm-3.2.jar:/usr/lib/hbase/bin/../lib/avro-1.7.1.cloudera.2.jar:/usr/lib/hbase/bin/../lib/commons-beanutils-1.7.0.jar:/usr/lib/hbase/bin/../lib/commons-beanutils-core-1.8.0.jar:/usr/lib/hbase/bin/../lib/commons-cli-1.2.jar:/usr/lib/hbase/bin/../lib/commons-codec-1.4.jar:/usr/lib/hbase/bin/../lib/commons-collections-3.2.1.jar:/usr/lib/hbase/bin/../lib/commons-configuration-1.6.jar:/usr/lib/hbase/bin/../lib/commons-daemon-1.0.3.jar:/usr/lib/hbase/bin/../lib/commons-digester-1.8.jar:/usr/lib/hbase/bin/../lib/commons-el-1.0.jar:/usr/lib/hbase/bin/../lib/commons-httpclient-3.1.jar:/usr/lib/hbase/bin/../lib/commons-io-2.1.jar:/usr/lib/hbase/bin/../lib/commons-lang-2.5.jar:/usr/lib/hbase/bin/../lib/commons-logging-1.1.1.jar:/usr/lib/hbase/bin/../lib/commons-net-3.1.jar:/usr/lib/hbase/bin/../lib/core-3.1.1.jar:/usr/lib/hbase/bin/../lib/gmbal-api-only-3.0.0-b023.jar:/usr/lib/hbase/bin/../lib/grizzly-framework-2.1.1.jar:/usr/lib/hbase/bin/../lib/grizzly-framework-2.1.1-tests.jar:/usr/lib/hbase/bin/../lib/grizzly-http-2.1.1.jar:/usr/lib/hbase/bin/../lib/grizzly-http-server-2.1.1.jar:/usr/lib/hbase/bin/../lib/grizzly-http-servlet-2.1.1.jar:/usr/lib/hbase/bin/../lib/grizzly-rcm-2.1.1.jar:/usr/lib/hbase/bin/../lib/guava-11.0.2.jar:/usr/lib/hbase/bin/../lib/guice-3.0.jar:/usr/lib/hbase/bin/../lib/guice-servlet-3.0.jar:/usr/lib/hbase/bin/../lib/high-scale-lib-1.1.1.jar:/usr/lib/hbase/bin/../lib/httpclient-4.0.1.jar:/usr/lib/hbase/bin/../lib/httpcore-4.0.1.jar:/usr/lib/hbase/bin/../lib/jackson-core-asl-1.8.8.jar:/usr/lib/hbase/bin/../lib/jackson-jaxrs-1.8.8.jar:/usr/lib/hbase/bin/../lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hbase/bin/../lib/jackson-xc-1.8.8.jar:/usr/lib/hbase/bin/../lib/jamon-runtime-2.3.1.jar:/usr/lib/hbase/bin/../lib/jasper-compiler-5.5.23.jar:/usr/lib/hbase/bin/../lib/jasper-runtime-5.5.23.jar:/usr/lib/hbase/bin/../lib/javax.inject-1.jar:/usr/lib/hbase/bin/../lib/javax.servlet-3.0.jar:/usr/lib/hbase/bin/../lib/jaxb-api-2.1.jar:/usr/lib/hbase/bin/../lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hbase/bin/../lib/jersey-client-1.8.jar:/usr/lib/hbase/bin/../lib/jersey-core-1.8.jar:/usr/lib/hbase/bin/../lib/jersey-grizzly2-1.8.jar:/usr/lib/hbase/bin/../lib/jersey-guice-1.8.jar:/usr/lib/hbase/bin/../lib/jersey-json-1.8.jar:/usr/lib/hbase/bin/../lib/jersey-server-1.8.jar:/usr/lib/hbase/bin/../lib/jersey-test-framework-core-1.8.jar:/usr/lib/hbase/bin/../lib/jersey-test-framework-grizzly2-1.8.jar:/usr/lib/hbase/bin/../lib/jets3t-0.6.1.jar:/usr/lib/hbase/bin/../lib/jettison-1.1.jar:/usr/lib/hbase/bin/../lib/jetty-6.1.26.cloudera.2.jar:/usr/lib/hbase/bin/../lib/jetty-util-6.1.26.cloudera.2.jar:/usr/lib/hbase/bin/../lib/jruby-complete-1.6.5.jar:/usr/lib/hbase/bin/../lib/jsch-0.1.42.jar:/usr/lib/hbase/bin/../lib/jsp-2.1-6.1.14.jar:/usr/lib/hbase/bin/../lib/jsp-api-2.1-6.1.14.jar:/usr/lib/hbase/bin/../lib/jsp-api-2.1.jar:/usr/lib/hbase/bin/../lib/jsr305-1.3.9.jar:/usr/lib/hbase/bin/../lib/kfs-0.3.jar:/usr/lib/hbase/bin/../lib/libthrift-0.7.0.jar:/usr/lib/hbase/bin/../lib/log4j-1.2.17.jar:/usr/lib/hbase/bin/../lib/management-api-3.0.0-b012.jar:/usr/lib/hbase/bin/../lib/metrics-core-2.1.2.jar:/usr/lib/hbase/bin/../lib/netty-3.2.4.Final.jar:/usr/lib/hbase/bin/../lib/paranamer-2.3.jar:/usr/lib/hbase/bin/../lib/protobuf-java-2.4.0a.jar:/usr/lib/hbase/bin/../lib/servlet-api-2.5-6.1.14.jar:/usr/lib/hbase/bin/../lib/servlet-api-2.5.jar:/usr/lib/hbase/bin/../lib/slf4j-api-1.6.1.jar:/usr/lib/hbase/bin/../lib/snappy-java-1.0.4.1.jar:/usr/lib/hbase/bin/../lib/stax-api-1.0.1.jar:/usr/lib/hbase/bin/../lib/xmlenc-0.52.jar:/usr/lib/hbase/bin/../lib/zookeeper.jar:/etc/hadoop/conf:/usr/lib/hadoop/lib/jersey-server-1.8.jar:/usr/lib/hadoop/lib/xmlenc-0.52.jar:/usr/lib/hadoop/lib/commons-beanutils-1.7.0.jar:/usr/lib/hadoop/lib/servlet-api-2.5.jar:/usr/lib/hadoop/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop/lib/jackson-jaxrs-1.8.8.jar:/usr/lib/hadoop/lib/commons-logging-1.1.1.jar:/usr/lib/hadoop/lib/mockito-all-1.8.5.jar:/usr/lib/hadoop/lib/stax-api-1.0.1.jar:/usr/lib/hadoop/lib/jettison-1.1.jar:/usr/lib/hadoop/lib/commons-cli-1.2.jar:/usr/lib/hadoop/lib/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop/lib/guava-11.0.2.jar:/usr/lib/hadoop/lib/jline-0.9.94.jar:/usr/lib/hadoop/lib/avro-1.7.1.cloudera.2.jar:/usr/lib/hadoop/lib/commons-math-2.1.jar:/usr/lib/hadoop/lib/jasper-runtime-5.5.23.jar:/usr/lib/hadoop/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop/lib/kfs-0.3.jar:/usr/lib/hadoop/lib/commons-collections-3.2.1.jar:/usr/lib/hadoop/lib/activation-1.1.jar:/usr/lib/hadoop/lib/jasper-compiler-5.5.23.jar:/usr/lib/hadoop/lib/jsp-api-2.1.jar:/usr/lib/hadoop/lib/log4j-1.2.17.jar:/usr/lib/hadoop/lib/jets3t-0.6.1.jar:/usr/lib/hadoop/lib/jetty-6.1.26.cloudera.2.jar:/usr/lib/hadoop/lib/jetty-util-6.1.26.cloudera.2.jar:/usr/lib/hadoop/lib/commons-httpclient-3.1.jar:/usr/lib/hadoop/lib/jersey-json-1.8.jar:/usr/lib/hadoop/lib/jersey-core-1.8.jar:/usr/lib/hadoop/lib/jsch-0.1.42.jar:/usr/lib/hadoop/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop/lib/jackson-xc-1.8.8.jar:/usr/lib/hadoop/lib/slf4j-api-1.6.1.jar:/usr/lib/hadoop/lib/jsr305-1.3.9.jar:/usr/lib/hadoop/lib/junit-4.8.2.jar:/usr/lib/hadoop/lib/hue-plugins-2.1.0-cdh4.1.2.jar:/usr/lib/hadoop/lib/zookeeper-3.4.3-cdh4.1.2.jar:/usr/lib/hadoop/lib/slf4j-log4j12-1.6.1.jar:/usr/lib/hadoop/lib/commons-net-3.1.jar:/usr/lib/hadoop/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop/lib/commons-codec-1.4.jar:/usr/lib/hadoop/lib/commons-io-2.1.jar:/usr/lib/hadoop/lib/commons-el-1.0.jar:/usr/lib/hadoop/lib/commons-lang-2.5.jar:/usr/lib/hadoop/lib/paranamer-2.3.jar:/usr/lib/hadoop/lib/commons-configuration-1.6.jar:/usr/lib/hadoop/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop/lib/commons-digester-1.8.jar:/usr/lib/hadoop/lib/asm-3.2.jar:/usr/lib/hadoop/.//hadoop-common-2.0.0-cdh4.1.2-tests.jar:/usr/lib/hadoop/.//hadoop-auth.jar:/usr/lib/hadoop/.//hadoop-annotations-2.0.0-cdh4.1.2.jar:/usr/lib/hadoop/.//hadoop-annotations.jar:/usr/lib/hadoop/.//hadoop-common.jar:/usr/lib/hadoop/.//hadoop-common-2.0.0-cdh4.1.2.jar:/usr/lib/hadoop/.//hadoop-auth-2.0.0-cdh4.1.2.jar:/usr/lib/hadoop-hdfs/./:/usr/lib/hadoop-hdfs/lib/jersey-server-1.8.jar:/usr/lib/hadoop-hdfs/lib/xmlenc-0.52.jar:/usr/lib/hadoop-hdfs/lib/servlet-api-2.5.jar:/usr/lib/hadoop-hdfs/lib/commons-daemon-1.0.3.jar:/usr/lib/hadoop-hdfs/lib/commons-logging-1.1.1.jar:/usr/lib/hadoop-hdfs/lib/commons-cli-1.2.jar:/usr/lib/hadoop-hdfs/lib/guava-11.0.2.jar:/usr/lib/hadoop-hdfs/lib/jline-0.9.94.jar:/usr/lib/hadoop-hdfs/lib/jasper-runtime-5.5.23.jar:/usr/lib/hadoop-hdfs/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-hdfs/lib/jsp-api-2.1.jar:/usr/lib/hadoop-hdfs/lib/log4j-1.2.17.jar:/usr/lib/hadoop-hdfs/lib/jetty-6.1.26.cloudera.2.jar:/usr/lib/hadoop-hdfs/lib/jetty-util-6.1.26.cloudera.2.jar:/usr/lib/hadoop-hdfs/lib/jersey-core-1.8.jar:/usr/lib/hadoop-hdfs/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-hdfs/lib/jsr305-1.3.9.jar:/usr/lib/hadoop-hdfs/lib/zookeeper-3.4.3-cdh4.1.2.jar:/usr/lib/hadoop-hdfs/lib/commons-codec-1.4.jar:/usr/lib/hadoop-hdfs/lib/commons-io-2.1.jar:/usr/lib/hadoop-hdfs/lib/commons-el-1.0.jar:/usr/lib/hadoop-hdfs/lib/commons-lang-2.5.jar:/usr/lib/hadoop-hdfs/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop-hdfs/lib/asm-3.2.jar:/usr/lib/hadoop-hdfs/.//hadoop-hdfs-2.0.0-cdh4.1.2-tests.jar:/usr/lib/hadoop-hdfs/.//hadoop-hdfs.jar:/usr/lib/hadoop-hdfs/.//hadoop-hdfs-2.0.0-cdh4.1.2.jar:/usr/lib/hadoop-yarn/lib/jersey-server-1.8.jar:/usr/lib/hadoop-yarn/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop-yarn/lib/javax.inject-1.jar:/usr/lib/hadoop-yarn/lib/avro-1.7.1.cloudera.2.jar:/usr/lib/hadoop-yarn/lib/guice-3.0.jar:/usr/lib/hadoop-yarn/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-yarn/lib/log4j-1.2.17.jar:/usr/lib/hadoop-yarn/lib/netty-3.2.4.Final.jar:/usr/lib/hadoop-yarn/lib/guice-servlet-3.0.jar:/usr/lib/hadoop-yarn/lib/jersey-guice-1.8.jar:/usr/lib/hadoop-yarn/lib/aopalliance-1.0.jar:/usr/lib/hadoop-yarn/lib/jersey-core-1.8.jar:/usr/lib/hadoop-yarn/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-yarn/lib/commons-io-2.1.jar:/usr/lib/hadoop-yarn/lib/paranamer-2.3.jar:/usr/lib/hadoop-yarn/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop-yarn/lib/asm-3.2.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-applications-distributedshell-2.0.0-cdh4.1.2.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-resourcemanager.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-common-2.0.0-cdh4.1.2.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-resourcemanager-2.0.0-cdh4.1.2.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-nodemanager.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-tests-2.0.0-cdh4.1.2-tests.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-web-proxy.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-tests.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-web-proxy-2.0.0-cdh4.1.2.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-common-2.0.0-cdh4.1.2.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-applications-distributedshell.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-tests-2.0.0-cdh4.1.2.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-nodemanager-2.0.0-cdh4.1.2.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-site-2.0.0-cdh4.1.2.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-site.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-common.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-common.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-api-2.0.0-cdh4.1.2.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-api.jar:/usr/lib/hadoop-0.20-mapreduce/./:/usr/lib/hadoop-0.20-mapreduce/lib/jersey-server-1.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/xmlenc-0.52.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-beanutils-1.7.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/servlet-api-2.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/hadoop-fairscheduler-2.0.0-mr1-cdh4.1.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-jaxrs-1.8.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/aspectjtools-1.6.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-logging-1.1.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/mockito-all-1.8.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/stax-api-1.0.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/hsqldb-1.8.0.10.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jettison-1.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-cli-1.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/guava-11.0.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/kfs-0.2.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/avro-1.7.1.cloudera.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-math-2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jasper-runtime-5.5.23.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/kfs-0.3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-collections-3.2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/activation-1.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/ant-contrib-1.0b3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jasper-compiler-5.5.23.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsp-api-2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/log4j-1.2.17.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jets3t-0.6.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jetty-6.1.26.cloudera.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/aspectjrt-1.6.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jetty-util-6.1.26.cloudera.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-httpclient-3.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/avro-compiler-1.7.1.cloudera.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jersey-json-1.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jersey-core-1.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsch-0.1.42.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-xc-1.8.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/slf4j-api-1.6.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsr305-1.3.9.jar:/usr/lib/hadoop-0.20-mapreduce/lib/junit-4.8.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-net-3.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-codec-1.4.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-io-2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-el-1.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-lang-2.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/paranamer-2.3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-configuration-1.6.jar:/usr/lib/hadoop-0.20-mapreduce/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-digester-1.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/asm-3.2.jar:/usr/lib/hadoop-0.20-mapreduce/.//hadoop-core.jar:/usr/lib/hadoop-0.20-mapreduce/.//hadoop-2.0.0-mr1-cdh4.1.2-test.jar:/usr/lib/hadoop-0.20-mapreduce/.//hadoop-examples.jar:/usr/lib/hadoop-0.20-mapreduce/.//hadoop-tools-2.0.0-mr1-cdh4.1.2.jar:/usr/lib/hadoop-0.20-mapreduce/.//hadoop-2.0.0-mr1-cdh4.1.2-tools.jar:/usr/lib/hadoop-0.20-mapreduce/.//hadoop-core-2.0.0-mr1-cdh4.1.2.jar:/usr/lib/hadoop-0.20-mapreduce/.//hadoop-examples-2.0.0-mr1-cdh4.1.2.jar:/usr/lib/hadoop-0.20-mapreduce/.//hadoop-2.0.0-mr1-cdh4.1.2-core.jar:/usr/lib/hadoop-0.20-mapreduce/.//hadoop-test.jar:/usr/lib/hadoop-0.20-mapreduce/.//hadoop-2.0.0-mr1-cdh4.1.2-examples.jar:/usr/lib/hadoop-0.20-mapreduce/.//hadoop-test-2.0.0-mr1-cdh4.1.2.jar:/usr/lib/hadoop-0.20-mapreduce/.//hadoop-ant.jar:/usr/lib/hadoop-0.20-mapreduce/.//hadoop-tools.jar:/usr/lib/hadoop-0.20-mapreduce/.//hadoop-2.0.0-mr1-cdh4.1.2-ant.jar:/usr/lib/hadoop-0.20-mapreduce/.//hadoop-ant-2.0.0-mr1-cdh4.1.2.jar:/usr/lib/sqoop/sqoop-1.4.1-cdh4.1.2.jar:/usr/lib/sqoop/sqoop-test-1.4.1-cdh4.1.2.jar::/usr/lib/hadoop-hdfs/./:/usr/lib/hadoop-hdfs/lib/jersey-server-1.8.jar:/usr/lib/hadoop-hdfs/lib/xmlenc-0.52.jar:/usr/lib/hadoop-hdfs/lib/servlet-api-2.5.jar:/usr/lib/hadoop-hdfs/lib/commons-daemon-1.0.3.jar:/usr/lib/hadoop-hdfs/lib/commons-logging-1.1.1.jar:/usr/lib/hadoop-hdfs/lib/commons-cli-1.2.jar:/usr/lib/hadoop-hdfs/lib/guava-11.0.2.jar:/usr/lib/hadoop-hdfs/lib/jline-0.9.94.jar:/usr/lib/hadoop-hdfs/lib/jasper-runtime-5.5.23.jar:/usr/lib/hadoop-hdfs/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-hdfs/lib/jsp-api-2.1.jar:/usr/lib/hadoop-hdfs/lib/log4j-1.2.17.jar:/usr/lib/hadoop-hdfs/lib/jetty-6.1.26.cloudera.2.jar:/usr/lib/hadoop-hdfs/lib/jetty-util-6.1.26.cloudera.2.jar:/usr/lib/hadoop-hdfs/lib/jersey-core-1.8.jar:/usr/lib/hadoop-hdfs/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-hdfs/lib/jsr305-1.3.9.jar:/usr/lib/hadoop-hdfs/lib/zookeeper-3.4.3-cdh4.1.2.jar:/usr/lib/hadoop-hdfs/lib/commons-codec-1.4.jar:/usr/lib/hadoop-hdfs/lib/commons-io-2.1.jar:/usr/lib/hadoop-hdfs/lib/commons-el-1.0.jar:/usr/lib/hadoop-hdfs/lib/commons-lang-2.5.jar:/usr/lib/hadoop-hdfs/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop-hdfs/lib/asm-3.2.jar:/usr/lib/hadoop-hdfs/.//hadoop-hdfs-2.0.0-cdh4.1.2-tests.jar:/usr/lib/hadoop-hdfs/.//hadoop-hdfs.jar:/usr/lib/hadoop-hdfs/.//hadoop-hdfs-2.0.0-cdh4.1.2.jar:/usr/lib/hadoop-yarn/lib/jersey-server-1.8.jar:/usr/lib/hadoop-yarn/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop-yarn/lib/javax.inject-1.jar:/usr/lib/hadoop-yarn/lib/avro-1.7.1.cloudera.2.jar:/usr/lib/hadoop-yarn/lib/guice-3.0.jar:/usr/lib/hadoop-yarn/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-yarn/lib/log4j-1.2.17.jar:/usr/lib/hadoop-yarn/lib/netty-3.2.4.Final.jar:/usr/lib/hadoop-yarn/lib/guice-servlet-3.0.jar:/usr/lib/hadoop-yarn/lib/jersey-guice-1.8.jar:/usr/lib/hadoop-yarn/lib/aopalliance-1.0.jar:/usr/lib/hadoop-yarn/lib/jersey-core-1.8.jar:/usr/lib/hadoop-yarn/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-yarn/lib/commons-io-2.1.jar:/usr/lib/hadoop-yarn/lib/paranamer-2.3.jar:/usr/lib/hadoop-yarn/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop-yarn/lib/asm-3.2.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-applications-distributedshell-2.0.0-cdh4.1.2.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-resourcemanager.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-common-2.0.0-cdh4.1.2.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-resourcemanager-2.0.0-cdh4.1.2.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-nodemanager.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-tests-2.0.0-cdh4.1.2-tests.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-web-proxy.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-tests.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-web-proxy-2.0.0-cdh4.1.2.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-common-2.0.0-cdh4.1.2.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-applications-distributedshell.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-tests-2.0.0-cdh4.1.2.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-nodemanager-2.0.0-cdh4.1.2.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-site-2.0.0-cdh4.1.2.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-site.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-common.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-common.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-api-2.0.0-cdh4.1.2.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-api.jar:/usr/lib/hadoop-0.20-mapreduce/./:/usr/lib/hadoop-0.20-mapreduce/lib/jersey-server-1.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/xmlenc-0.52.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-beanutils-1.7.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/servlet-api-2.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/hadoop-fairscheduler-2.0.0-mr1-cdh4.1.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-jaxrs-1.8.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/aspectjtools-1.6.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-logging-1.1.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/mockito-all-1.8.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/stax-api-1.0.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/hsqldb-1.8.0.10.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jettison-1.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-cli-1.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/guava-11.0.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/kfs-0.2.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/avro-1.7.1.cloudera.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-math-2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jasper-runtime-5.5.23.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/kfs-0.3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-collections-3.2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/activation-1.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/ant-contrib-1.0b3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jasper-compiler-5.5.23.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsp-api-2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/log4j-1.2.17.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jets3t-0.6.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jetty-6.1.26.cloudera.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/aspectjrt-1.6.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jetty-util-6.1.26.cloudera.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-httpclient-3.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/avro-compiler-1.7.1.cloudera.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jersey-json-1.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jersey-core-1.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsch-0.1.42.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-xc-1.8.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/slf4j-api-1.6.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsr305-1.3.9.jar:/usr/lib/hadoop-0.20-mapreduce/lib/junit-4.8.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-net-3.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-codec-1.4.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-io-2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-el-1.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-lang-2.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/paranamer-2.3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-configuration-1.6.jar:/usr/lib/hadoop-0.20-mapreduce/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-digester-1.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/asm-3.2.jar:/usr/lib/hadoop-0.20-mapreduce/.//hadoop-core.jar:/usr/lib/hadoop-0.20-mapreduce/.//hadoop-2.0.0-mr1-cdh4.1.2-test.jar:/usr/lib/hadoop-0.20-mapreduce/.//hadoop-examples.jar:/usr/lib/hadoop-0.20-mapreduce/.//hadoop-tools-2.0.0-mr1-cdh4.1.2.jar:/usr/lib/hadoop-0.20-mapreduce/.//hadoop-2.0.0-mr1-cdh4.1.2-tools.jar:/usr/lib/hadoop-0.20-mapreduce/.//hadoop-core-2.0.0-mr1-cdh4.1.2.jar:/usr/lib/hadoop-0.20-mapreduce/.//hadoop-examples-2.0.0-mr1-cdh4.1.2.jar:/usr/lib/hadoop-0.20-mapreduce/.//hadoop-2.0.0-mr1-cdh4.1.2-core.jar:/usr/lib/hadoop-0.20-mapreduce/.//hadoop-test.jar:/usr/lib/hadoop-0.20-mapreduce/.//hadoop-2.0.0-mr1-cdh4.1.2-examples.jar:/usr/lib/hadoop-0.20-mapreduce/.//hadoop-test-2.0.0-mr1-cdh4.1.2.jar:/usr/lib/hadoop-0.20-mapreduce/.//hadoop-ant.jar:/usr/lib/hadoop-0.20-mapreduce/.//hadoop-tools.jar:/usr/lib/hadoop-0.20-mapreduce/.//hadoop-2.0.0-mr1-cdh4.1.2-ant.jar:/usr/lib/hadoop-0.20-mapreduce/.//hadoop-ant-2.0.0-mr1-cdh4.1.2.jar:/usr/lib/hadoop-0.20-mapreduce/hadoop-core-2.0.0-mr1-cdh4.1.2.jar:/usr/lib/sqoop/sqoop-1.4.1-cdh4.1.2.jar
> Note: /tmp/sqoop-user/compile/485bb295736ca89d6d471d61c3690220/table1.java
> uses or overrides a deprecated API.
> Note: Recompile with -Xlint:deprecation for details.
> 13/02/01 12:54:25 DEBUG orm.CompilationManager: Could not rename
> /tmp/sqoop-user/compile/485bb295736ca89d6d471d61c3690220/table1.java to
> /home/hogarthj/./table1.java
> org.apache.commons.io.FileExistsException: Destination
> '/home/user/./table1.java' already exists
> at org.apache.commons.io.FileUtils.moveFile(FileUtils.java:2378)
> at
> org.apache.sqoop.orm.CompilationManager.compile(CompilationManager.java:227)
> at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:83)
> at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:390)
> at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476)
> at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
> at com.cloudera.sqoop.Sqoop.main(Sqoop.java:57)
> 13/02/01 12:54:25 INFO orm.CompilationManager: Writing jar file:
> /tmp/sqoop-user/compile/485bb295736ca89d6d471d61c3690220/table1.jar
> 13/02/01 12:54:25 DEBUG orm.CompilationManager: Scanning for .class files
> in directory: /tmp/sqoop-user/compile/485bb295736ca89d6d471d61c3690220
> 13/02/01 12:54:25 DEBUG orm.CompilationManager: Got classfile:
> /tmp/sqoop-user/compile/485bb295736ca89d6d471d61c3690220/table1.class ->
> table1.class
> 13/02/01 12:54:25 DEBUG orm.CompilationManager: Finished writing jar file
> /tmp/sqoop-user/compile/485bb295736ca89d6d471d61c3690220/table1.jar
> 13/02/01 12:54:25 DEBUG manager.OracleManager$ConnCache: Caching released
> connection for jdbc:oracle:thin:@database:1525:SID/user
> 13/02/01 12:54:25 DEBUG manager.OracleManager$ConnCache: Got cached
> connection for jdbc:oracle:thin:@database:1525:SID/user
> 13/02/01 12:54:25 INFO manager.OracleManager: Time zone has been set to GMT
> 13/02/01 12:54:25 DEBUG manager.OracleManager$ConnCache: Caching released
> connection for jdbc:oracle:thin:@database:1525:SID/user
> 13/02/01 12:54:25 INFO mapreduce.ImportJobBase: Beginning import of table1
> 13/02/01 12:54:26 DEBUG mapreduce.DataDrivenImportJob: Using table class:
> table1
> 13/02/01 12:54:26 DEBUG mapreduce.DataDrivenImportJob: Using InputFormat:
> class com.cloudera.sqoop.mapreduce.db.OracleDataDrivenDBInputFormat
> 13/02/01 12:54:26 DEBUG mapreduce.JobBase: Adding to job classpath:
> file:/usr/lib/sqoop/sqoop-1.4.1-cdh4.1.2.jar
> 13/02/01 12:54:26 DEBUG mapreduce.JobBase: Adding to job classpath:
> file:/usr/lib/sqoop/lib/ojdbc6.jar
> 13/02/01 12:54:26 DEBUG mapreduce.JobBase: Adding to job classpath:
> file:/usr/lib/sqoop/sqoop-1.4.1-cdh4.1.2.jar
> 13/02/01 12:54:26 DEBUG mapreduce.JobBase: Adding to job classpath:
> file:/usr/lib/sqoop/sqoop-1.4.1-cdh4.1.2.jar
> 13/02/01 12:54:26 DEBUG mapreduce.JobBase: Adding to job classpath:
> file:/usr/lib/sqoop/lib/snappy-java-1.0.4.1.jar
> 13/02/01 12:54:26 DEBUG mapreduce.JobBase: Adding to job classpath:
> file:/usr/lib/sqoop/lib/hsqldb-1.8.0.10.jar
> 13/02/01 12:54:26 DEBUG mapreduce.JobBase: Adding to job classpath:
> file:/usr/lib/sqoop/lib/avro-ipc-1.7.1.cloudera.2.jar
> 13/02/01 12:54:26 DEBUG mapreduce.JobBase: Adding to job classpath:
> file:/usr/lib/sqoop/lib/servlet-api-2.5-20081211.jar
> 13/02/01 12:54:26 DEBUG mapreduce.JobBase: Adding to job classpath:
> file:/usr/lib/sqoop/lib/avro-1.7.1.cloudera.2.jar
> 13/02/01 12:54:26 DEBUG mapreduce.JobBase: Adding to job classpath:
> file:/usr/lib/sqoop/lib/jackson-core-asl-1.8.8.jar
> 13/02/01 12:54:26 DEBUG mapreduce.JobBase: Adding to job classpath:
> file:/usr/lib/sqoop/lib/ant-eclipse-1.0-jvm1.2.jar
> 13/02/01 12:54:26 DEBUG mapreduce.JobBase: Adding to job classpath:
> file:/usr/lib/sqoop/lib/ant-contrib-1.0b3.jar
> 13/02/01 12:54:26 DEBUG mapreduce.JobBase: Adding to job classpath:
> file:/usr/lib/sqoop/lib/commons-io-1.4.jar
> 13/02/01 12:54:26 DEBUG mapreduce.JobBase: Adding to job classpath:
> file:/usr/lib/sqoop/lib/avro-mapred-1.7.1.cloudera.2.jar
> 13/02/01 12:54:26 DEBUG mapreduce.JobBase: Adding to job classpath:
> file:/usr/lib/sqoop/lib/ojdbc6.jar
> 13/02/01 12:54:26 DEBUG mapreduce.JobBase: Adding to job classpath:
> file:/usr/lib/sqoop/lib/netty-3.4.0.Final.jar
> 13/02/01 12:54:26 DEBUG mapreduce.JobBase: Adding to job classpath:
> file:/usr/lib/sqoop/lib/ifxjdbc.jar
> 13/02/01 12:54:26 DEBUG mapreduce.JobBase: Adding to job classpath:
> file:/usr/lib/sqoop/lib/jackson-mapper-asl-1.8.8.jar
> 13/02/01 12:54:26 DEBUG mapreduce.JobBase: Adding to job classpath:
> file:/usr/lib/sqoop/lib/jetty-util-6.1.26.jar
> 13/02/01 12:54:26 DEBUG mapreduce.JobBase: Adding to job classpath:
> file:/usr/lib/sqoop/lib/paranamer-2.3.jar
> 13/02/01 12:54:26 WARN mapred.JobClient: Use GenericOptionsParser for
> parsing the arguments. Applications should implement Tool for the same.
> 13/02/01 12:54:28 INFO mapred.JobClient: Running job: job_201301311050_0193
> 13/02/01 12:54:29 INFO mapred.JobClient:  map 0% reduce 0%
> 13/02/01 12:55:13 INFO mapred.JobClient:  map 100% reduce 0%
> 13/02/01 12:55:14 INFO mapred.JobClient: Job complete: job_201301311050_0193
> 13/02/01 12:55:14 INFO mapred.JobClient: Counters: 23
> 13/02/01 12:55:14 INFO mapred.JobClient:   File System Counters
> 13/02/01 12:55:14 INFO mapred.JobClient:     FILE: Number of bytes read=0
> 13/02/01 12:55:14 INFO mapred.JobClient:     FILE: Number of bytes
> written=170649
> 13/02/01 12:55:14 INFO mapred.JobClient:     FILE: Number of read
> operations=0
> 13/02/01 12:55:14 INFO mapred.JobClient:     FILE: Number of large read
> operations=0
> 13/02/01 12:55:14 INFO mapred.JobClient:     FILE: Number of write
> operations=0
> 13/02/01 12:55:14 INFO mapred.JobClient:     HDFS: Number of bytes read=87
> 13/02/01 12:55:14 INFO mapred.JobClient:     HDFS: Number of bytes
> written=150810094
> 13/02/01 12:55:14 INFO mapred.JobClient:     HDFS: Number of read
> operations=2
> 13/02/01 12:55:14 INFO mapred.JobClient:     HDFS: Number of large read
> operations=0
> 13/02/01 12:55:14 INFO mapred.JobClient:     HDFS: Number of write
> operations=1
> 13/02/01 12:55:14 INFO mapred.JobClient:   Job Counters
> 13/02/01 12:55:14 INFO mapred.JobClient:     Launched map tasks=1
> 13/02/01 12:55:14 INFO mapred.JobClient:     Total time spent by all maps
> in occupied slots (ms)=43762
> 13/02/01 12:55:14 INFO mapred.JobClient:     Total time spent by all
> reduces in occupied slots (ms)=0
> 13/02/01 12:55:14 INFO mapred.JobClient:     Total time spent by all maps
> waiting after reserving slots (ms)=0
> 13/02/01 12:55:14 INFO mapred.JobClient:     Total time spent by all
> reduces waiting after reserving slots (ms)=0
> 13/02/01 12:55:14 INFO mapred.JobClient:   Map-Reduce Framework
> 13/02/01 12:55:14 INFO mapred.JobClient:     Map input records=664595
> 13/02/01 12:55:14 INFO mapred.JobClient:     Map output records=664595
> 13/02/01 12:55:14 INFO mapred.JobClient:     Input split bytes=87
> 13/02/01 12:55:14 INFO mapred.JobClient:     Spilled Records=0
> 13/02/01 12:55:14 INFO mapred.JobClient:     CPU time spent (ms)=18410
> 13/02/01 12:55:14 INFO mapred.JobClient:     Physical memory (bytes)
> snapshot=428912640
> 13/02/01 12:55:14 INFO mapred.JobClient:     Virtual memory (bytes)
> snapshot=1376935936
> 13/02/01 12:55:14 INFO mapred.JobClient:     Total committed heap usage
> (bytes)=658440192
> 13/02/01 12:55:14 INFO mapreduce.ImportJobBase: Transferred 0 bytes in
> 47.9138 seconds (0 bytes/sec)
> 13/02/01 12:55:14 INFO mapreduce.ImportJobBase: Retrieved 664595 records.
> 
> However if I try and get it straight to hive I get the same problem as the
> first with no columns specified and with columns specified the MR to HDFS
> part is as above but then in addition I get:
> 
> 13/02/01 13:08:01 DEBUG hive.HiveImport: Hive.inputTable: table1
> 13/02/01 13:08:01 DEBUG hive.HiveImport: Hive.outputTable: table1_test
> 13/02/01 13:08:01 DEBUG manager.OracleManager: Using column names query:
> SELECT t.* FROM table1 t WHERE 1=0
> 13/02/01 13:08:01 DEBUG manager.OracleManager$ConnCache: Got cached
> connection for jdbc:oracle:thin:@database:1525:SID/user
> 13/02/01 13:08:01 INFO manager.OracleManager: Time zone has been set to GMT
> 13/02/01 13:08:01 DEBUG manager.SqlManager: Using fetchSize for next query:
> 1000
> 13/02/01 13:08:01 INFO manager.SqlManager: Executing SQL statement: SELECT
> t.* FROM table1 t WHERE 1=0
> 13/02/01 13:08:01 ERROR util.SqlTypeMap: It seems like you are looking up a
> column that does not
> 13/02/01 13:08:01 ERROR util.SqlTypeMap: exist in the table. Please ensure
> that you've specified
> 13/02/01 13:08:01 ERROR util.SqlTypeMap: correct column names in Sqoop
> options.
> 13/02/01 13:08:01 ERROR tool.ImportTool: Imported Failed: column not found:
> col1
> 13/02/01 13:08:01 DEBUG manager.OracleManager$ConnCache: Caching released
> connection for jdbc:oracle:thin:@database:1525:SID/user
> 
> Using eval to test the connection/query works fine:
> 
> [me@client ~]$ sqoop eval --connect jdbc:oracle:thin:@database:1525:SID
> --username user --password pass -e 'select t.* from table1 t where 1=0'
> --verbose
> 13/02/01 13:11:17 DEBUG tool.BaseSqoopTool: Enabled debug logging.
> 13/02/01 13:11:17 WARN tool.BaseSqoopTool: Setting your password on the
> command-line is insecure. Consider using -P instead.
> 13/02/01 13:11:17 DEBUG sqoop.ConnFactory: Loaded manager factory:
> com.cloudera.sqoop.manager.DefaultManagerFactory
> 13/02/01 13:11:17 DEBUG sqoop.ConnFactory: Trying ManagerFactory:
> com.cloudera.sqoop.manager.DefaultManagerFactory
> 13/02/01 13:11:17 DEBUG manager.DefaultManagerFactory: Trying with scheme:
> jdbc:oracle:thin:@database:1525
> 13/02/01 13:11:17 DEBUG manager.OracleManager$ConnCache: Instantiated new
> connection cache.
> 13/02/01 13:11:17 INFO manager.SqlManager: Using default fetchSize of 1000
> 13/02/01 13:11:17 DEBUG sqoop.ConnFactory: Instantiated ConnManager
> org.apache.sqoop.manager.OracleManager@16bdb503
> 13/02/01 13:11:17 DEBUG manager.OracleManager: Creating a new connection
> for jdbc:oracle:thin:@database:1525:SID, using username: user
> 13/02/01 13:11:17 DEBUG manager.OracleManager: No connection paramenters
> specified. Using regular API for making connection.
> 13/02/01 13:12:01 INFO manager.OracleManager: Time zone has been set to GMT
> 13/02/01 13:12:01 DEBUG tool.EvalSqlTool: SQL query: select t.* from table1
> t where 1=0
> 13/02/01 13:12:02 DEBUG tool.EvalSqlTool: resultType=true
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> | COL1 | COL2              | COL3        | COL4              | COL5
>  | COL6           | COL7              | COL8          | COL9           |
> COL10        | COL11          | COL12            |
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> 13/02/01 13:12:02 DEBUG tool.EvalSqlTool: resultType=false
> 13/02/01 13:12:02 DEBUG tool.EvalSqlTool: updateCount=-1
> 13/02/01 13:12:02 DEBUG manager.OracleManager$ConnCache: Caching released
> connection for jdbc:oracle:thin:@database:1525:SID/user
> 
> Finally I changed it to have the column names explicitly in capitals and
> the sqoop to hive worked ...
> 
> 13/02/01 13:21:57 DEBUG hive.HiveImport: Hive.inputTable: table1
> 13/02/01 13:21:57 DEBUG hive.HiveImport: Hive.outputTable: table1_test
> 13/02/01 13:21:57 DEBUG manager.OracleManager: Using column names query:
> SELECT t.* FROM table1 t WHERE 1=0
> 13/02/01 13:21:57 DEBUG manager.OracleManager$ConnCache: Got cached
> connection for jdbc:oracle:thin:@database:1525:SID/user
> 13/02/01 13:21:57 INFO manager.OracleManager: Time zone has been set to GMT
> 13/02/01 13:21:57 DEBUG manager.SqlManager: Using fetchSize for next query:
> 1000
> 13/02/01 13:21:57 INFO manager.SqlManager: Executing SQL statement: SELECT
> t.* FROM table1 t WHERE 1=0
> 13/02/01 13:21:57 WARN hive.TableDefWriter: Column COL1 had to be cast to a
> less precise type in Hive
> 13/02/01 13:21:57 WARN hive.TableDefWriter: Column COL2 had to be cast to a
> less precise type in Hive
> 13/02/01 13:21:57 WARN hive.TableDefWriter: Column COL10 had to be cast to
> a less precise type in Hive
> 13/02/01 13:21:57 WARN hive.TableDefWriter: Column COL12 had to be cast to
> a less precise type in Hive
> 13/02/01 13:21:57 DEBUG hive.TableDefWriter: Create statement: CREATE TABLE
> IF NOT EXISTS `table1_test` ( `COL1` STRING, `COL2` DOUBLE, `COL3` STRING,
> `COL4` STRING, `COL5` STRING, `COL6` STRING, `COL7` STRING, `COL8` STRING,
> `COL9` STRING, `COL10` DOUBLE, `COL11` STRING, `COL12` DOUBLE) COMMENT
> 'Imported by sqoop on 2013/02/01 13:21:57' ROW FORMAT DELIMITED FIELDS
> TERMINATED BY '\001' LINES TERMINATED BY '\012' STORED AS TEXTFILE
> 13/02/01 13:21:57 DEBUG hive.TableDefWriter: Load statement: LOAD DATA
> INPATH 'hdfs://nameservice1/user/user/table1' INTO TABLE `table1_test`
> 13/02/01 13:21:57 INFO hive.HiveImport: Removing temporary files from
> import process: hdfs://nameservice1/user/user/table1/_logs
> 13/02/01 13:21:57 INFO hive.HiveImport: Loading uploaded data into Hive
> 13/02/01 13:21:57 DEBUG hive.HiveImport: Using external Hive process.
> 13/02/01 13:22:02 INFO hive.HiveImport: Logging initialized using
> configuration in file:/etc/hive/conf.dist/hive-log4j.properties
> 13/02/01 13:22:02 INFO hive.HiveImport: Hive history
> file=/tmp/user/hive_job_log_user_201302011322_1919095966.txt
> 13/02/01 13:22:10 INFO hive.HiveImport: OK
> 13/02/01 13:22:10 INFO hive.HiveImport: Time taken: 6.624 seconds
> 13/02/01 13:22:10 INFO hive.HiveImport: Loading data to table default.table1
> 13/02/01 13:22:11 INFO hive.HiveImport: OK
> 13/02/01 13:22:11 INFO hive.HiveImport: Time taken: 0.46 seconds
> 13/02/01 13:22:11 INFO hive.HiveImport: Hive import complete.
> 13/02/01 13:22:11 INFO hive.HiveImport: Export directory is empty, removing
> it.
> 13/02/01 13:22:11 DEBUG manager.OracleManager$ConnCache: Caching released
> connection for jdbc:oracle:thin:@database:1525:SID/user
> 
> So I got to a workaround in the end ... but having to explicitly name the
> columns (and the counter intuitive behaviour) is obviously far from ideal
> and the automatic column detection (where the select did as eval revealed
> have the columns in capitals) did not work to pass those in a
> case sensitive manner to the type/existence detection methods... so I guess
> there's a possible bug there...
> 
> James

Mime
View raw message