sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jarek Jarcec Cecho <jar...@apache.org>
Subject Re: Java to Sqoop commands
Date Wed, 06 Mar 2013 19:42:22 GMT
Hi Cyril,
would you mind rerunning the command with --verbose argument? I would like to see what factories
is Sqoop trying to use.

Jarcec

On Tue, Mar 05, 2013 at 12:21:44PM -0500, Cyril Bogus wrote:
> Hi Jarek, I have managed to get it to internet. Here is the output with the
> content of the $CLASSPATH passed.
> 
> owner@owner-7:~/Desktop$ $CLASSPATH
> bash:
> .:/home/owner/hadoop/hadoop-core-1.0.4.jar:/home/owner/hive/conf:/home/owner/hive/lib/antlr-runtime-3.0.1.jar:/home/owner/hive/lib/commons-cli-1.2.jar:/home/owner/hive/lib/commons-codec-1.3.jar:/home/owner/hive/lib/commons-collections-3.2.1.jar:/home/owner/hive/lib/commons-dbcp-1.4.jar:/home/owner/hive/lib/commons-lang-2.4.jar:/home/owner/hive/lib/commons-logging-1.0.4.jar:/home/owner/hive/lib/commons-logging-api-1.0.4.jar:/home/owner/hive/lib/commons-pool-1.5.4.jar:/home/owner/hive/lib/datanucleus-connectionpool-2.0.3.jar:/home/owner/hive/lib/datanucleus-core-2.0.3.jar:/home/owner/hive/lib/datanucleus-enhancer-2.0.3.jar:/home/owner/hive/lib/datanucleus-rdbms-2.0.3.jar:/home/owner/hive/lib/derby-10.4.2.0.jar:/home/owner/hive/lib/guava-r09.jar:/home/owner/hive/lib/hbase-0.92.0.jar:/home/owner/hive/lib/hbase-0.92.0-tests.jar:/home/owner/hive/lib/hive-builtins-0.9.0.jar:/home/owner/hive/lib/hive-cli-0.9.0.jar:/home/owner/hive/lib/hive-common-0.9.0.jar:/home/owner/hive/lib/hive-contrib-0.9.0.jar:/home/owner/hive/lib/hive_contrib.jar:/home/owner/hive/lib/hive-exec-0.9.0.jar:/home/owner/hive/lib/hive-hbase-handler-0.9.0.jar:/home/owner/hive/lib/hive-hwi-0.9.0.jar:/home/owner/hive/lib/hive-jdbc-0.9.0.jar:/home/owner/hive/lib/hive-metastore-0.9.0.jar:/home/owner/hive/lib/hive-pdk-0.9.0.jar:/home/owner/hive/lib/hive-serde-0.9.0.jar:/home/owner/hive/lib/hive-service-0.9.0.jar:/home/owner/hive/lib/hive-shims-0.9.0.jar:/home/owner/hive/lib/jackson-core-asl-1.8.8.jar:/home/owner/hive/lib/jackson-jaxrs-1.8.8.jar:/home/owner/hive/lib/jackson-mapper-asl-1.8.8.jar:/home/owner/hive/lib/jackson-xc-1.8.8.jar:/home/owner/hive/lib/JavaEWAH-0.3.2.jar:/home/owner/hive/lib/jdo2-api-2.3-ec.jar:/home/owner/hive/lib/jline-0.9.94.jar:/home/owner/hive/lib/json-20090211.jar:/home/owner/hive/lib/libfb303-0.7.0.jar:/home/owner/hive/lib/libfb303.jar:/home/owner/hive/lib/libthrift-0.7.0.jar:/home/owner/hive/lib/libthrift.jar:/home/owner/hive/lib/log4j-1.2.16.jar:/home/owner/hive/lib/mysql-connector-java-5.1.23-bin.jar:/home/owner/hive/lib/slf4j-api-1.6.1.jar:/home/owner/hive/lib/slf4j-log4j12-1.6.1.jar:/home/owner/hive/lib/sqljdbc4.jar:/home/owner/hive/lib/stringtemplate-3.1-b1.jar:/home/owner/hive/lib/zookeeper-3.4.3.jar:/home/owner/hadoop/lib/asm-3.2.jar:/home/owner/hadoop/lib/aspectjrt-1.6.5.jar:/home/owner/hadoop/lib/aspectjtools-1.6.5.jar:/home/owner/hadoop/lib/commons-beanutils-1.7.0.jar:/home/owner/hadoop/lib/commons-beanutils-core-1.8.0.jar:/home/owner/hadoop/lib/commons-cli-1.2.jar:/home/owner/hadoop/lib/commons-codec-1.4.jar:/home/owner/hadoop/lib/commons-collections-3.2.1.jar:/home/owner/hadoop/lib/commons-configuration-1.6.jar:/home/owner/hadoop/lib/commons-daemon-1.0.1.jar:/home/owner/hadoop/lib/commons-digester-1.8.jar:/home/owner/hadoop/lib/commons-el-1.0.jar:/home/owner/hadoop/lib/commons-httpclient-3.0.1.jar:/home/owner/hadoop/lib/commons-io-2.1.jar:/home/owner/hadoop/lib/commons-lang-2.4.jar:/home/owner/hadoop/lib/commons-logging-1.1.1.jar:/home/owner/hadoop/lib/commons-logging-api-1.0.4.jar:/home/owner/hadoop/lib/commons-math-2.1.jar:/home/owner/hadoop/lib/commons-net-1.4.1.jar:/home/owner/hadoop/lib/core-3.1.1.jar:/home/owner/hadoop/lib/hadoop-capacity-scheduler-1.0.4.jar:/home/owner/hadoop/lib/hadoop-fairscheduler-1.0.4.jar:/home/owner/hadoop/lib/hadoop-thriftfs-1.0.4.jar:/home/owner/hadoop/lib/hsqldb-1.8.0.10.jar:/home/owner/hadoop/lib/jackson-core-asl-1.8.8.jar:/home/owner/hadoop/lib/jackson-mapper-asl-1.8.8.jar:/home/owner/hadoop/lib/jasper-compiler-5.5.12.jar:/home/owner/hadoop/lib/jasper-runtime-5.5.12.jar:/home/owner/hadoop/lib/jdeb-0.8.jar:/home/owner/hadoop/lib/jersey-core-1.8.jar:/home/owner/hadoop/lib/jersey-json-1.8.jar:/home/owner/hadoop/lib/jersey-server-1.8.jar:/home/owner/hadoop/lib/jets3t-0.6.1.jar:/home/owner/hadoop/lib/jetty-6.1.26.jar:/home/owner/hadoop/lib/jetty-util-6.1.26.jar:/home/owner/hadoop/lib/jsch-0.1.42.jar:/home/owner/hadoop/lib/junit-4.5.jar:/home/owner/hadoop/lib/kfs-0.2.2.jar:/home/owner/hadoop/lib/log4j-1.2.15.jar:/home/owner/hadoop/lib/mockito-all-1.8.5.jar:/home/owner/hadoop/lib/oro-2.0.8.jar:/home/owner/hadoop/lib/servlet-api-2.5-20081211.jar:/home/owner/hadoop/lib/slf4j-api-1.4.3.jar:/home/owner/hadoop/lib/slf4j-log4j12-1.4.3.jar:/home/owner/hadoop/lib/xmlenc-0.52.jar:/home/owner/sqoop/lib/ant-contrib-1.0b3.jar:/home/owner/sqoop/lib/ant-eclipse-1.0-jvm1.2.jar:/home/owner/sqoop/lib/avro-1.5.3.jar:/home/owner/sqoop/lib/avro-ipc-1.5.3.jar:/home/owner/sqoop/lib/avro-mapred-1.5.3.jar:/home/owner/sqoop/lib/commons-io-1.4.jar:/home/owner/sqoop/lib/hsqldb-1.8.0.10.jar:/home/owner/sqoop/lib/jackson-core-asl-1.7.3.jar:/home/owner/sqoop/lib/jackson-mapper-asl-1.7.3.jar:/home/owner/sqoop/lib/jopt-simple-3.2.jar:/home/owner/sqoop/lib/paranamer-2.3.jar:/home/owner/sqoop/lib/snappy-java-1.0.3.2.jar:/home/owner/sqoop/lib/sqljdbc4.jar:/home/owner/sqoop/lib/sqoop-sqlserver-1.0.jar:/home/owner/trunk/lib/*.jar:
> File name too long
> owner@owner-7:~/Desktop$ javac Scheduler.java
> owner@owner-7:~/Desktop$ java -cp $CLASSPATH Scheduler
> /home/owner/sqoop/bin/sqoop import --connect
> 'jdbc:sqlserver://111.111.111.111:1111;username=sa;password=owner%2007;database=database'
> --table minedrive_driver_scoring_table --hive-import --hive-overwrite
> Warning: $HADOOP_HOME is deprecated.
> 
> 13/03/05 12:16:51 INFO tool.BaseSqoopTool: Using Hive-specific delimiters
> for output. You can override
> 13/03/05 12:16:51 INFO tool.BaseSqoopTool: delimiters with
> --fields-terminated-by, etc.
> 13/03/05 12:16:53 ERROR tool.BaseSqoopTool: Got error creating database
> manager: java.io.IOException: No manager for connect string:
> 'jdbc:sqlserver://111.111.111.111:1111
> ;username=sa;password=owner%2007;database=database'
>     at org.apache.sqoop.ConnFactory.getManager(ConnFactory.java:185)
>     at org.apache.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:200)
>     at org.apache.sqoop.tool.ImportTool.init(ImportTool.java:83)
>     at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:464)
>     at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
>     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>     at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
>     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
>     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
>     at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
>     at com.cloudera.sqoop.Sqoop.main(Sqoop.java:57)
> 
> Process exitValue: 1
> PROGRAM DONE
> owner@owner-7:~/Desktop$
> 
> 
> On Mon, Mar 4, 2013 at 11:07 AM, Jarek Jarcec Cecho <jarcec@apache.org>wrote:
> 
> > Hi Cyril,
> > I do understand that your second computer might not be connector to the
> > internet, however it's very hard to help you without logs. Would you mind
> > saving entire generated log with the exception and your classpath into file
> > and move it to a computer where you have internet connectivity and share
> > that with us? (floppy, usb stick, ...)
> >
> > Jarcec
> >
> > On Sat, Mar 02, 2013 at 01:55:00PM -0800, Venkat wrote:
> > > It looks like your JDBC spec is wrong.   Can you  please make sure for
> > > SQLServer your connect string is of the form
> > >
> > jdbc:sqlserver://[serverName[\instanceName][:portNumber]][;property=value[;property=value]]
> > >
> > > See http://msdn.microsoft.com/en-us/library/ms378428(v=sql.90).aspx
> > >
> > >
> > > On Sat, Mar 2, 2013 at 1:33 PM, Cyril Bogus <cyrilbogus@gmail.com>
> > wrote:
> > >
> > > > Hi Jarek, thank you for your reply.
> > > >
> > > > The computer On which I run the job is not connected to the internet so
> > > > all I can do is describe the output very briefly.
> > > >
> > > > And by the way I am running Sqoop 1.4.2
> > > >
> > > > Following your advice I have done the following
> > > >
> > > > javac Importer.java
> > > >
> > > > java -cp $CLASSPATH Importer
> > > >
> > > > CLASSPATH contains all libraries for sqoop, hive and Hadoop
> > > > this is a brief description of the output:
> > > >
> > > >
> > > > Using Hive Specific Import ....
> > > > delimiters with --field-terminated by, etc.
> > > > Got error creating database manager: java.io.IOException. No manager
> > for
> > > > connect string 'myserver:port/database;user;pass'
> > > >
> > > >
> > > > Again the classpath contains all the jars AND when I just copy and
> > paste
> > > > the same command line that gave me an error in the program as a linux
> > > > command everything runs smoothly to the end.
> > > >
> > > >
> > > >
> > > > On Sat, Mar 2, 2013 at 3:28 PM, Jarek Jarcec Cecho <jarcec@apache.org
> > >wrote:
> > > >
> > > >> Hi Cyril,
> > > >> when executing Sqoop from within your Java application, do you have
> > all
> > > >> the jars (connector, JDBC Driver) in the classpath? Also it would
be
> > > >> helpful if you could share exactly the error message that you're
> > getting.
> > > >>
> > > >> Jarcec
> > > >>
> > > >> On Sat, Mar 02, 2013 at 03:11:40PM -0500, Cyril Bogus wrote:
> > > >> > So, I am running a java program to import data from a sqlserver
> > using
> > > >> sqoop.
> > > >> >
> > > >> > I have the Microsoft SQL - Hadoop connector in the lib and whenever
> > I
> > > >> run
> > > >> > commands from the command line the import runs fine.
> > > >> >
> > > >> > Problem is when my Java is the one trying to run the command
then
> > sqoop
> > > >> > can't find the lib somehow so it does not connect to the DB.
> > > >> > I try running the exact same query directly in command line and
> > > >> everything
> > > >> > works fine.
> > > >> >
> > > >> > Is there a way where I can define both the driver and connection
> > manager
> > > >> > through Java as not to worry about it not finding the proper
> > manager for
> > > >> > import?
> > > >> >
> > > >> > Regards
> > > >> >
> > > >> > Cyril
> > > >>
> > > >
> > > >
> > >
> > >
> > > --
> > > Regards
> > >
> > > Venkat
> >

Mime
View raw message