sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cyril Bogus <cyrilbo...@gmail.com>
Subject Re: Java to Sqoop commands
Date Tue, 05 Mar 2013 17:21:44 GMT
Hi Jarek, I have managed to get it to internet. Here is the output with the
content of the $CLASSPATH passed.

owner@owner-7:~/Desktop$ $CLASSPATH
bash:
.:/home/owner/hadoop/hadoop-core-1.0.4.jar:/home/owner/hive/conf:/home/owner/hive/lib/antlr-runtime-3.0.1.jar:/home/owner/hive/lib/commons-cli-1.2.jar:/home/owner/hive/lib/commons-codec-1.3.jar:/home/owner/hive/lib/commons-collections-3.2.1.jar:/home/owner/hive/lib/commons-dbcp-1.4.jar:/home/owner/hive/lib/commons-lang-2.4.jar:/home/owner/hive/lib/commons-logging-1.0.4.jar:/home/owner/hive/lib/commons-logging-api-1.0.4.jar:/home/owner/hive/lib/commons-pool-1.5.4.jar:/home/owner/hive/lib/datanucleus-connectionpool-2.0.3.jar:/home/owner/hive/lib/datanucleus-core-2.0.3.jar:/home/owner/hive/lib/datanucleus-enhancer-2.0.3.jar:/home/owner/hive/lib/datanucleus-rdbms-2.0.3.jar:/home/owner/hive/lib/derby-10.4.2.0.jar:/home/owner/hive/lib/guava-r09.jar:/home/owner/hive/lib/hbase-0.92.0.jar:/home/owner/hive/lib/hbase-0.92.0-tests.jar:/home/owner/hive/lib/hive-builtins-0.9.0.jar:/home/owner/hive/lib/hive-cli-0.9.0.jar:/home/owner/hive/lib/hive-common-0.9.0.jar:/home/owner/hive/lib/hive-contrib-0.9.0.jar:/home/owner/hive/lib/hive_contrib.jar:/home/owner/hive/lib/hive-exec-0.9.0.jar:/home/owner/hive/lib/hive-hbase-handler-0.9.0.jar:/home/owner/hive/lib/hive-hwi-0.9.0.jar:/home/owner/hive/lib/hive-jdbc-0.9.0.jar:/home/owner/hive/lib/hive-metastore-0.9.0.jar:/home/owner/hive/lib/hive-pdk-0.9.0.jar:/home/owner/hive/lib/hive-serde-0.9.0.jar:/home/owner/hive/lib/hive-service-0.9.0.jar:/home/owner/hive/lib/hive-shims-0.9.0.jar:/home/owner/hive/lib/jackson-core-asl-1.8.8.jar:/home/owner/hive/lib/jackson-jaxrs-1.8.8.jar:/home/owner/hive/lib/jackson-mapper-asl-1.8.8.jar:/home/owner/hive/lib/jackson-xc-1.8.8.jar:/home/owner/hive/lib/JavaEWAH-0.3.2.jar:/home/owner/hive/lib/jdo2-api-2.3-ec.jar:/home/owner/hive/lib/jline-0.9.94.jar:/home/owner/hive/lib/json-20090211.jar:/home/owner/hive/lib/libfb303-0.7.0.jar:/home/owner/hive/lib/libfb303.jar:/home/owner/hive/lib/libthrift-0.7.0.jar:/home/owner/hive/lib/libthrift.jar:/home/owner/hive/lib/log4j-1.2.16.jar:/home/owner/hive/lib/mysql-connector-java-5.1.23-bin.jar:/home/owner/hive/lib/slf4j-api-1.6.1.jar:/home/owner/hive/lib/slf4j-log4j12-1.6.1.jar:/home/owner/hive/lib/sqljdbc4.jar:/home/owner/hive/lib/stringtemplate-3.1-b1.jar:/home/owner/hive/lib/zookeeper-3.4.3.jar:/home/owner/hadoop/lib/asm-3.2.jar:/home/owner/hadoop/lib/aspectjrt-1.6.5.jar:/home/owner/hadoop/lib/aspectjtools-1.6.5.jar:/home/owner/hadoop/lib/commons-beanutils-1.7.0.jar:/home/owner/hadoop/lib/commons-beanutils-core-1.8.0.jar:/home/owner/hadoop/lib/commons-cli-1.2.jar:/home/owner/hadoop/lib/commons-codec-1.4.jar:/home/owner/hadoop/lib/commons-collections-3.2.1.jar:/home/owner/hadoop/lib/commons-configuration-1.6.jar:/home/owner/hadoop/lib/commons-daemon-1.0.1.jar:/home/owner/hadoop/lib/commons-digester-1.8.jar:/home/owner/hadoop/lib/commons-el-1.0.jar:/home/owner/hadoop/lib/commons-httpclient-3.0.1.jar:/home/owner/hadoop/lib/commons-io-2.1.jar:/home/owner/hadoop/lib/commons-lang-2.4.jar:/home/owner/hadoop/lib/commons-logging-1.1.1.jar:/home/owner/hadoop/lib/commons-logging-api-1.0.4.jar:/home/owner/hadoop/lib/commons-math-2.1.jar:/home/owner/hadoop/lib/commons-net-1.4.1.jar:/home/owner/hadoop/lib/core-3.1.1.jar:/home/owner/hadoop/lib/hadoop-capacity-scheduler-1.0.4.jar:/home/owner/hadoop/lib/hadoop-fairscheduler-1.0.4.jar:/home/owner/hadoop/lib/hadoop-thriftfs-1.0.4.jar:/home/owner/hadoop/lib/hsqldb-1.8.0.10.jar:/home/owner/hadoop/lib/jackson-core-asl-1.8.8.jar:/home/owner/hadoop/lib/jackson-mapper-asl-1.8.8.jar:/home/owner/hadoop/lib/jasper-compiler-5.5.12.jar:/home/owner/hadoop/lib/jasper-runtime-5.5.12.jar:/home/owner/hadoop/lib/jdeb-0.8.jar:/home/owner/hadoop/lib/jersey-core-1.8.jar:/home/owner/hadoop/lib/jersey-json-1.8.jar:/home/owner/hadoop/lib/jersey-server-1.8.jar:/home/owner/hadoop/lib/jets3t-0.6.1.jar:/home/owner/hadoop/lib/jetty-6.1.26.jar:/home/owner/hadoop/lib/jetty-util-6.1.26.jar:/home/owner/hadoop/lib/jsch-0.1.42.jar:/home/owner/hadoop/lib/junit-4.5.jar:/home/owner/hadoop/lib/kfs-0.2.2.jar:/home/owner/hadoop/lib/log4j-1.2.15.jar:/home/owner/hadoop/lib/mockito-all-1.8.5.jar:/home/owner/hadoop/lib/oro-2.0.8.jar:/home/owner/hadoop/lib/servlet-api-2.5-20081211.jar:/home/owner/hadoop/lib/slf4j-api-1.4.3.jar:/home/owner/hadoop/lib/slf4j-log4j12-1.4.3.jar:/home/owner/hadoop/lib/xmlenc-0.52.jar:/home/owner/sqoop/lib/ant-contrib-1.0b3.jar:/home/owner/sqoop/lib/ant-eclipse-1.0-jvm1.2.jar:/home/owner/sqoop/lib/avro-1.5.3.jar:/home/owner/sqoop/lib/avro-ipc-1.5.3.jar:/home/owner/sqoop/lib/avro-mapred-1.5.3.jar:/home/owner/sqoop/lib/commons-io-1.4.jar:/home/owner/sqoop/lib/hsqldb-1.8.0.10.jar:/home/owner/sqoop/lib/jackson-core-asl-1.7.3.jar:/home/owner/sqoop/lib/jackson-mapper-asl-1.7.3.jar:/home/owner/sqoop/lib/jopt-simple-3.2.jar:/home/owner/sqoop/lib/paranamer-2.3.jar:/home/owner/sqoop/lib/snappy-java-1.0.3.2.jar:/home/owner/sqoop/lib/sqljdbc4.jar:/home/owner/sqoop/lib/sqoop-sqlserver-1.0.jar:/home/owner/trunk/lib/*.jar:
File name too long
owner@owner-7:~/Desktop$ javac Scheduler.java
owner@owner-7:~/Desktop$ java -cp $CLASSPATH Scheduler
/home/owner/sqoop/bin/sqoop import --connect
'jdbc:sqlserver://111.111.111.111:1111;username=sa;password=owner%2007;database=database'
--table minedrive_driver_scoring_table --hive-import --hive-overwrite
Warning: $HADOOP_HOME is deprecated.

13/03/05 12:16:51 INFO tool.BaseSqoopTool: Using Hive-specific delimiters
for output. You can override
13/03/05 12:16:51 INFO tool.BaseSqoopTool: delimiters with
--fields-terminated-by, etc.
13/03/05 12:16:53 ERROR tool.BaseSqoopTool: Got error creating database
manager: java.io.IOException: No manager for connect string:
'jdbc:sqlserver://111.111.111.111:1111
;username=sa;password=owner%2007;database=database'
    at org.apache.sqoop.ConnFactory.getManager(ConnFactory.java:185)
    at org.apache.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:200)
    at org.apache.sqoop.tool.ImportTool.init(ImportTool.java:83)
    at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:464)
    at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
    at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
    at com.cloudera.sqoop.Sqoop.main(Sqoop.java:57)

Process exitValue: 1
PROGRAM DONE
owner@owner-7:~/Desktop$


On Mon, Mar 4, 2013 at 11:07 AM, Jarek Jarcec Cecho <jarcec@apache.org>wrote:

> Hi Cyril,
> I do understand that your second computer might not be connector to the
> internet, however it's very hard to help you without logs. Would you mind
> saving entire generated log with the exception and your classpath into file
> and move it to a computer where you have internet connectivity and share
> that with us? (floppy, usb stick, ...)
>
> Jarcec
>
> On Sat, Mar 02, 2013 at 01:55:00PM -0800, Venkat wrote:
> > It looks like your JDBC spec is wrong.   Can you  please make sure for
> > SQLServer your connect string is of the form
> >
> jdbc:sqlserver://[serverName[\instanceName][:portNumber]][;property=value[;property=value]]
> >
> > See http://msdn.microsoft.com/en-us/library/ms378428(v=sql.90).aspx
> >
> >
> > On Sat, Mar 2, 2013 at 1:33 PM, Cyril Bogus <cyrilbogus@gmail.com>
> wrote:
> >
> > > Hi Jarek, thank you for your reply.
> > >
> > > The computer On which I run the job is not connected to the internet so
> > > all I can do is describe the output very briefly.
> > >
> > > And by the way I am running Sqoop 1.4.2
> > >
> > > Following your advice I have done the following
> > >
> > > javac Importer.java
> > >
> > > java -cp $CLASSPATH Importer
> > >
> > > CLASSPATH contains all libraries for sqoop, hive and Hadoop
> > > this is a brief description of the output:
> > >
> > >
> > > Using Hive Specific Import ....
> > > delimiters with --field-terminated by, etc.
> > > Got error creating database manager: java.io.IOException. No manager
> for
> > > connect string 'myserver:port/database;user;pass'
> > >
> > >
> > > Again the classpath contains all the jars AND when I just copy and
> paste
> > > the same command line that gave me an error in the program as a linux
> > > command everything runs smoothly to the end.
> > >
> > >
> > >
> > > On Sat, Mar 2, 2013 at 3:28 PM, Jarek Jarcec Cecho <jarcec@apache.org
> >wrote:
> > >
> > >> Hi Cyril,
> > >> when executing Sqoop from within your Java application, do you have
> all
> > >> the jars (connector, JDBC Driver) in the classpath? Also it would be
> > >> helpful if you could share exactly the error message that you're
> getting.
> > >>
> > >> Jarcec
> > >>
> > >> On Sat, Mar 02, 2013 at 03:11:40PM -0500, Cyril Bogus wrote:
> > >> > So, I am running a java program to import data from a sqlserver
> using
> > >> sqoop.
> > >> >
> > >> > I have the Microsoft SQL - Hadoop connector in the lib and whenever
> I
> > >> run
> > >> > commands from the command line the import runs fine.
> > >> >
> > >> > Problem is when my Java is the one trying to run the command then
> sqoop
> > >> > can't find the lib somehow so it does not connect to the DB.
> > >> > I try running the exact same query directly in command line and
> > >> everything
> > >> > works fine.
> > >> >
> > >> > Is there a way where I can define both the driver and connection
> manager
> > >> > through Java as not to worry about it not finding the proper
> manager for
> > >> > import?
> > >> >
> > >> > Regards
> > >> >
> > >> > Cyril
> > >>
> > >
> > >
> >
> >
> > --
> > Regards
> >
> > Venkat
>

Mime
View raw message