sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Traiano Welcome <trai...@gmail.com>
Subject How to debug SQOOP / JDBC Driver Failures for MySQL Import Jobs
Date Tue, 17 Mar 2015 15:01:33 GMT
Hi

I'm following the guide for testing Sqoop data imports from mysql:

 http://blog.cloudera.com/blog/2013/11/sqooping-data-with-hue/

But the mysql sqoop job fails with "" in the HUE interface.  I'd
appreciate some help interpreting the sqoop java error trace I'm
seeing in the sqoop logs:

My connection settings are as follows:

---
JDBC driver class: com.mysql.jdbc.Driver
JDBC Connection string: jdbc:mysql://xx.yy.xx.zz/dbname
---

 I've downloaded and added the jdbc driver from Oracle in
/var/lib/sqoop on the sqoop server node, following the cloudera guide
here:

 http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cdh_ig_jdbc_driver_install.html

However, the job keeps failing with the following java trace from the
sqoop logs (excuse the noise!):


---
2015-03-17 17:44:06,368 INFO
org.apache.sqoop.repository.JdbcRepositoryTransaction: Attempting
transaction commit
2015-03-17 17:44:06,385 INFO
org.apache.sqoop.repository.JdbcRepositoryTransaction: Attempting
transaction commit
2015-03-17 17:44:06,393 INFO
org.apache.sqoop.repository.JdbcRepositoryTransaction: Attempting
transaction commit
2015-03-17 17:45:09,396 ERROR
org.apache.sqoop.server.SqoopProtocolServlet: Exception in POST
http://lol-dev-hdpdn8.hadoop.local:12000/sqoop/v1/submission/action/2
org.apache.sqoop.common.SqoopException:
GENERIC_JDBC_CONNECTOR_0001:Unable to get a connection
        at org.apache.sqoop.connector.jdbc.GenericJdbcExecutor.<init>(GenericJdbcExecutor.java:51)
        at org.apache.sqoop.connector.jdbc.GenericJdbcImportInitializer.configureJdbcProperties(GenericJdbcImportInitializer.java:125)
        at org.apache.sqoop.connector.jdbc.GenericJdbcImportInitializer.initialize(GenericJdbcImportInitializer.java:49)
        at org.apache.sqoop.connector.jdbc.GenericJdbcImportInitializer.initialize(GenericJdbcImportInitializer.java:40)
        at org.apache.sqoop.framework.JobManager.submit(JobManager.java:378)
        at org.apache.sqoop.handler.SubmissionRequestHandler.submissionSubmit(SubmissionRequestHandler.java:152)
        at org.apache.sqoop.handler.SubmissionRequestHandler.handleActionEvent(SubmissionRequestHandler.java:122)
        at org.apache.sqoop.handler.SubmissionRequestHandler.handleEvent(SubmissionRequestHandler.java:75)
        at org.apache.sqoop.server.v1.SubmissionServlet.handlePostRequest(SubmissionServlet.java:44)
        at org.apache.sqoop.server.SqoopProtocolServlet.doPost(SqoopProtocolServlet.java:63)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:643)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:723)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
        at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:861)
        at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:606)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
        at java.lang.Thread.run(Thread.java:744)
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException:
Communications link failure

The last packet sent successfully to the server was 0 milliseconds
ago. The driver has not received any packets from the server.
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:377)
        at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1036)
        at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:338)
        at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2232)
        at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2265)
        at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2064)
        at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:790)
        at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:44)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
.
.
.
---


I've checked for attempts to connect to the remove db server (tcpdump)

I've checked for attempts to connect to the remove db server (tcpdump)
but it looks like connection attempts are not even leaving the sqoop
node, so it seems sqoop is not successful at creating a mysql
connection to begin with. I've checked the remote mysql server
connection from the sqoop server using the mysql client as well - no
issues there.

I'm using sqoop on cloudera hadoop with cloudera manager 5.2.0 and
java java version "1.7.0_45"

Specific package details:

hue-sqoop-3.6.0+cdh5.2.0+509-1.cdh5.2.0.p0.37.el6.x86_64
sqoop2-client-1.99.3+cdh5.2.0+30-1.cdh5.2.0.p0.26.el6.noarch
sqoop2-1.99.3+cdh5.2.0+30-1.cdh5.2.0.p0.26.el6.noarch
sqoop-1.4.5+cdh5.2.0+47-1.cdh5.2.0.p0.26.el6.noarch

I've checked the permissions on the .jar file in /usr/lib/sqoop2, and
they seem fine:

---
[root@lol-dev-hdpdn8 sqoop2]# ls -l /var/lib/sqoop2/
total 8680
drwxr-xr-x 4 sqoop2 sqoop    4096 Sep 10  2014 mysql-connector-java-5.1.33
-rw-r--r-- 1 sqoop2 sqoop 3797240 Sep 10  2014
mysql-connector-java-5.1.33.tar.gz
-rwxr-xr-x 1 sqoop2 sqoop  960374 Mar 17 17:36
mysql-connector-java-5.1.34-bin.jar
-rwxr-xr-x 1 sqoop2 sqoop  539705 Mar 17 17:37 postgresql-9.0-801.jdbc4.jar
drwxr-xr-x 3 sqoop2 sqoop    4096 Oct 31 21:02 repository
drwxr-xr-x 3 sqoop2 sqoop    4096 Nov  1 17:07 sqljdbc_4.0
-rwxr-xr-x 1 sqoop2 sqoop 2391092 Nov  1 17:06 sqljdbc_4.0.2206.100_enu.tar.gz
-rwxr-xr-x 1 sqoop2 sqoop  584207 Nov  1 17:12 sqljdbc4.jar
-rwxr-xr-x 1 sqoop2 sqoop  563117 Nov  1 17:12 sqljdbc.jar
drwxr-xr-x 5 sqoop2 sqoop    4096 Mar 17 17:37 tomcat-deployment
[root@lol-dev-hdpdn8 sqoop2]#

---

I've also tested bot  mysql-connector-java-5.1.34-bin.jar and
mysql-connector-java-5.1.33-bin.jar, restarting the sqoop service each
time I changed the module, to no avail.

Is there anyway to debug further into why the job is failing?

Many thanks in advance for any help!

Traiano

Mime
View raw message