sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xu, Qian A" <qian.a...@intel.com>
Subject RE: How to debug SQOOP / JDBC Driver Failures for MySQL Import Jobs
Date Wed, 18 Mar 2015 03:29:40 GMT
Hi Traiano,

This is usually a firewall issue of the server xx.yy.xx.zz. Please make sure the xx.yy.xx.zz
:3306 is accessible from where your Sqoop query initiates. 

Stanley


-----Original Message-----
From: Traiano Welcome [mailto:traiano@gmail.com] 
Sent: Tuesday, March 17, 2015 11:02 PM
To: user@sqoop.apache.org
Subject: How to debug SQOOP / JDBC Driver Failures for MySQL Import Jobs

Hi

I'm following the guide for testing Sqoop data imports from mysql:

 http://blog.cloudera.com/blog/2013/11/sqooping-data-with-hue/

But the mysql sqoop job fails with "" in the HUE interface.  I'd appreciate some help interpreting
the sqoop java error trace I'm seeing in the sqoop logs:

My connection settings are as follows:

---
JDBC driver class: com.mysql.jdbc.Driver JDBC Connection string: jdbc:mysql://xx.yy.xx.zz/dbname
---

 I've downloaded and added the jdbc driver from Oracle in /var/lib/sqoop on the sqoop server
node, following the cloudera guide
here:

 http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cdh_ig_jdbc_driver_install.html

However, the job keeps failing with the following java trace from the sqoop logs (excuse the
noise!):


---
2015-03-17 17:44:06,368 INFO
org.apache.sqoop.repository.JdbcRepositoryTransaction: Attempting transaction commit
2015-03-17 17:44:06,385 INFO
org.apache.sqoop.repository.JdbcRepositoryTransaction: Attempting transaction commit
2015-03-17 17:44:06,393 INFO
org.apache.sqoop.repository.JdbcRepositoryTransaction: Attempting transaction commit
2015-03-17 17:45:09,396 ERROR
org.apache.sqoop.server.SqoopProtocolServlet: Exception in POST
http://lol-dev-hdpdn8.hadoop.local:12000/sqoop/v1/submission/action/2
org.apache.sqoop.common.SqoopException:
GENERIC_JDBC_CONNECTOR_0001:Unable to get a connection
        at org.apache.sqoop.connector.jdbc.GenericJdbcExecutor.<init>(GenericJdbcExecutor.java:51)
        at org.apache.sqoop.connector.jdbc.GenericJdbcImportInitializer.configureJdbcProperties(GenericJdbcImportInitializer.java:125)
        at org.apache.sqoop.connector.jdbc.GenericJdbcImportInitializer.initialize(GenericJdbcImportInitializer.java:49)
        at org.apache.sqoop.connector.jdbc.GenericJdbcImportInitializer.initialize(GenericJdbcImportInitializer.java:40)
        at org.apache.sqoop.framework.JobManager.submit(JobManager.java:378)
        at org.apache.sqoop.handler.SubmissionRequestHandler.submissionSubmit(SubmissionRequestHandler.java:152)
        at org.apache.sqoop.handler.SubmissionRequestHandler.handleActionEvent(SubmissionRequestHandler.java:122)
        at org.apache.sqoop.handler.SubmissionRequestHandler.handleEvent(SubmissionRequestHandler.java:75)
        at org.apache.sqoop.server.v1.SubmissionServlet.handlePostRequest(SubmissionServlet.java:44)
        at org.apache.sqoop.server.SqoopProtocolServlet.doPost(SqoopProtocolServlet.java:63)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:643)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:723)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
        at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:861)
        at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:606)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
        at java.lang.Thread.run(Thread.java:744)
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException:
Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not
received any packets from the server.
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:377)
        at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1036)
        at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:338)
        at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2232)
        at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2265)
        at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2064)
        at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:790)
        at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:44)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
.
.
.
---


I've checked for attempts to connect to the remove db server (tcpdump)

I've checked for attempts to connect to the remove db server (tcpdump) but it looks like connection
attempts are not even leaving the sqoop node, so it seems sqoop is not successful at creating
a mysql connection to begin with. I've checked the remote mysql server connection from the
sqoop server using the mysql client as well - no issues there.

I'm using sqoop on cloudera hadoop with cloudera manager 5.2.0 and java java version "1.7.0_45"

Specific package details:

hue-sqoop-3.6.0+cdh5.2.0+509-1.cdh5.2.0.p0.37.el6.x86_64
sqoop2-client-1.99.3+cdh5.2.0+30-1.cdh5.2.0.p0.26.el6.noarch
sqoop2-1.99.3+cdh5.2.0+30-1.cdh5.2.0.p0.26.el6.noarch
sqoop-1.4.5+cdh5.2.0+47-1.cdh5.2.0.p0.26.el6.noarch

I've checked the permissions on the .jar file in /usr/lib/sqoop2, and they seem fine:

---
[root@lol-dev-hdpdn8 sqoop2]# ls -l /var/lib/sqoop2/ total 8680
drwxr-xr-x 4 sqoop2 sqoop    4096 Sep 10  2014 mysql-connector-java-5.1.33
-rw-r--r-- 1 sqoop2 sqoop 3797240 Sep 10  2014 mysql-connector-java-5.1.33.tar.gz
-rwxr-xr-x 1 sqoop2 sqoop  960374 Mar 17 17:36 mysql-connector-java-5.1.34-bin.jar
-rwxr-xr-x 1 sqoop2 sqoop  539705 Mar 17 17:37 postgresql-9.0-801.jdbc4.jar
drwxr-xr-x 3 sqoop2 sqoop    4096 Oct 31 21:02 repository
drwxr-xr-x 3 sqoop2 sqoop    4096 Nov  1 17:07 sqljdbc_4.0
-rwxr-xr-x 1 sqoop2 sqoop 2391092 Nov  1 17:06 sqljdbc_4.0.2206.100_enu.tar.gz -rwxr-xr-x
1 sqoop2 sqoop  584207 Nov  1 17:12 sqljdbc4.jar -rwxr-xr-x 1 sqoop2 sqoop  563117 Nov  1
17:12 sqljdbc.jar
drwxr-xr-x 5 sqoop2 sqoop    4096 Mar 17 17:37 tomcat-deployment
[root@lol-dev-hdpdn8 sqoop2]#

---

I've also tested bot  mysql-connector-java-5.1.34-bin.jar and mysql-connector-java-5.1.33-bin.jar,
restarting the sqoop service each time I changed the module, to no avail.

Is there anyway to debug further into why the job is failing?

Many thanks in advance for any help!

Traiano
Mime
View raw message