sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Traiano Welcome <trai...@gmail.com>
Subject Re: How to debug SQOOP / JDBC Driver Failures for MySQL Import Jobs
Date Thu, 19 Mar 2015 06:19:29 GMT
Hi Abe


 Thanks for the feedback.

On Thu, Mar 19, 2015 at 3:17 AM, Abraham Elmahrek <abe@cloudera.com> wrote:
> Which version of Sqoop are you trying to use? See
> http://ingest.tips/2014/10/21/sqoop-1-or-sqoop-2/ for details on that.
>
> With Hue, you'll be using Sqoop2 (which is being actively developed). The
> command line snippet you posted is of Sqoop1. Here's a blog post
> highlighting features of Sqoop2 and Sqoop1:
> https://blogs.apache.org/sqoop/entry/apache_sqoop_highlights_of_sqoop.
>
> Any ways, the error above is a connection error
> (http://stackoverflow.com/questions/2983248/com-mysql-jdbc-exceptions-jdbc4-communicationsexception-communications-link-fai).
> Given network connectivity might not be the issue, please make sure you're
> using the correct JDBC string. I'm highlighting this because the Sqoop1
> command you gave has a different JDBC URL than the Sqoop2 configuration.
> Underneath, Sqoop2 is using JDBC to connect to MySQL (via the Generic JDBC
> Connector). It should literally follow the same semantics and have the same
> parameters as a normal JDBC string.
>
> -Abe


These are the sqoop versions installed:

---
hue-sqoop-3.6.0+cdh5.2.0+509-1.cdh5.2.0.p0.37.el6.x86_64
sqoop2-client-1.99.3+cdh5.2.0+30-1.cdh5.2.0.p0.26.el6.noarch
sqoop2-1.99.3+cdh5.2.0+30-1.cdh5.2.0.p0.26.el6.noarch
sqoop-1.4.5+cdh5.2.0+47-1.cdh5.2.0.p0.26.el6.noarch
---

I've tried with the sqoop2 shell, as follows:

---

set server --host <sqoop2_host>
show connector --all
create connection --cid 1
JDBC Driver Class: com.mysql.jdbc.Driver
JDBC Connection String: jdbc:mysql://<db_server>/DB
Username:<db_username>
Password: <db_password>

---

sqoop:000> create connection --cid 1
Creating connection for connector with id 1
Please fill following values to create new connection object
Name: mysql-ingestion

Connection configuration

JDBC Driver Class: com.mysql.jdbc.Driver
JDBC Connection String: jdbc:mysql://<db_server>/MDB
Username: <db_user>
Password: ***********
JDBC Connection Properties:
There are currently 0 values in the map:
entry#

Security related configuration options

Max connections: 100
New connection was successfully created with validation status FINE
and persistent id 4
sqoop:000>

Crete the job:

create job --xid 1 --type import

Security related configuration options

Max connections: 100
New connection was successfully created with validation status FINE
and persistent id 4
sqoop:000> create job --xid 1 --type import
Creating job for connection with id 1
Please fill following values to create new job object
Name: sw-mdb-import

Database configuration

Schema name: DB
Table name: hist_1
Table SQL statement:
Table column names:
Partition column name:
Nulls in partition column:
Boundary query:

Output configuration

Storage type:
  0 : HDFS
Choose:


Choose: 0
Output format:
  0 : TEXT_FILE
  1 : SEQUENCE_FILE
Choose: 0
Compression format:
  0 : NONE
  1 : DEFAULT
  2 : DEFLATE
  3 : GZIP
  4 : BZIP2
  5 : LZO
  6 : LZ4
  7 : SNAPPY
Choose: 0
Output directory: /user/hive/warehouse/hist_n

Throttling resources

Extractors:
Loaders:
New job was successfully created with validation status FINE  and
persistent id 3
sqoop:000>

sqoop:000> start job --jid 1
sqoop:000> start job --jid 1
Exception has occurred during processing command
Exception: org.apache.sqoop.common.SqoopException Message:
CLIENT_0001:Server has returned exception
sqoop:000>

---


Here are the log traces from this attempt: Interestingly, I see sqoop1
being called even through the job was created and run in the sqoop2
shell:

---

2015-03-19 09:10:51,182 ERROR
org.apache.sqoop.server.SqoopProtocolServlet: Exception in POST
http://sanitized_sqoop2_node_address:12000/sqoop/v1/submission/action/1
org.apache.sqoop.common.SqoopException:
GENERIC_JDBC_CONNECTOR_0001:Unable to get a connection
---

Complete:


--
2015-03-19 09:09:48,170 INFO
org.apache.sqoop.repository.JdbcRepositoryTransaction: Attempting
transaction commit
2015-03-19 09:09:48,180 INFO
org.apache.sqoop.repository.JdbcRepositoryTransaction: Attempting
transaction commit
2015-03-19 09:09:58,831 INFO
org.apache.sqoop.repository.JdbcRepositoryTransaction: Attempting
transaction commit
2015-03-19 09:10:51,182 ERROR
org.apache.sqoop.server.SqoopProtocolServlet: Exception in POST
http://sanitized_sqoop2_node_address:12000/sqoop/v1/submission/action/1
org.apache.sqoop.common.SqoopException:
GENERIC_JDBC_CONNECTOR_0001:Unable to get a connection
        at org.apache.sqoop.connector.jdbc.GenericJdbcExecutor.<init>(GenericJdbcExecutor.java:51)
        at org.apache.sqoop.connector.jdbc.GenericJdbcImportInitializer.configureJdbcProperties(GenericJdbcImportInitializer.java:125)
        at org.apache.sqoop.connector.jdbc.GenericJdbcImportInitializer.initialize(GenericJdbcImportInitializer.java:49)
        at org.apache.sqoop.connector.jdbc.GenericJdbcImportInitializer.initialize(GenericJdbcImportInitializer.java:40)
        at org.apache.sqoop.framework.JobManager.submit(JobManager.java:378)
        at org.apache.sqoop.handler.SubmissionRequestHandler.submissionSubmit(SubmissionRequestHandler.java:152)
        at org.apache.sqoop.handler.SubmissionRequestHandler.handleActionEvent(SubmissionRequestHandler.java:122)
        at org.apache.sqoop.handler.SubmissionRequestHandler.handleEvent(SubmissionRequestHandler.java:75)
        at org.apache.sqoop.server.v1.SubmissionServlet.handlePostRequest(SubmissionServlet.java:44)
        at org.apache.sqoop.server.SqoopProtocolServlet.doPost(SqoopProtocolServlet.java:63)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:643)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:723)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
        at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:861)
        at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:606)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
        at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:861)
        at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:606)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
        at java.lang.Thread.run(Thread.java:744)
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException:
Communications link failure

--








>
> On Wed, Mar 18, 2015 at 2:04 AM, Traiano Welcome <traiano@gmail.com> wrote:
>>
>> Some further information:
>>
>> On Wed, Mar 18, 2015 at 8:37 AM, Traiano Welcome <traiano@gmail.com>
>> wrote:
>> > Further testing:
>> >
>> >
>> > On Wed, Mar 18, 2015 at 7:59 AM, Traiano Welcome <traiano@gmail.com>
>> > wrote:
>> >> Thanks for the response, Stanley:
>> >>
>> >>
>> >> On Wed, Mar 18, 2015 at 6:29 AM, Xu, Qian A <qian.a.xu@intel.com>
>> >> wrote:
>> >>> Hi Traiano,
>> >>>
>> >>> This is usually a firewall issue of the server xx.yy.xx.zz. Please
>> >>> make sure the xx.yy.xx.zz :3306 is accessible from where your Sqoop
query
>> >>> initiates.
>> >>
>> >>
>> >> I can confirm this is not the case:
>> >>
>> >>
>> >> - Can connect to port 3306 on xx.yy.xx.zz:
>> >>
>> >> ---
>> >> [root@lol-dev-hdpdn8 ~]# nc xx.yy.xx.zz 3306
>> >> [
>> >> 5.5.41-0ubuntu0.12.04.1�\FUV10;xn��>*VT!F{[7A|omysql_native_password
>> >> ^C
>> >> ---
>> >>
>> >> - Can connect using the mysql client and list tables in the db:
>> >>
>> >> ---
>> >> [root@lol-dev-hdpdn8 ~]# mysql -h xx.yy.xx.zz -u<user> -p <db>
>> >> Enter password:
>> >> Reading table information for completion of table and column names
>> >> You can turn off this feature to get a quicker startup with -A
>> >>
>> >> Welcome to the MySQL monitor.  Commands end with ; or \g.
>> >> Your MySQL connection id is 6076407
>> >>
>> >> ---
>> >>
>> >> - Finally, when I run a packet sniff listening for outbound packets to
>> >> xx.yy.xx.zz from the sqoop server, I see no packets leaving the node
>> >> to the db server:
>> >>
>> >>
>> >> ---
>> >> [root@lol-dev-hdpdn8 jdbc]# tcpdump host xx.yy.xx.zz and port 3306
>> >> tcpdump: verbose output suppressed, use -v or -vv for full protocol
>> >> decode
>> >> listening on eth0, link-type EN10MB (Ethernet), capture size 65535
>> >> bytes
>> >> ^C
>> >> 0 packets captured
>> >> 0 packets received by filter
>> >> 0 packets dropped by kernel
>> >> ---
>> >>
>> >> Is there any other way I could drill down deeper into this ?
>> >>
>> >
>> > Just to confirm the JDBC driver is not the problem, I've put together
>> > a test script that uses the java jdbc driver that sqoop is using, and
>> > the script seems to successfully connect to the mysql db:
>> >
>> > ---
>> > [root@lol-dev-hdpdn8 jdbc]# java -cp
>> > .:/var/lib/sqoop2/mysql-connector-java-5.1.34-bin.jar JDBCExample
>> > -------- MySQL JDBC Connection Testing ------------
>> > MySQL JDBC Driver Registered!
>> > You made it, take control your database now!
>> > ---
>> >
>> > I can see communications traffic to the mysql db going outbound using
>> > tcpdump, so this confirms no networking issue:
>> >
>> > ---
>> > [root@lol-dev-hdpdn8 jdbc]# tcpdump -n host xx.yy.xx.zz and port 3306
>> > tcpdump: verbose output suppressed, use -v or -vv for full protocol
>> > decode
>> > listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
>> > 08:27:11.643603 IP lol-dev-hdpdn8.39977 > xx.yy.xx.zz.mysql: Flags
>> > [S], seq 3476709965, win 14600, options [mss
>> > 1460,nop,nop,sackOK,nop,wscale 7], length 0
>> > 08:27:11.643836 IP xx.yy.xx.zz.mysql > lol-dev-hdpdn8.39977: Flags
>> > [S.], seq 1131758852, ack 3476709966, win 29200, options [mss
>> > 1460,nop,nop,sackOK,nop,wscale 7], length 0
>> > 08:27:11.643857 IP lol-dev-hdpdn8.39977 > xx.yy.xx.zz.mysql: Flags
>> > [.], ack 1, win 115, length 0
>> > 08:27:11.644321 IP xx.yy.xx.zz.mysql > lol-dev-hdpdn8.39977: Flags
>> > [P.], seq 1:96, ack 1, win 229, length 95
>> > 08:27:11.644341 IP lol-dev-hdpdn8.39977 > xx.yy.xx.zz.mysql: Flags
>> > [.], ack 96, win 115, length 0
>> > 08:27:11.678496 IP lol-dev-hdpdn8.39977 > xx.yy.xx.zz.mysql: Flags
>> > [P.], seq 1:91, ack 96, win 115, length 90
>> > 08:27:11.678777 IP xx.yy.xx.zz.mysql > lol-dev-hdpdn8.39977: Flags
>> > [.], ack 91, win 229, length 0
>> > 08:27:11.678848 IP xx.yy.xx.zz.mysql > lol-dev-hdpdn8.39977: Flags
>> > [P.], seq 96:107, ack 91, win 229, length 11
>> > 08:27:11.683985 IP lol-dev-hdpdn8.39977 > xx.yy.xx.zz.mysql: Flags
>> > [P.], seq 91:1034, ack 107, win 115, length 943
>> > 08:27:11.684769 IP xx.yy.xx.zz.mysql > lol-dev-hdpdn8.39977: Flags
>> > [P.], seq 107:774, ack 1034, win 243, length 667
>> > 08:27:11.705410 IP lol-dev-hdpdn8.39977 > xx.yy.xx.zz.mysql: Flags
>> > [P.], seq 1034:1182, ack 774, win 125, length 148
>> > 08:27:11.705847 IP xx.yy.xx.zz.mysql > lol-dev-hdpdn8.39977: Flags
>> > [P.], seq 774:863, ack 1182, win 258, length 89
>> > 08:27:11.707315 IP lol-dev-hdpdn8.39977 > xx.yy.xx.zz.mysql: Flags
>> > [P.], seq 1182:1203, ack 863, win 125, length 21
>> > 08:27:11.707511 IP xx.yy.xx.zz.mysql > lol-dev-hdpdn8.39977: Flags
>> > [P.], seq 863:874, ack 1203, win 258, length 11
>> > 08:27:11.707645 IP lol-dev-hdpdn8.39977 > xx.yy.xx.zz.mysql: Flags
>> > [P.], seq 1203:1240, ack 874, win 125, length 37
>> > 08:27:11.708241 IP xx.yy.xx.zz.mysql > lol-dev-hdpdn8.39977: Flags
>> > [P.], seq 874:885, ack 1240, win 258, length 11
>> > 08:27:11.708564 IP lol-dev-hdpdn8.39977 > xx.yy.xx.zz.mysql: Flags
>> > [P.], seq 1240:1261, ack 885, win 125, length 21
>> > 08:27:11.708737 IP xx.yy.xx.zz.mysql > lol-dev-hdpdn8.39977: Flags
>> > [P.], seq 885:896, ack 1261, win 258, length 11
>> > 08:27:11.708923 IP lol-dev-hdpdn8.39977 > xx.yy.xx.zz.mysql: Flags
>> > [P.], seq 1261:1300, ack 896, win 125, length 39
>> > 08:27:11.709078 IP xx.yy.xx.zz.mysql > lol-dev-hdpdn8.39977: Flags
>> > [P.], seq 896:907, ack 1300, win 258, length 11
>> > 08:27:11.712236 IP lol-dev-hdpdn8.39977 > xx.yy.xx.zz.mysql: Flags
>> > [F.], seq 1300, ack 907, win 125, length 0
>> > 08:27:11.712447 IP xx.yy.xx.zz.mysql > lol-dev-hdpdn8.39977: Flags
>> > [F.], seq 907, ack 1301, win 258, length 0
>> > 08:27:11.712461 IP lol-dev-hdpdn8.39977 > xx.yy.xx.zz.mysql: Flags
>> > [.], ack 908, win 125, length 0
>> >
>> > ---
>> >
>> >
>> > So this would appear to rule out an issue with the driver. Here is the
>> > java test script FYI:
>> >
>> > ---
>> > import java.sql.DriverManager;
>> > import java.sql.Connection;
>> > import java.sql.SQLException;
>> >
>> > // compile with: javac JDBCExample.java and run like: java -cp
>> > .:/var/lib/sqoop2/mysql-connector-java-5.1.34-bin.jar JDBCExampl
>> >
>> > public class JDBCExample {
>> >
>> >   public static void main(String[] argv) {
>> >
>> >         System.out.println("-------- MySQL JDBC Connection Testing
>> > ------------");
>> >
>> >         try {
>> >                 Class.forName("com.mysql.jdbc.Driver");
>> >         } catch (ClassNotFoundException e) {
>> >                 System.out.println("Where is your MySQL JDBC Driver?");
>> >                 e.printStackTrace();
>> >                 return;
>> >         }
>> >
>> >         System.out.println("MySQL JDBC Driver Registered!");
>> >         Connection connection = null;
>> >
>> >         try {
>> >                 connection = DriverManager
>> >                 .getConnection("jdbc:mysql://xx.yy.xx.zz:3306/<db
>> > name>","<db user>", "<db user password>");
>> >
>> >         } catch (SQLException e) {
>> >                 System.out.println("Connection Failed! Check output
>> > console");
>> >                 e.printStackTrace();
>> >                 return;
>> >         }
>> >
>> >         if (connection != null) {
>> >                 System.out.println("You made it, take control your
>> > database now!");
>> >         } else {
>> >                 System.out.println("Failed to make connection!");
>> >         }
>> >   }
>> > }
>> >
>> > ---
>> >
>> > How would I increase the debugging level in sqoop to check if it might
>> > be having issues calling the driver correctly?
>> >
>> >
>> >>
>> >>
>> >>
>> >>>
>> >>> Stanley
>> >>>
>> >>>
>> >>> -----Original Message-----
>> >>> From: Traiano Welcome [mailto:traiano@gmail.com]
>> >>> Sent: Tuesday, March 17, 2015 11:02 PM
>> >>> To: user@sqoop.apache.org
>> >>> Subject: How to debug SQOOP / JDBC Driver Failures for MySQL Import
>> >>> Jobs
>> >>>
>> >>> Hi
>> >>>
>> >>> I'm following the guide for testing Sqoop data imports from mysql:
>> >>>
>> >>>  http://blog.cloudera.com/blog/2013/11/sqooping-data-with-hue/
>> >>>
>> >>> But the mysql sqoop job fails with "" in the HUE interface.  I'd
>> >>> appreciate some help interpreting the sqoop java error trace I'm seeing
in
>> >>> the sqoop logs:
>> >>>
>> >>> My connection settings are as follows:
>> >>>
>> >>> ---
>> >>> JDBC driver class: com.mysql.jdbc.Driver JDBC Connection string:
>> >>> jdbc:mysql://xx.yy.xx.zz/dbname
>> >>> ---
>> >>>
>> >>>  I've downloaded and added the jdbc driver from Oracle in
>> >>> /var/lib/sqoop on the sqoop server node, following the cloudera guide
>> >>> here:
>> >>>
>> >>>
>> >>> http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cdh_ig_jdbc_driver_install.html
>> >>>
>> >>> However, the job keeps failing with the following java trace from the
>> >>> sqoop logs (excuse the noise!):
>> >>>
>> >>>
>> >>> ---
>> >>> 2015-03-17 17:44:06,368 INFO
>> >>> org.apache.sqoop.repository.JdbcRepositoryTransaction: Attempting
>> >>> transaction commit
>> >>> 2015-03-17 17:44:06,385 INFO
>> >>> org.apache.sqoop.repository.JdbcRepositoryTransaction: Attempting
>> >>> transaction commit
>> >>> 2015-03-17 17:44:06,393 INFO
>> >>> org.apache.sqoop.repository.JdbcRepositoryTransaction: Attempting
>> >>> transaction commit
>> >>> 2015-03-17 17:45:09,396 ERROR
>> >>> org.apache.sqoop.server.SqoopProtocolServlet: Exception in POST
>> >>> http://lol-dev-hdpdn8.hadoop.local:12000/sqoop/v1/submission/action/2
>> >>> org.apache.sqoop.common.SqoopException:
>> >>> GENERIC_JDBC_CONNECTOR_0001:Unable to get a connection
>> >>>         at
>> >>> org.apache.sqoop.connector.jdbc.GenericJdbcExecutor.<init>(GenericJdbcExecutor.java:51)
>> >>>         at
>> >>> org.apache.sqoop.connector.jdbc.GenericJdbcImportInitializer.configureJdbcProperties(GenericJdbcImportInitializer.java:125)
>> >>>         at
>> >>> org.apache.sqoop.connector.jdbc.GenericJdbcImportInitializer.initialize(GenericJdbcImportInitializer.java:49)
>> >>>         at
>> >>> org.apache.sqoop.connector.jdbc.GenericJdbcImportInitializer.initialize(GenericJdbcImportInitializer.java:40)
>> >>>         at
>> >>> org.apache.sqoop.framework.JobManager.submit(JobManager.java:378)
>> >>>         at
>> >>> org.apache.sqoop.handler.SubmissionRequestHandler.submissionSubmit(SubmissionRequestHandler.java:152)
>> >>>         at
>> >>> org.apache.sqoop.handler.SubmissionRequestHandler.handleActionEvent(SubmissionRequestHandler.java:122)
>> >>>         at
>> >>> org.apache.sqoop.handler.SubmissionRequestHandler.handleEvent(SubmissionRequestHandler.java:75)
>> >>>         at
>> >>> org.apache.sqoop.server.v1.SubmissionServlet.handlePostRequest(SubmissionServlet.java:44)
>> >>>         at
>> >>> org.apache.sqoop.server.SqoopProtocolServlet.doPost(SqoopProtocolServlet.java:63)
>> >>>         at
>> >>> javax.servlet.http.HttpServlet.service(HttpServlet.java:643)
>> >>>         at
>> >>> javax.servlet.http.HttpServlet.service(HttpServlet.java:723)
>> >>>         at
>> >>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
>> >>>         at
>> >>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
>> >>>         at
>> >>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
>> >>>         at
>> >>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
>> >>>         at
>> >>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
>> >>>         at
>> >>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
>> >>>         at
>> >>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
>> >>>         at
>> >>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
>> >>>         at
>> >>> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:861)
>> >>>         at
>> >>> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:606)
>> >>>         at
>> >>> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
>> >>>         at java.lang.Thread.run(Thread.java:744)
>> >>> Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException:
>> >>> Communications link failure
>> >>>
>> >>> The last packet sent successfully to the server was 0 milliseconds
>> >>> ago. The driver has not received any packets from the server.
>> >>>         at
>> >>> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>> >>>         at
>> >>> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
>> >>>         at
>> >>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>> >>>         at
>> >>> java.lang.reflect.Constructor.newInstance(Constructor.java:526)
>> >>>         at com.mysql.jdbc.Util.handleNewInstance(Util.java:377)
>> >>>         at
>> >>> com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1036)
>> >>>         at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:338)
>> >>>         at
>> >>> com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2232)
>> >>>         at
>> >>> com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2265)
>> >>>         at
>> >>> com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2064)
>> >>>         at
>> >>> com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:790)
>> >>>         at
>> >>> com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:44)
>> >>>         at
>> >>> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>> >>>         at
>> >>> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
>> >>> .
>> >>> .
>> >>> .
>> >>> ---
>> >>>
>> >>>
>> >>> I've checked for attempts to connect to the remove db server (tcpdump)
>> >>>
>> >>> I've checked for attempts to connect to the remove db server (tcpdump)
>> >>> but it looks like connection attempts are not even leaving the sqoop
node,
>> >>> so it seems sqoop is not successful at creating a mysql connection to
begin
>> >>> with. I've checked the remote mysql server connection from the sqoop
server
>> >>> using the mysql client as well - no issues there.
>> >>>
>> >>> I'm using sqoop on cloudera hadoop with cloudera manager 5.2.0 and
>> >>> java java version "1.7.0_45"
>> >>>
>> >>> Specific package details:
>> >>>
>> >>> hue-sqoop-3.6.0+cdh5.2.0+509-1.cdh5.2.0.p0.37.el6.x86_64
>> >>> sqoop2-client-1.99.3+cdh5.2.0+30-1.cdh5.2.0.p0.26.el6.noarch
>> >>> sqoop2-1.99.3+cdh5.2.0+30-1.cdh5.2.0.p0.26.el6.noarch
>> >>> sqoop-1.4.5+cdh5.2.0+47-1.cdh5.2.0.p0.26.el6.noarch
>> >>>
>> >>> I've checked the permissions on the .jar file in /usr/lib/sqoop2, and
>> >>> they seem fine:
>> >>>
>> >>> ---
>> >>> [root@lol-dev-hdpdn8 sqoop2]# ls -l /var/lib/sqoop2/ total 8680
>> >>> drwxr-xr-x 4 sqoop2 sqoop    4096 Sep 10  2014
>> >>> mysql-connector-java-5.1.33
>> >>> -rw-r--r-- 1 sqoop2 sqoop 3797240 Sep 10  2014
>> >>> mysql-connector-java-5.1.33.tar.gz
>> >>> -rwxr-xr-x 1 sqoop2 sqoop  960374 Mar 17 17:36
>> >>> mysql-connector-java-5.1.34-bin.jar
>> >>> -rwxr-xr-x 1 sqoop2 sqoop  539705 Mar 17 17:37
>> >>> postgresql-9.0-801.jdbc4.jar
>> >>> drwxr-xr-x 3 sqoop2 sqoop    4096 Oct 31 21:02 repository
>> >>> drwxr-xr-x 3 sqoop2 sqoop    4096 Nov  1 17:07 sqljdbc_4.0
>> >>> -rwxr-xr-x 1 sqoop2 sqoop 2391092 Nov  1 17:06
>> >>> sqljdbc_4.0.2206.100_enu.tar.gz -rwxr-xr-x 1 sqoop2 sqoop  584207 Nov
 1
>> >>> 17:12 sqljdbc4.jar -rwxr-xr-x 1 sqoop2 sqoop  563117 Nov  1 17:12
>> >>> sqljdbc.jar
>> >>> drwxr-xr-x 5 sqoop2 sqoop    4096 Mar 17 17:37 tomcat-deployment
>> >>> [root@lol-dev-hdpdn8 sqoop2]#
>> >>>
>> >>> ---
>> >>>
>> >>> I've also tested bot  mysql-connector-java-5.1.34-bin.jar and
>> >>> mysql-connector-java-5.1.33-bin.jar, restarting the sqoop service each
time
>> >>> I changed the module, to no avail.
>> >>>
>> >>> Is there anyway to debug further into why the job is failing?
>>
>>
>> If I run sqoop from the cli, the job seems successful in connecting to
>> the remote db:
>>
>> ----
>> [root@lol-dev-hdpdn8 yum.repos.d]# sqoop list-tables --connect
>> jdbc:mysql://xx.yy.xx.zz:3306/DB   --username=<user>
>> --password=<pass>
>>
>> Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will
>> fail.
>> Please set $ACCUMULO_HOME to the root of your Accumulo installation.
>> 15/03/18 11:23:20 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5-cdh5.2.0
>> 15/03/18 11:23:20 WARN tool.BaseSqoopTool: Setting your password on
>> the command-line is insecure. Consider using -P instead.
>> 15/03/18 11:23:20 INFO manager.SqlManager: Using default fetchSize of 1000
>> hist_1
>> settings
>> sites_list
>> ----
>>
>> So, neither a jdbc, java or network issue. It's beginning to look
>> pretty SQOOP specific.
>>
>>
>> >>>
>> >>> Many thanks in advance for any help!
>> >>>
>> >>> Traiano
>
>

Mime
View raw message