sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jarek Jarcec Cecho <jar...@apache.org>
Subject Re: One question about "sqoop export"
Date Tue, 08 Apr 2014 13:57:01 GMT
Hi Lei,
please do include entire log generated with parameter --verbose next time. The same (well
similar) exception can be thrown from multiple places and hence having entire log is usually
critical to triage the problem.

In your case it seems that Sqoop is able to get all the way down to mapreduce job submission
and hence I'm wondering if *all* your Hadoop nodes can access the database?

Jarcec

On Tue, Apr 08, 2014 at 09:34:53PM +0800, Lei Lei wrote:
> Hi all,
>     I have one question about "sqoop export". Thanks!
>     My Hadoop version is 1.2.1, and it works well; my sqoop version is
> 1.4.4.
>     I also installed MySQL whose version is 5.1.73. In MySQL, I created one
> table as below:
> 
> create table dimension_result (
> result int not null primary key,
> description VARCHAR(32) not null
> );
> 
>     On HDFS, I have a file called dimension_result.txt under /tmp/dr
>     See below.
> hadoop fs -cat /tmp/dr/dimension_result.txt
> 1,BlackWin
> 2,WhiteWin
> 0,Draw
> -1,Unknown
> 
>     I want to export the content of dimension_result.txt to the MySQL
> table, but it always failed.
> My command is:
> sqoop-export --connect jdbc:mysql://127.0.0.1:3306/test --driver
> com.mysql.jdbc.Driver --username hadoop --table dimension_result
> --export-dir /tmp/dr
> 
> I also tried:
> sqoop-export --connect jdbc:mysql://127.0.0.1:3306/test --driver
> com.mysql.jdbc.Driver --username hadoop --table dimension_result
> --export-dir /tmp/dr/dimension_result.txt
> 
>      (My MySQL instance does not need password, and the following command
> can succeed.
> sqoop-list-databases --connect jdbc:mysql://127.0.0.1:3306 --username
> hadoop )
> 
>       The error info is as below. Does anyone know why? Thanks!
> 
> 14/04/08 21:33:03 INFO mapred.JobClient: Task Id :
> attempt_201403071512_0021_m_000000_1, Status : FAILED
> java.io.IOException:
> com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications
> link failure
> 
> The last packet sent successfully to the server was 0 milliseconds ago. The
> driver has not received any packets from the server.
>         at
> org.apache.sqoop.mapreduce.ExportOutputFormat.getRecordWriter(ExportOutputFormat.java:79)
>         at
> org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.<init>(MapTask.java:627)
>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:753)
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364)
>         at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
>         at org.apache.hadoop.mapred.Child.main(Child.java:249)
> Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException:
> Communications link failure
> 
> The last packet sent successfully to the server was 0 milliseconds ago. The
> driver has not received any packets from the server.
>         at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
>         at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
>         at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>         at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
>         at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
>         at
> com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1129)
>         at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:358)
>         at
> com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2498)
>         at
> com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2535)
>         at
> com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2320)
>         at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:834)
>         at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:46)
>         at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
>         at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
>         at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>         at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
>         at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
>         at
> com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:416)
>         at
> com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:347)
>         at java.sql.DriverManager.getConnection(DriverManager.java:571)
>         at java.sql.DriverManager.getConnection(DriverManager.java:215)
>         at
> org.apache.sqoop.mapreduce.db.DBConfiguration.getConnection(DBConfiguration.java:294)
>         at
> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.<init>(AsyncSqlRecordWriter.java:76)
>         at
> org.apache.sqoop.mapreduce.ExportOutputFormat$ExportRecordWriter.<init>(ExportOutputFormat.java:95)
>         at
> org.apache.sqoop.mapreduce.ExportOutputFormat.getRecordWriter(ExportOutputFormat.java:77)
>         ... 8 more
> Caused by: java.net.ConnectException: Connection refused
>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>         at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>         at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>         at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
>         at java.net.Socket.connect(Socket.java:579)
>         at java.net.Socket.connect(Socket.java:528)
>         at java.net.Socket.<init>(Socket.java:425)
>         at java.net.Socket.<init>(Socket.java:241)
>         at
> com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:256)
>         at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:308)
>         ... 26 more
> 
> 
> Regards,
> Finix

Mime
View raw message