sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jarek Jarcec Cecho <jar...@apache.org>
Subject Re: One question about "sqoop export"
Date Tue, 08 Apr 2014 14:02:37 GMT
On second look - you are specifying "localhost" in the connection URL. As Sqoop will open connections
from different machines, this won't work. You always need to use fully qualified domain name.

Jarcec

On Tue, Apr 08, 2014 at 06:57:01AM -0700, Jarek Jarcec Cecho wrote:
> Hi Lei,
> please do include entire log generated with parameter --verbose next time. The same (well
similar) exception can be thrown from multiple places and hence having entire log is usually
critical to triage the problem.
> 
> In your case it seems that Sqoop is able to get all the way down to mapreduce job submission
and hence I'm wondering if *all* your Hadoop nodes can access the database?
> 
> Jarcec
> 
> On Tue, Apr 08, 2014 at 09:34:53PM +0800, Lei Lei wrote:
> > Hi all,
> >     I have one question about "sqoop export". Thanks!
> >     My Hadoop version is 1.2.1, and it works well; my sqoop version is
> > 1.4.4.
> >     I also installed MySQL whose version is 5.1.73. In MySQL, I created one
> > table as below:
> > 
> > create table dimension_result (
> > result int not null primary key,
> > description VARCHAR(32) not null
> > );
> > 
> >     On HDFS, I have a file called dimension_result.txt under /tmp/dr
> >     See below.
> > hadoop fs -cat /tmp/dr/dimension_result.txt
> > 1,BlackWin
> > 2,WhiteWin
> > 0,Draw
> > -1,Unknown
> > 
> >     I want to export the content of dimension_result.txt to the MySQL
> > table, but it always failed.
> > My command is:
> > sqoop-export --connect jdbc:mysql://127.0.0.1:3306/test --driver
> > com.mysql.jdbc.Driver --username hadoop --table dimension_result
> > --export-dir /tmp/dr
> > 
> > I also tried:
> > sqoop-export --connect jdbc:mysql://127.0.0.1:3306/test --driver
> > com.mysql.jdbc.Driver --username hadoop --table dimension_result
> > --export-dir /tmp/dr/dimension_result.txt
> > 
> >      (My MySQL instance does not need password, and the following command
> > can succeed.
> > sqoop-list-databases --connect jdbc:mysql://127.0.0.1:3306 --username
> > hadoop )
> > 
> >       The error info is as below. Does anyone know why? Thanks!
> > 
> > 14/04/08 21:33:03 INFO mapred.JobClient: Task Id :
> > attempt_201403071512_0021_m_000000_1, Status : FAILED
> > java.io.IOException:
> > com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications
> > link failure
> > 
> > The last packet sent successfully to the server was 0 milliseconds ago. The
> > driver has not received any packets from the server.
> >         at
> > org.apache.sqoop.mapreduce.ExportOutputFormat.getRecordWriter(ExportOutputFormat.java:79)
> >         at
> > org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.<init>(MapTask.java:627)
> >         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:753)
> >         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364)
> >         at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> >         at java.security.AccessController.doPrivileged(Native Method)
> >         at javax.security.auth.Subject.doAs(Subject.java:415)
> >         at
> > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
> >         at org.apache.hadoop.mapred.Child.main(Child.java:249)
> > Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException:
> > Communications link failure
> > 
> > The last packet sent successfully to the server was 0 milliseconds ago. The
> > driver has not received any packets from the server.
> >         at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> > Method)
> >         at
> > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> >         at
> > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> >         at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> >         at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
> >         at
> > com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1129)
> >         at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:358)
> >         at
> > com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2498)
> >         at
> > com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2535)
> >         at
> > com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2320)
> >         at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:834)
> >         at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:46)
> >         at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> > Method)
> >         at
> > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> >         at
> > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> >         at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> >         at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
> >         at
> > com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:416)
> >         at
> > com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:347)
> >         at java.sql.DriverManager.getConnection(DriverManager.java:571)
> >         at java.sql.DriverManager.getConnection(DriverManager.java:215)
> >         at
> > org.apache.sqoop.mapreduce.db.DBConfiguration.getConnection(DBConfiguration.java:294)
> >         at
> > org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.<init>(AsyncSqlRecordWriter.java:76)
> >         at
> > org.apache.sqoop.mapreduce.ExportOutputFormat$ExportRecordWriter.<init>(ExportOutputFormat.java:95)
> >         at
> > org.apache.sqoop.mapreduce.ExportOutputFormat.getRecordWriter(ExportOutputFormat.java:77)
> >         ... 8 more
> > Caused by: java.net.ConnectException: Connection refused
> >         at java.net.PlainSocketImpl.socketConnect(Native Method)
> >         at
> > java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
> >         at
> > java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
> >         at
> > java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
> >         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> >         at java.net.Socket.connect(Socket.java:579)
> >         at java.net.Socket.connect(Socket.java:528)
> >         at java.net.Socket.<init>(Socket.java:425)
> >         at java.net.Socket.<init>(Socket.java:241)
> >         at
> > com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:256)
> >         at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:308)
> >         ... 26 more
> > 
> > 
> > Regards,
> > Finix



Mime
View raw message