sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lei Lei <learnerfi...@gmail.com>
Subject Re: One question about "sqoop export"
Date Tue, 08 Apr 2014 14:41:19 GMT
Ah, I think you are right, Jarek. I will try it later. Thanks.


2014-04-08 22:02 GMT+08:00 Jarek Jarcec Cecho <jarcec@apache.org>:

> On second look - you are specifying "localhost" in the connection URL. As
> Sqoop will open connections from different machines, this won't work. You
> always need to use fully qualified domain name.
>
> Jarcec
>
> On Tue, Apr 08, 2014 at 06:57:01AM -0700, Jarek Jarcec Cecho wrote:
> > Hi Lei,
> > please do include entire log generated with parameter --verbose next
> time. The same (well similar) exception can be thrown from multiple places
> and hence having entire log is usually critical to triage the problem.
> >
> > In your case it seems that Sqoop is able to get all the way down to
> mapreduce job submission and hence I'm wondering if *all* your Hadoop nodes
> can access the database?
> >
> > Jarcec
> >
> > On Tue, Apr 08, 2014 at 09:34:53PM +0800, Lei Lei wrote:
> > > Hi all,
> > >     I have one question about "sqoop export". Thanks!
> > >     My Hadoop version is 1.2.1, and it works well; my sqoop version is
> > > 1.4.4.
> > >     I also installed MySQL whose version is 5.1.73. In MySQL, I
> created one
> > > table as below:
> > >
> > > create table dimension_result (
> > > result int not null primary key,
> > > description VARCHAR(32) not null
> > > );
> > >
> > >     On HDFS, I have a file called dimension_result.txt under /tmp/dr
> > >     See below.
> > > hadoop fs -cat /tmp/dr/dimension_result.txt
> > > 1,BlackWin
> > > 2,WhiteWin
> > > 0,Draw
> > > -1,Unknown
> > >
> > >     I want to export the content of dimension_result.txt to the MySQL
> > > table, but it always failed.
> > > My command is:
> > > sqoop-export --connect jdbc:mysql://127.0.0.1:3306/test --driver
> > > com.mysql.jdbc.Driver --username hadoop --table dimension_result
> > > --export-dir /tmp/dr
> > >
> > > I also tried:
> > > sqoop-export --connect jdbc:mysql://127.0.0.1:3306/test --driver
> > > com.mysql.jdbc.Driver --username hadoop --table dimension_result
> > > --export-dir /tmp/dr/dimension_result.txt
> > >
> > >      (My MySQL instance does not need password, and the following
> command
> > > can succeed.
> > > sqoop-list-databases --connect jdbc:mysql://127.0.0.1:3306 --username
> > > hadoop )
> > >
> > >       The error info is as below. Does anyone know why? Thanks!
> > >
> > > 14/04/08 21:33:03 INFO mapred.JobClient: Task Id :
> > > attempt_201403071512_0021_m_000000_1, Status : FAILED
> > > java.io.IOException:
> > > com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications
> > > link failure
> > >
> > > The last packet sent successfully to the server was 0 milliseconds
> ago. The
> > > driver has not received any packets from the server.
> > >         at
> > >
> org.apache.sqoop.mapreduce.ExportOutputFormat.getRecordWriter(ExportOutputFormat.java:79)
> > >         at
> > >
> org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.<init>(MapTask.java:627)
> > >         at
> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:753)
> > >         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364)
> > >         at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> > >         at java.security.AccessController.doPrivileged(Native Method)
> > >         at javax.security.auth.Subject.doAs(Subject.java:415)
> > >         at
> > >
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
> > >         at org.apache.hadoop.mapred.Child.main(Child.java:249)
> > > Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException:
> > > Communications link failure
> > >
> > > The last packet sent successfully to the server was 0 milliseconds
> ago. The
> > > driver has not received any packets from the server.
> > >         at
> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> > > Method)
> > >         at
> > >
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> > >         at
> > >
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> > >         at
> java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> > >         at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
> > >         at
> > >
> com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1129)
> > >         at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:358)
> > >         at
> > > com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2498)
> > >         at
> > >
> com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2535)
> > >         at
> > > com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2320)
> > >         at
> com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:834)
> > >         at
> com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:46)
> > >         at
> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> > > Method)
> > >         at
> > >
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> > >         at
> > >
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> > >         at
> java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> > >         at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
> > >         at
> > > com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:416)
> > >         at
> > >
> com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:347)
> > >         at java.sql.DriverManager.getConnection(DriverManager.java:571)
> > >         at java.sql.DriverManager.getConnection(DriverManager.java:215)
> > >         at
> > >
> org.apache.sqoop.mapreduce.db.DBConfiguration.getConnection(DBConfiguration.java:294)
> > >         at
> > >
> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.<init>(AsyncSqlRecordWriter.java:76)
> > >         at
> > >
> org.apache.sqoop.mapreduce.ExportOutputFormat$ExportRecordWriter.<init>(ExportOutputFormat.java:95)
> > >         at
> > >
> org.apache.sqoop.mapreduce.ExportOutputFormat.getRecordWriter(ExportOutputFormat.java:77)
> > >         ... 8 more
> > > Caused by: java.net.ConnectException: Connection refused
> > >         at java.net.PlainSocketImpl.socketConnect(Native Method)
> > >         at
> > >
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
> > >         at
> > >
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
> > >         at
> > >
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
> > >         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> > >         at java.net.Socket.connect(Socket.java:579)
> > >         at java.net.Socket.connect(Socket.java:528)
> > >         at java.net.Socket.<init>(Socket.java:425)
> > >         at java.net.Socket.<init>(Socket.java:241)
> > >         at
> > >
> com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:256)
> > >         at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:308)
> > >         ... 26 more
> > >
> > >
> > > Regards,
> > > Finix
>
>
>

Mime
View raw message