sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jarek Jarcec Cecho <jar...@apache.org>
Subject Re: sqoop import : communication links failure
Date Sat, 02 Nov 2013 17:59:56 GMT
Hi sir,
Sqoop will need access to your database from *all* machines in your cluster. Having db access
only from some limited number of machines (such as gateways, NameNode or other "special" nodes)
is not sufficient. I would recommend trying that mysql command from all your nodes or checking
your user configuration on MySQL server side.

Jarcec

On Sat, Nov 02, 2013 at 01:24:22AM +0800, wan razali wrote:
> Hi all,
> 
> I tried to import remote database using sqoop
> 
> I did try the solutions mentioned here:
> http://solaimurugan.blogspot.com/2013/07/resolved-issue-error-in-sqoop-14.html
> https://cwiki.apache.org/confluence/download/attachments/27361435/sqoop_meetup_kate_ting_110711.pdf?version=1&modificationDate=1323721159000
> 
> & did test the connection using mysql command below & successfully
> connected to the remote mysql server (@ 192.168.0.151):-
> 
> 
> 
> >mysql -h 192.168.0.151 -u sqoopuser -pscooppassword
> 
> mysql> show databases;
> +--------------------+
> | Database           |
> +--------------------+
> | information_schema |
> | mysql              |
> | performance_schema |
> | propertytrend      |
> | test               |
> | webscrapper        |
> +--------------------+
> 6 rows in set (0.39 sec)
> 
> -------------------------------------
> mysql> status;
> --------------
> mysql  Ver 14.14 Distrib 5.5.32, for debian-linux-gnu (x86_64) using
> readline 6.2
> 
> Connection id:        73
> Current database:
> Current user: sqoopuser@192.168.0.150 <mailto:sqoopuser@192.168.0.150>
> SSL:            Not in use
> Current pager:        stdout
> Using outfile:        ''
> Using delimiter:    ;
> Server version:        5.5.32-0ubuntu7 (Ubuntu)
> Protocol version:    10
> Connection:        192.168.0.151 via TCP/IP
> Server characterset:    latin1
> Db     characterset:    latin1
> Client characterset:    utf8
> Conn.  characterset:    utf8
> TCP port:        3306
> Uptime:            32 min 47 sec
> 
> 
> 
> 
> 
> I want to import the remote db into HFDS with setting options as follows:
> -----------------------
> import
> --connect
> jdbc:mysql://198.168.0.151/propertytrend
> --verbose
> --username
> sqoopuser
> --password
> scooppasword
> --as-textfile
> --split-by
> id
> --table
> hd_columns_varchar
> --columns
> id,rowId,value
> --where
> columnId = 5 AND reportId = 200
> --target-dir
> /user/hduser/sqoop/import/output
> 
> ------------------------
> 
> 
> But when I ran the sqoop import I got these errors:
> 
> ---------------------------------------
> bin/sqoop --options-file options/setting-import3.txt
> Warning: /usr/lib/hbase does not exist! HBase imports will fail.
> Please set $HBASE_HOME to the root of your HBase installation.
> Warning: /usr/lib/hcatalog does not exist! HCatalog jobs will fail.
> Please set $HCAT_HOME to the root of your HCatalog installation.
> Warning: $HADOOP_HOME is deprecated.
> 
> 13/11/01 23:42:15 DEBUG tool.BaseSqoopTool: Enabled debug logging.
> 13/11/01 23:42:15 WARN tool.BaseSqoopTool: Setting your password on
> the command-line is insecure. Consider using -P instead.
> 13/11/01 23:42:16 DEBUG sqoop.ConnFactory: Loaded manager factory:
> com.cloudera.sqoop.manager.DefaultManagerFactory
> 13/11/01 23:42:16 DEBUG sqoop.ConnFactory: Trying ManagerFactory:
> com.cloudera.sqoop.manager.DefaultManagerFactory
> 13/11/01 23:42:16 DEBUG manager.DefaultManagerFactory: Trying with
> scheme: jdbc:mysql:
> 13/11/01 23:42:16 INFO manager.MySQLManager: Preparing to use a
> MySQL streaming resultset.
> 13/11/01 23:42:16 DEBUG sqoop.ConnFactory: Instantiated ConnManager
> org.apache.sqoop.manager.MySQLManager@7618d289
> 13/11/01 23:42:16 INFO tool.CodeGenTool: Beginning code generation
> 13/11/01 23:42:16 DEBUG manager.SqlManager: Execute
> getColumnTypesRawQuery : SELECT t.* FROM `hd_columns_varchar` AS t
> LIMIT 1
> 13/11/01 23:42:16 DEBUG manager.SqlManager: No connection
> paramenters specified. Using regular API for making connection.
> 13/11/01 23:43:20 ERROR manager.SqlManager: Error executing
> statement: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException:
> Communications link failure
> 
> The last packet sent successfully to the server was 0 milliseconds
> ago. The driver has not received any packets from the server.
> com.mysql.jdbc.exceptions.jdbc4.CommunicationsException:
> Communications link failure
> 
> The last packet sent successfully to the server was 0 milliseconds
> ago. The driver has not received any packets from the server.
>     at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
>     at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
>     at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>     at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
>     at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
>     at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1121)
>     at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:357)
>     at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2482)
>     at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2519)
>     at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2304)
>     at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:834)
>     at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:47)
>     at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
>     at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
>     at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>     at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
>     at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
>     at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:416)
>     at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:346)
>     at java.sql.DriverManager.getConnection(DriverManager.java:571)
>     at java.sql.DriverManager.getConnection(DriverManager.java:215)
>     at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:801)
>     at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
>     at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:660)
>     at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:683)
>     at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:240)
>     at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:223)
>     at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:347)
>     at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1277)
>     at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1089)
>     at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:96)
>     at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:396)
>     at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:502)
>     at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
>     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>     at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
>     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
>     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
>     at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
> Caused by: java.net.ConnectException: Connection timed out
>     at java.net.PlainSocketImpl.socketConnect(Native Method)
>     at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>     at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>     at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>     at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
>     at java.net.Socket.connect(Socket.java:579)
>     at java.net.Socket.connect(Socket.java:528)
>     at java.net.Socket.<init>(Socket.java:425)
>     at java.net.Socket.<init>(Socket.java:241)
>     at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:259)
>     at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:307)
>     ... 32 more
> 13/11/01 23:43:20 ERROR tool.ImportTool: Encountered IOException
> running import job: java.io.IOException: No columns to generate for
> ClassWriter
>     at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1095)
>     at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:96)
>     at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:396)
>     at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:502)
>     at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
>     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>     at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
>     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
>     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
>     at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
> 
> ---------------------------------
> 
> any assistance is much appreciated.
> 
> thank you

Mime
View raw message