sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wan Razali <max_57...@yahoo.com>
Subject Re: sqoop import : communication links failure
Date Tue, 05 Nov 2013 05:12:41 GMT
Hi all,

it works now....actually something wrong with ubuntu....
I have 2 PCs using ubuntu 12 n ubuntu 13.....when I ping one another, I got "destination host
unreachable" message.
after solving that now I can import smoothly.

description of "destination host unreachable"
http://ubuntuforums.org/showthread.php?t=2176338



thnx all....



On Monday, November 4, 2013 1:30 AM, Venkat Ranganathan <vranganathan@hortonworks.com>
wrote:
 
May be you have firewall blocking remote connections to the DB host (check iptables if it
is running on Linux and Windows FIrewall settings if running on Windows).  Also, make sure
that you can run mysql client without problems -



On Sat, Nov 2, 2013 at 7:49 PM, Wan Razali <max_57070@yahoo.com> wrote:

Hi Jarek, 
>thnx for the response. 
>actually I only have one node running where the namenode, secondarynode, datanode,jobtracker
in one PC only.
>
>
>the remote PC contains the remote database which I failed to access using sqoop version
1.4.4
>
>
>my configuration
>IP: 192.168.0.150 - running namenode, secondarynode, datanode, jobstracker
>IP: 192.168.0.151 - running the remote mysql database
>
>
>FYI, I did try accessing local database at IP 192.168.0.150 and it works.
>
>
>any other things I need to look into?
>
>
>tq
>
>
>
>On Sunday, November 3, 2013 1:59 AM, Jarek Jarcec Cecho <jarcec@apache.org> wrote:
> 
>Hi sir,
>Sqoop will need access to your database from *all* machines in your cluster. Having db
access only from some limited number of machines (such as gateways, NameNode or other "special"
nodes) is not sufficient. I would recommend trying that mysql command from all your nodes
or checking your user configuration on MySQL server side.
>
>Jarcec
>
>On Sat, Nov 02, 2013 at 01:24:22AM +0800, wan razali wrote:
>> Hi all,
>> 
>> I tried to import remote database using sqoop
>> 
>> I did try the solutions mentioned here:
>> http://solaimurugan.blogspot.com/2013/07/resolved-issue-error-in-sqoop-14.html
>> https://cwiki.apache.org/confluence/download/attachments/27361435/sqoop_meetup_kate_ting_110711.pdf?version=1&modificationDate=1323721159000
>> 
>> & did test the connection using mysql command below & successfully
>> connected to the remote mysql server (@ 192.168.0.151):-
>> 
>> 
>> 
>> >mysql -h 192.168.0.151 -u sqoopuser -pscooppassword
>> 
>> mysql> show databases;
>> +--------------------+
>> | Database           |
>> +--------------------+
>> | information_schema |
>> | mysql              |
>> | performance_schema
 |
>> | propertytrend      |
>> | test               |
>> | webscrapper        |
>> +--------------------+
>> 6 rows in set (0.39 sec)
>> 
>> -------------------------------------
>> mysql> status;
>> --------------
>> mysql  Ver 14.14 Distrib 5.5.32, for debian-linux-gnu (x86_64) using
>> readline 6.2
>> 
>> Connection id:        73
>> Current database:
>> Current user: sqoopuser@192.168.0.150 <mailto:sqoopuser@192.168.0.150>
>> SSL:            Not in use
>> Current pager:        stdout
>> Using outfile:        ''
>>
 Using delimiter:    ;
>> Server version:        5.5.32-0ubuntu7 (Ubuntu)
>> Protocol version:    10
>> Connection:        192.168.0.151 via TCP/IP
>> Server characterset:    latin1
>> Db     characterset:    latin1
>> Client characterset:    utf8
>> Conn.  characterset:    utf8
>> TCP port:        3306
>> Uptime:            32 min 47 sec
>> 
>> 
>> 
>> 
>> 
>> I want to import the remote db into HFDS with setting options as follows:
>> -----------------------
>> import
>> --connect
>> jdbc:mysql://198.168.0.151/propertytrend
>> --verbose
>> --username
>> sqoopuser
>> --password
>> scooppasword
>> --as-textfile
>> --split-by
>> id
>> --table
>>
 hd_columns_varchar
>> --columns
>> id,rowId,value
>> --where
>> columnId = 5 AND reportId = 200
>> --target-dir
>> /user/hduser/sqoop/import/output
>> 
>> ------------------------
>> 
>> 
>> But when I ran the sqoop import I got these errors:
>> 
>> ---------------------------------------
>> bin/sqoop --options-file options/setting-import3.txt
>> Warning: /usr/lib/hbase does not exist! HBase imports will fail.
>> Please set $HBASE_HOME to the root of your HBase installation.
>> Warning: /usr/lib/hcatalog does not exist! HCatalog jobs will fail.
>> Please set $HCAT_HOME to the root of your HCatalog installation.
>> Warning: $HADOOP_HOME is deprecated.
>> 
>> 13/11/01 23:42:15 DEBUG tool.BaseSqoopTool: Enabled debug logging.
>> 13/11/01 23:42:15 WARN tool.BaseSqoopTool: Setting your password on
>> the command-line is insecure. Consider using
 -P instead.
>> 13/11/01 23:42:16 DEBUG sqoop.ConnFactory: Loaded manager factory:
>> com.cloudera.sqoop.manager.DefaultManagerFactory
>> 13/11/01 23:42:16 DEBUG sqoop.ConnFactory: Trying ManagerFactory:
>> com.cloudera.sqoop.manager.DefaultManagerFactory
>> 13/11/01 23:42:16 DEBUG manager.DefaultManagerFactory: Trying with
>> scheme: jdbc:mysql:
>> 13/11/01 23:42:16 INFO manager.MySQLManager: Preparing to use a
>> MySQL streaming resultset.
>> 13/11/01 23:42:16 DEBUG sqoop.ConnFactory: Instantiated ConnManager
>> org.apache.sqoop.manager.MySQLManager@7618d289
>> 13/11/01 23:42:16 INFO tool.CodeGenTool: Beginning code generation
>> 13/11/01 23:42:16 DEBUG manager.SqlManager: Execute
>> getColumnTypesRawQuery : SELECT t.* FROM `hd_columns_varchar` AS t
>>
 LIMIT 1
>> 13/11/01 23:42:16 DEBUG manager.SqlManager: No connection
>> paramenters specified. Using regular API for making connection.
>> 13/11/01 23:43:20 ERROR manager.SqlManager: Error executing
>> statement: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException:
>> Communications link failure
>> 
>> The last packet sent successfully to the server was 0 milliseconds
>> ago. The driver has not received any packets from the server.
>> com.mysql.jdbc.exceptions.jdbc4.CommunicationsException:
>> Communications link failure
>> 
>> The last packet sent successfully to the server was 0 milliseconds
>> ago. The driver has not received any packets from the server.
>>     at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
>> Method)
>>     at
 sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
>>     at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>>     at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
>>     at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
>>     at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1121)
>>     at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:357)
>>     at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2482)
>>     at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2519)
>>     at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2304)
>>     at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:834)
>>     at
 com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:47)
>>     at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
>> Method)
>>     at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
>>     at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>>     at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
>>     at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
>>     at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:416)
>>     at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:346)
>>     at java.sql.DriverManager.getConnection(DriverManager.java:571)
>>     at java.sql.DriverManager.getConnection(DriverManager.java:215)
>>   
  at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:801)
>>     at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
>>     at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:660)
>>     at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:683)
>>     at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:240)
>>     at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:223)
>>     at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:347)
>>     at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1277)
>>     at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1089)
>>     at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:96)
>> 
    at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:396)
>>     at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:502)
>>     at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
>>     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>     at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
>>     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
>>     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
>>     at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
>> Caused by: java.net.ConnectException: Connection timed out
>>     at java.net.PlainSocketImpl.socketConnect(Native Method)
>>     at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>     at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>> 
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>     at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
>>     at java.net.Socket.connect(Socket.java:579)
>>     at java.net.Socket.connect(Socket.java:528)
>>     at java.net.Socket.<init>(Socket.java:425)
>>     at java.net.Socket.<init>(Socket.java:241)
>>     at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:259)
>>     at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:307)
>>     ... 32 more
>> 13/11/01 23:43:20 ERROR tool.ImportTool: Encountered IOException
>> running import job: java.io.IOException: No columns to generate for
>> ClassWriter
>>     at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1095)
>>     at
 org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:96)
>>     at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:396)
>>     at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:502)
>>     at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
>>     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>     at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
>>     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
>>     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
>>     at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
>> 
>> ---------------------------------
>> 
>> any assistance is much appreciated.
>> 
>> thank you
>
>

CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed
and may contain information that is confidential, privileged and exempt from disclosure under
applicable law. If the reader of this message is not the intended recipient, you are hereby
notified that any printing, copying, dissemination, distribution, disclosure or forwarding
of this communication is strictly prohibited. If you have received this communication in error,
please contact the sender immediately and delete it from your system. Thank You.
Mime
View raw message