sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jarek Jarcec Cecho <jar...@apache.org>
Subject Re: Sqoop error
Date Sat, 26 May 2012 13:54:59 GMT
Hi Srinivas,
exception "com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure"
is very generic and basically means that there was something wrong with connecting to the
MySQL. Could you share your entire execution log that was generated with --verbose flag?

Blindly shooting - it's suspicious that your issue appears only when certain amount of jobs
is generated. MySQL is allowing limiting maximal number of concurrent connections. I would
recommend checking defined connection limit on MySQL side (by SQL query show variables like
"max_connections";) and compare it to number of created map tasks. In case that configured
maximal value for connections is lower, then you need to either reconfigure mysql or sqoop
(for example limit number of used mappers with --num-mappers argument or do not import all
tables at once).

Jarcec

On Sat, May 26, 2012 at 09:40:47AM -0400, Srinivas wrote:
> 
> > I have couple of problems that I am running into:
> > 
> > 	1. I was able to initiate only 5 sqoop jobs parallel which is = number of datanode's.

> > 	2. If i try to invoke more jobs parallel then only 5 jobs are picked by and the
rest of them throwing an error saying "com.mysql.jdbc.exceptions.jdbc4.CommunicationsException:
Communications link failure"
> > 
> > 	Do you have any idea why this is happening.
> > 
> > Also, I have made sure that the mysql allows parallel connections from datanode's.
> > 
> > 
> > 

Mime
View raw message