sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kathleen Ting <kathl...@apache.org>
Subject Re: Sqoop error
Date Mon, 28 May 2012 07:39:06 GMT
Hi Srinivas,

In addition to Jarcec's excellent advice, perhaps you might find
http://archive.cloudera.com/cdh/3/sqoop/SqoopUserGuide.html#_mysql_connection_failurehelpful?

Regards,
Kathleen

On Sat, May 26, 2012 at 6:54 AM, Jarek Jarcec Cecho <jarcec@apache.org>wrote:

> Hi Srinivas,
> exception "com.mysql.jdbc.exceptions.jdbc4.CommunicationsException:
> Communications link failure" is very generic and basically means that there
> was something wrong with connecting to the MySQL. Could you share your
> entire execution log that was generated with --verbose flag?
>
> Blindly shooting - it's suspicious that your issue appears only when
> certain amount of jobs is generated. MySQL is allowing limiting maximal
> number of concurrent connections. I would recommend checking defined
> connection limit on MySQL side (by SQL query show variables like
> "max_connections";) and compare it to number of created map tasks. In case
> that configured maximal value for connections is lower, then you need to
> either reconfigure mysql or sqoop (for example limit number of used mappers
> with --num-mappers argument or do not import all tables at once).
>
> Jarcec
>
> On Sat, May 26, 2012 at 09:40:47AM -0400, Srinivas wrote:
> >
> > > I have couple of problems that I am running into:
> > >
> > >     1. I was able to initiate only 5 sqoop jobs parallel which is =
> number of datanode's.
> > >     2. If i try to invoke more jobs parallel then only 5 jobs are
> picked by and the rest of them throwing an error saying
> "com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications
> link failure"
> > >
> > >     Do you have any idea why this is happening.
> > >
> > > Also, I have made sure that the mysql allows parallel connections from
> datanode's.
> > >
> > >
> > >
>

Mime
View raw message