spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Manoj Samel <manojsamelt...@gmail.com>
Subject Re: Exception in thread "DAGScheduler" java.lang.OutOfMemoryError: GC overhead limit exceeded
Date Thu, 23 Jan 2014 21:22:49 GMT
I could increase the memory for master.

However, my understanding of the Master is that it just does the DAG
scheduling for workers and does not do any RDD processing itself. If this
is true and since only one application was running; does the master needs >
512 Mb just to execute the DAG for 3105 tasks ?


On Thu, Jan 23, 2014 at 11:52 AM, Guillaume Pitel <
guillaume.pitel@exensa.com> wrote:

>  Why not trying to increase the master memory ?
>
> export SPARK_DAEMON_MEMORY=1g in master spark-env.sh
>
> Guillaume
>
>  I have a standalone master cluster with 512Mb for master and 12 g for
> each worker. I start spark-shell with 10g and spark.task.maxFailures=99999
>
> I run a job that plans 3105 tasks. 3104 tasks out of 3105 tasks run OK,
> then the tasks start failing and after 36K failed tasks, I get following.
> Is the master running out of memory ?
>
>  Exception in thread "DAGScheduler" java.lang.OutOfMemoryError: GC
> overhead limit exceeded
> Jan 23, 2014 6:33:36 PM
> org.jboss.netty.channel.socket.nio.AbstractNioWorker
> WARNING: Unexpected exception in the selector loop.
> java.lang.OutOfMemoryError: GC overhead limit exceeded
> Uncaught error from thread [spark-11] shutting down JVM since
> 'akka.jvm-exit-on-fatal-error' is enabled for ActorSystem[spark]
> java.lang.OutOfMemoryError: GC overhead limit exceeded
> Jan 23, 2014 6:36:43 PM
> org.jboss.netty.channel.socket.nio.AbstractNioWorker
> WARNING: Unexpected exception in the selector loop.
> java.lang.OutOfMemoryError: GC overhead limit exceeded
> Jan 23, 2014 6:36:36 PM
> org.jboss.netty.channel.socket.nio.AbstractNioWorker
> WARNING: Unexpected exception in the selector loop.
> java.lang.OutOfMemoryError: GC overhead limit exceeded
>
>
>
>
>
>
> --
>    [image: eXenSa]
>  *Guillaume PITEL, Président*
> +33(0)6 25 48 86 80 / +33(0)9 70 44 67 53
>
>  eXenSa S.A.S. <http://www.exensa.com/>
>  41, rue Périer - 92120 Montrouge - FRANCE
> Tel +33(0)1 84 16 36 77 / Fax +33(0)9 72 28 37 05
>

Mime
View raw message