spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dillian Murphey <crackshotm...@gmail.com>
Subject Spark master keeps running out of RAM
Date Thu, 31 Mar 2016 18:16:14 GMT
Why would the spark master run out of RAM if I have too  many slaves?  Is
this a flaw in the coding?  I'm just a user of spark. The developer that
set this up left the company, so I'm starting from the top here.

So I noticed if I spawn lots of jobs, my spark master ends up crashing due
to low memory.  It makes sense to me the master would just be a
brain/controller to dish out jobs and the resources on the slaves is what
would get used up, not the master.

Thanks for any ideas/concepts/info.

Appreciate it much

Mime
View raw message