spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Timothy Chen <tnac...@gmail.com>
Subject Re: Please reply if you use Mesos fine grained mode
Date Wed, 04 Nov 2015 06:12:01 GMT
Fine grain mode does reuse the same JVM but perhaps different placement or different allocated
cores comparing to the same total memory allocation.

Tim

Sent from my iPhone

> On Nov 3, 2015, at 6:00 PM, Reynold Xin <rxin@databricks.com> wrote:
> 
> Soren,
> 
> If I understand how Mesos works correctly, even the fine grained mode keeps the JVMs
around?
> 
> 
>> On Tue, Nov 3, 2015 at 4:22 PM, Soren Macbeth <soren@yieldbot.com> wrote:
>> we use fine-grained mode. coarse-grained mode keeps JVMs around which often leads
to OOMs, which in turn kill the entire executor, causing entire stages to be retried. In fine-grained
mode, only the task fails and subsequently gets retried without taking out an entire stage
or worse. 
>> 
>>> On Tue, Nov 3, 2015 at 3:54 PM, Reynold Xin <rxin@databricks.com> wrote:
>>> If you are using Spark with Mesos fine grained mode, can you please respond to
this email explaining why you use it over the coarse grained mode?
>>> 
>>> Thanks.
> 

Mime
View raw message