spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kant kodali <kanth...@gmail.com>
Subject Re: Do we really need mesos or yarn? or is standalone sufficent?
Date Sat, 17 Dec 2016 00:45:58 GMT
Hi Saif,

What do you mean by small cluster? Any specific size?

Also can you shine some light on how YARN takes a win over mesos?

Thanks,
kant

On Fri, Dec 16, 2016 at 10:45 AM, <Saif.A.Ellafi@wellsfargo.com> wrote:

> In my experience, Standalone works very well in small cluster where there
> isn’t anything else running.
>
>
>
> Bigger cluster or shared resources, YARN takes a win, surpassing the
> overhead of spawning containers as opposed to a background running worker.
>
>
>
> Best is if you try both, if standalone is good enough keep it till you
> need more. Otherwise, try YARN or MESOS depending on the rest of your
> components.
>
>
>
> 2cents
>
>
>
> Saif
>
>
>
> *From:* kant kodali [mailto:kanth909@gmail.com]
> *Sent:* Friday, December 16, 2016 3:14 AM
> *To:* user @spark
> *Subject:* Do we really need mesos or yarn? or is standalone sufficent?
>
>
>
> Do we really need mesos or yarn? or is standalone sufficient for
> production systems? I understand the difference but I don't know the
> capabilities of standalone cluster. does anyone have experience deploying
> standalone in the production?
>
>
>
>
>

Mime
View raw message