spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nick Chammas <>
Subject Re: spark-ec2 vs. EMR
Date Tue, 01 Dec 2015 16:15:50 GMT
Pinging this thread in case anyone has thoughts on the matter they want to

On Sat, Nov 21, 2015 at 11:32 AM Nicholas Chammas <> wrote:

> Spark has come bundled with spark-ec2
> <> for many years. At
> the same time, EMR has been capable of running Spark for a while, and
> earlier this year it added "official" support
> <>.
> If you're looking for a way to provision Spark clusters, there are some
> clear differences between these 2 options. I think the biggest one would be
> that EMR is a "production" solution backed by a company, whereas spark-ec2
> is not really intended for production use (as far as I know).
> That particular difference in intended use may or may not matter to you,
> but I'm curious:
> What are some of the other differences between the 2 that do matter to
> you? If you were considering these 2 solutions for your use case at one
> point recently, why did you choose one over the other?
> I'd be especially interested in hearing about why people might choose
> spark-ec2 over EMR, since the latter option seems to have shaped up nicely
> this year.
> Nick

View this message in context:
Sent from the Apache Spark User List mailing list archive at
View raw message