spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nick Chammas <nicholas.cham...@gmail.com>
Subject Re: spark-ec2 vs. EMR
Date Tue, 01 Dec 2015 16:15:50 GMT
Pinging this thread in case anyone has thoughts on the matter they want to
share.

On Sat, Nov 21, 2015 at 11:32 AM Nicholas Chammas <
nicholas.chammas@gmail.com> wrote:

> Spark has come bundled with spark-ec2
> <http://spark.apache.org/docs/latest/ec2-scripts.html> for many years. At
> the same time, EMR has been capable of running Spark for a while, and
> earlier this year it added "official" support
> <https://aws.amazon.com/blogs/aws/new-apache-spark-on-amazon-emr/>.
>
> If you're looking for a way to provision Spark clusters, there are some
> clear differences between these 2 options. I think the biggest one would be
> that EMR is a "production" solution backed by a company, whereas spark-ec2
> is not really intended for production use (as far as I know).
>
> That particular difference in intended use may or may not matter to you,
> but I'm curious:
>
> What are some of the other differences between the 2 that do matter to
> you? If you were considering these 2 solutions for your use case at one
> point recently, why did you choose one over the other?
>
> I'd be especially interested in hearing about why people might choose
> spark-ec2 over EMR, since the latter option seems to have shaped up nicely
> this year.
>
> Nick
>
>




--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Re-spark-ec2-vs-EMR-tp25538.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
Mime
View raw message