spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shivaram Venkataraman <shiva...@eecs.berkeley.edu>
Subject Re: spark-ec2 default to Hadoop 2
Date Mon, 02 Mar 2015 17:22:00 GMT
FWIW there is a PR open to add support for Hadoop 2.4 to spark-ec2 scripts
at https://github.com/mesos/spark-ec2/pull/77 -- But it hasnt' received
much review or testing to be merged.

Thanks
Shivaram

On Sun, Mar 1, 2015 at 11:49 PM, Sean Owen <sowen@cloudera.com> wrote:

> I agree with that. My anecdotal impression is that Hadoop 1.x usage
> out there is maybe a couple percent, and so we should shift towards
> 2.x at least as defaults.
>
> On Sun, Mar 1, 2015 at 10:59 PM, Nicholas Chammas
> <nicholas.chammas@gmail.com> wrote:
> >
> https://github.com/apache/spark/blob/fd8d283eeb98e310b1e85ef8c3a8af9e547ab5e0/ec2/spark_ec2.py#L162-L164
> >
> > Is there any reason we shouldn't update the default Hadoop major version
> in
> > spark-ec2 to 2?
> >
> > Nick
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
> For additional commands, e-mail: dev-help@spark.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message