spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shivaram Venkataraman <shiva...@eecs.berkeley.edu>
Subject [ANNOUNCE] YARN support in Spark EC2
Date Wed, 03 Jun 2015 23:32:53 GMT
Hi all

We recently merged support for launching YARN clusters using Spark EC2
scripts as a part of
https://issues.apache.org/jira/browse/SPARK-3674. To use this you can pass
in hadoop-major-version as "yarn" to the spark-ec2 script and this will
setup Hadoop 2.4 HDFS, YARN and Spark built for YARN on the EC2 cluster.

Developers who work on features related to YARN might find this useful for
testing / benchmarking Spark with YARN. If anyone has questions or feedback
please let me know.

Thanks
Shivaram

Mime
View raw message