spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Greg Hill <greg.h...@RACKSPACE.COM>
Subject Spark on YARN question
Date Tue, 02 Sep 2014 15:06:19 GMT
I'm working on setting up Spark on YARN using the HDP technical preview - http://hortonworks.com/kb/spark-1-0-1-technical-preview-hdp-2-1-3/

I have installed the Spark JARs on all the slave nodes and configured YARN to find the JARs.
 It seems like everything is working.

Unless I'm misunderstanding, it seems like there isn't any configuration required on the YARN
slave nodes at all, apart from telling YARN where to find the Spark JAR files.  Do the YARN
processes even pick up local Spark configuration files on the slave nodes, or is that all
just pulled in on the client and passed along to YARN?

Greg

Mime
View raw message