What is yarn cluster?
And, does spark necessarily need Hadoop already installed in the cluster? For example, can one download spark and run it on a bunch of nodes, with no prior installation of Hadoop?
You can write `HADOOP_CONF_DIR=your_hadoop_conf_path` to `conf/spark-env.sh` to enable:
1 connect to your yarn cluster
On Oct 27, 2014, at 07:59, Pagliari, Roberto <email@example.com> wrote:
I’m a newbie with Spark. After installing it on all the machines I want to use, do I need to tell it about Hadoop configuration, or will it be able to find it himself?