I have a stupid question, but I am not sure how to get out of it.
I deployed spark 1.2.1 on a cluster of 30 nodes. Looking at master:8088 I can see all the workers I have created so far. (I start the cluster with sbin/start-all.sh)
However, when running a Spark SQL query or even spark-shell, I cannot see any job executing at master webUI, but the jobs are able to finish. I suspect they are executing locally on the master, but I don't understand why/how and why not on slave machines.
My conf/spark-env.sh is as following:
While conf/slaves is populated with the list of machines used for workers. I have to mention that spark-env.sh and slaves files are deployed on all machines.