Your configuration looks largely correct. I have very recently confirmed that the way you launch SparkPi also works for me.
I have run into the same problem a bunch of times. My best guess is that this is a Java version issue. If the Spark assembly jar is built with Java 7, it cannot be opened by Java 6 because the two versions use different packaging schemes. This is a known issue: https://issues.apache.org/jira/browse/SPARK-1520
The workaround is to either make sure that all your executor nodes are running Java 7, and, very importantly, have JAVA_HOME point to this version. You can achieve this through
One last thing, launching Spark applications through org.apache.spark.deploy.yarn.Client is deprecated in Spark 1.0. You should use bin/spark-submit instead. You can find information about its usage on the docs I linked to you, or simply through the --help option.