spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dean Wampler <>
Subject Re: SparkSQL Thriftserver in Mesos
Date Mon, 22 Sep 2014 19:40:32 GMT
The Mesos install guide says this:

"To use Mesos from Spark, you need a Spark binary package available in a
place accessible by Mesos, and a Spark driver program configured to connect
to Mesos."

For example, putting it in HDFS or copying it to each node in the same
location should do the trick.

Dean Wampler, Ph.D.
Author: Programming Scala, 2nd Edition
<> (O'Reilly)
Typesafe <>
@deanwampler <>

On Mon, Sep 22, 2014 at 2:35 PM, John Omernik <> wrote:

> Any thoughts on this?
> On Sat, Sep 20, 2014 at 12:16 PM, John Omernik <> wrote:
>> I am running the Thrift server in SparkSQL, and running it on the node I
>> compiled spark on.  When I run it, tasks only work if they landed on that
>> node, other executors started on nodes I didn't compile spark on (and thus
>> don't have the compile directory) fail.  Should spark be distributed
>> properly with the executor uri in my spark-defaults for mesos?
>> Here is the error on nodes with Lost executors
>> sh: 1: /opt/mapr/spark/spark-1.1.0-SNAPSHOT/sbin/spark-executor: not found

View raw message