spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marcelo Vanzin <van...@cloudera.com>
Subject Re: Spark deployed by Cloudera Manager
Date Wed, 23 Jul 2014 16:26:24 GMT
Discussions about how CDH packages Spark aside, you should be using
the spark-class script (assuming you're still in 0.9) instead of
executing Java directly. That will make sure that the environment
needed to run Spark apps is set up correctly.

CDH 5.1 ships with Spark 1.0.0, so it has spark-submit available.

On Wed, Jul 23, 2014 at 12:28 AM, Debasish Das <debasish.das83@gmail.com> wrote:
> I found the issue...
>
> If you use spark git and generate the assembly jar then
> org.apache.hadoop.io.Writable.class is packaged with it....
>
> If you use the assembly jar that ships with CDH in
> /opt/cloudera/parcels/CDH/lib/spark/assembly/lib/spark-assembly_2.10-0.9.0-cdh5.0.2-hadoop2.3.0-cdh5.0.2.jar,
> they don't put org.apache.hadoop.io.Writable.class in it..
>
> That's weird...
>
> If I can run the spark app using bare bone java I am sure it will run with
> Ooyala's job server as well...
>
>
>
> On Wed, Jul 23, 2014 at 12:15 AM, buntu <buntudev@gmail.com> wrote:
>>
>> If you need to run Spark apps through Hue, see if Ooyala's job server
>> helps:
>>
>>
>>
>> http://gethue.com/get-started-with-spark-deploy-spark-server-and-compute-pi-from-your-web-browser/
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-spark-user-list.1001560.n3.nabble.com/Spark-deployed-by-Cloudera-Manager-tp10472p10474.html
>> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
>



-- 
Marcelo

Mime
View raw message