spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sandy Ryza <sandy.r...@cloudera.com>
Subject Re: Applications for Spark on HDFS
Date Wed, 12 Mar 2014 00:09:06 GMT
Hi Paul,

What do you mean by distributing the jars manually?  If you register jars
that are local to the client with SparkContext.addJars, Spark should handle
distributing them to the workers.  Are you taking advantage of this?

-Sandy


On Tue, Mar 11, 2014 at 3:09 PM, Paul Schooss <paulmschooss@gmail.com>wrote:

> Hello Folks,
>
> I was wondering if anyone had experience placing application jars for
> Spark onto HDFS. Currently I have distributing the jars manually and would
> love to source the jar via HDFS a la distributed caching with MR. Any
> ideas?
>
> Regards,
>
> Paul
>

Mime
View raw message