spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Akhil Das <ak...@sigmoidanalytics.com>
Subject Re: set up spark cluster with heterogeneous hardware
Date Fri, 13 Mar 2015 06:45:11 GMT
You could also add SPARK_MASTER_IP to bind to a specific host/IP so that it
won't get confused with those hosts in your /etc/hosts file.

Thanks
Best Regards

On Fri, Mar 13, 2015 at 12:00 PM, Du Li <lidu@yahoo-inc.com.invalid> wrote:

> Hi Spark community,
>
> I searched for a way to configure a heterogeneous cluster because the need
> recently emerged in my project. I didn't find any solution out there. Now I
> have thought out a solution and thought it might be useful to many other
> people with similar needs. Following is a blog post I just wrote. Feedback
> is welcome.
>
> The capability to accommodate heterogeneous hardware has already been
> there in Spark. We learn how to use it. Oscar goes to those contributors.
> :-)
>
> Du
>
> How to set up a Spark cluster with heterogeneous hardware
> <http://scala4fun.tumblr.com/post/113486166952>
>
>
>
>
>
>
> How to set up a Spark cluster with heterogeneous hardware
> <http://scala4fun.tumblr.com/post/113486166952>
> Apache Spark (http://spark.apache.org) is a big data processing platform
> that has become widely adopted across the industries in just a few years.
> The strength of Spark is to make use of the main memory that is getting
> larger and cheaper to support low-latency use cases.  In a pra...
> View on scala4fun.tumblr.com
> <http://scala4fun.tumblr.com/post/113486166952>
> Preview by Yahoo
>
>
>
>

Mime
View raw message