spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mayur Rustagi <mayur.rust...@gmail.com>
Subject Re: Spark on google compute engine
Date Mon, 13 Jan 2014 16:51:14 GMT
They are in active development but themselves ask not be used in
production.I can create one to play around I guess.
Regards
Mayur

Mayur Rustagi
Ph: +919632149971
h <https://twitter.com/mayur_rustagi>ttp://www.sigmoidanalytics.com
https://twitter.com/mayur_rustagi



On Mon, Jan 13, 2014 at 9:40 PM, Aureliano Buendia <buendia360@gmail.com>wrote:

> Why not using docker for spark? GCE recently supports custom kernels
> allowing docker to work on it.
>
> A docket image can be shared between GCE and EC2.
>
>
> On Mon, Jan 13, 2014 at 8:56 AM, Mayur Rustagi <mayur.rustagi@gmail.com>wrote:
>
>> Hi,
>> I have setup a basic guide here.
>>
>>
>> http://docs.sigmoidanalytics.com/index.php/How_to_Install_Spark_on_Google_Compute_Engine
>>
>> GCE unfortunately doesnt provide a easy way to manage images( It has
>> interface to download the tar.gz image and then reload it, I didnt find it
>> as convinient as AMI). If you create one let me know and I'll update the
>> guide accordingly. If you create a sharable image let me know i'll host it
>> on S3 and make it available for all.
>>
>> Regards
>> Mayur
>>
>>
>> Mayur Rustagi
>> Ph: +919632149971
>> h <https://twitter.com/mayur_rustagi>ttp://www.sigmoidanalytics.com
>> https://twitter.com/mayur_rustagi
>>
>>
>>
>> On Mon, Jan 13, 2014 at 11:01 AM, Debasish Das <debasish.das83@gmail.com>wrote:
>>
>>> Hi Aureliano,
>>>
>>> Look for google compute engine scripts from typesafe repo. They recently
>>> tested Akka Cluster on 2400 nodes from Google Compute Engine. You should be
>>> able to reuse the scripts.
>>>
>>> Thanks.
>>> Deb
>>>
>>>
>>> On Sun, Jan 12, 2014 at 8:00 PM, Aureliano Buendia <buendia360@gmail.com
>>> > wrote:
>>>
>>>> Hi,
>>>>
>>>> Has anyone worked on a script similar to spark-ec2 for google compute
>>>> engine?
>>>>
>>>> Google compute engine claims that they have faster instance start up
>>>> time, and that together with by minute charging makes it a desirable choice
>>>> for spark.
>>>>
>>>
>>>
>>
>

Mime
View raw message