spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shmuel Blitz <shmuel.bl...@similarweb.com>
Subject Re: Open sourcing Sparklens: Qubole's Spark Tuning Tool
Date Thu, 22 Mar 2018 15:05:43 GMT
Thanks.

We will give this a try and report back.

Shmuel

On Thu, Mar 22, 2018 at 4:22 PM, Rohit Karlupia <rohitk@qubole.com> wrote:

> Thanks everyone!
> Please share how it works and how it doesn't. Both help.
>
> Fawaze, just made few changes to make this work with spark 1.6. Can you
> please try building from branch *spark_1.6*
>
> thanks,
> rohitk
>
>
>
> On Thu, Mar 22, 2018 at 10:18 AM, Fawze Abujaber <fawzeaj@gmail.com>
> wrote:
>
>> It's super amazing .... i see it was tested on spark 2.0.0 and above,
>> what about Spark 1.6 which is still part of Cloudera's main versions?
>>
>> We have a vast Spark applications with version 1.6.0
>>
>> On Thu, Mar 22, 2018 at 6:38 AM, Holden Karau <holden@pigscanfly.ca>
>> wrote:
>>
>>> Super exciting! I look forward to digging through it this weekend.
>>>
>>> On Wed, Mar 21, 2018 at 9:33 PM ☼ R Nair (रविशंकर नायर)
<
>>> ravishankar.nair@gmail.com> wrote:
>>>
>>>> Excellent. You filled a missing link.
>>>>
>>>> Best,
>>>> Passion
>>>>
>>>> On Wed, Mar 21, 2018 at 11:36 PM, Rohit Karlupia <rohitk@qubole.com>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Happy to announce the availability of Sparklens as open source
>>>>> project. It helps in understanding the  scalability limits of spark
>>>>> applications and can be a useful guide on the path towards tuning
>>>>> applications for lower runtime or cost.
>>>>>
>>>>> Please clone from here: https://github.com/qubole/sparklens
>>>>> Old blogpost: https://www.qubole.com/blog/introducing-quboles-sp
>>>>> ark-tuning-tool/
>>>>>
>>>>> thanks,
>>>>> rohitk
>>>>>
>>>>> PS: Thanks for the patience. It took couple of months to get back on
>>>>> this.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>> --
>>> Twitter: https://twitter.com/holdenkarau
>>>
>>
>>
>


-- 
Shmuel Blitz
Big Data Developer
Email: shmuel.blitz@similarweb.com
www.similarweb.com
<https://www.facebook.com/SimilarWeb/>
<https://www.linkedin.com/company/429838/> <https://twitter.com/similarweb>

Mime
View raw message