spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rohit Karlupia <roh...@qubole.com>
Subject Re: Open sourcing Sparklens: Qubole's Spark Tuning Tool
Date Thu, 22 Mar 2018 14:22:22 GMT
Thanks everyone!
Please share how it works and how it doesn't. Both help.

Fawaze, just made few changes to make this work with spark 1.6. Can you
please try building from branch *spark_1.6*

thanks,
rohitk



On Thu, Mar 22, 2018 at 10:18 AM, Fawze Abujaber <fawzeaj@gmail.com> wrote:

> It's super amazing .... i see it was tested on spark 2.0.0 and above, what
> about Spark 1.6 which is still part of Cloudera's main versions?
>
> We have a vast Spark applications with version 1.6.0
>
> On Thu, Mar 22, 2018 at 6:38 AM, Holden Karau <holden@pigscanfly.ca>
> wrote:
>
>> Super exciting! I look forward to digging through it this weekend.
>>
>> On Wed, Mar 21, 2018 at 9:33 PM ☼ R Nair (रविशंकर नायर) <
>> ravishankar.nair@gmail.com> wrote:
>>
>>> Excellent. You filled a missing link.
>>>
>>> Best,
>>> Passion
>>>
>>> On Wed, Mar 21, 2018 at 11:36 PM, Rohit Karlupia <rohitk@qubole.com>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> Happy to announce the availability of Sparklens as open source project.
>>>> It helps in understanding the  scalability limits of spark applications and
>>>> can be a useful guide on the path towards tuning applications for lower
>>>> runtime or cost.
>>>>
>>>> Please clone from here: https://github.com/qubole/sparklens
>>>> Old blogpost: https://www.qubole.com/blog/introducing-quboles-sp
>>>> ark-tuning-tool/
>>>>
>>>> thanks,
>>>> rohitk
>>>>
>>>> PS: Thanks for the patience. It took couple of months to get back on
>>>> this.
>>>>
>>>>
>>>>
>>>>
>>>>
>>> --
>> Twitter: https://twitter.com/holdenkarau
>>
>
>

Mime
View raw message