spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shmuel Blitz <shmuel.bl...@similarweb.com>
Subject Re: Open sourcing Sparklens: Qubole's Spark Tuning Tool
Date Thu, 22 Mar 2018 22:43:48 GMT
Hi Rohit,

Thanks for sharing this great tool.
I tried running a spark job with the tool, but it failed with an
*IncompatibleClassChangeError
*Exception.

I have opened an issue on Github.(
https://github.com/qubole/sparklens/issues/1)

Shmuel

On Thu, Mar 22, 2018 at 5:05 PM, Shmuel Blitz <shmuel.blitz@similarweb.com>
wrote:

> Thanks.
>
> We will give this a try and report back.
>
> Shmuel
>
> On Thu, Mar 22, 2018 at 4:22 PM, Rohit Karlupia <rohitk@qubole.com> wrote:
>
>> Thanks everyone!
>> Please share how it works and how it doesn't. Both help.
>>
>> Fawaze, just made few changes to make this work with spark 1.6. Can you
>> please try building from branch *spark_1.6*
>>
>> thanks,
>> rohitk
>>
>>
>>
>> On Thu, Mar 22, 2018 at 10:18 AM, Fawze Abujaber <fawzeaj@gmail.com>
>> wrote:
>>
>>> It's super amazing .... i see it was tested on spark 2.0.0 and above,
>>> what about Spark 1.6 which is still part of Cloudera's main versions?
>>>
>>> We have a vast Spark applications with version 1.6.0
>>>
>>> On Thu, Mar 22, 2018 at 6:38 AM, Holden Karau <holden@pigscanfly.ca>
>>> wrote:
>>>
>>>> Super exciting! I look forward to digging through it this weekend.
>>>>
>>>> On Wed, Mar 21, 2018 at 9:33 PM ☼ R Nair (रविशंकर नायर)
<
>>>> ravishankar.nair@gmail.com> wrote:
>>>>
>>>>> Excellent. You filled a missing link.
>>>>>
>>>>> Best,
>>>>> Passion
>>>>>
>>>>> On Wed, Mar 21, 2018 at 11:36 PM, Rohit Karlupia <rohitk@qubole.com>
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Happy to announce the availability of Sparklens as open source
>>>>>> project. It helps in understanding the  scalability limits of spark
>>>>>> applications and can be a useful guide on the path towards tuning
>>>>>> applications for lower runtime or cost.
>>>>>>
>>>>>> Please clone from here: https://github.com/qubole/sparklens
>>>>>> Old blogpost: https://www.qubole.com/blog/introducing-quboles-sp
>>>>>> ark-tuning-tool/
>>>>>>
>>>>>> thanks,
>>>>>> rohitk
>>>>>>
>>>>>> PS: Thanks for the patience. It took couple of months to get back
on
>>>>>> this.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>> --
>>>> Twitter: https://twitter.com/holdenkarau
>>>>
>>>
>>>
>>
>
>
> --
> Shmuel Blitz
> Big Data Developer
> Email: shmuel.blitz@similarweb.com
> www.similarweb.com
> <https://www.facebook.com/SimilarWeb/>
> <https://www.linkedin.com/company/429838/>
> <https://twitter.com/similarweb>
>



-- 
Shmuel Blitz
Big Data Developer
Email: shmuel.blitz@similarweb.com
www.similarweb.com
<https://www.facebook.com/SimilarWeb/>
<https://www.linkedin.com/company/429838/> <https://twitter.com/similarweb>

Mime
View raw message