spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fawze Abujaber <fawz...@gmail.com>
Subject Re: Open sourcing Sparklens: Qubole's Spark Tuning Tool
Date Fri, 23 Mar 2018 04:33:49 GMT
Hi Shmuel,

Did you compile the code against the right branch for Spark 1.6.

I tested it and it looks working and now i'm testing the branch for a wide
tests, Please use the branch for Spark 1.6

On Fri, Mar 23, 2018 at 12:43 AM, Shmuel Blitz <shmuel.blitz@similarweb.com>
wrote:

> Hi Rohit,
>
> Thanks for sharing this great tool.
> I tried running a spark job with the tool, but it failed with an *IncompatibleClassChangeError
> *Exception.
>
> I have opened an issue on Github.(https://github.com/
> qubole/sparklens/issues/1)
>
> Shmuel
>
> On Thu, Mar 22, 2018 at 5:05 PM, Shmuel Blitz <shmuel.blitz@similarweb.com
> > wrote:
>
>> Thanks.
>>
>> We will give this a try and report back.
>>
>> Shmuel
>>
>> On Thu, Mar 22, 2018 at 4:22 PM, Rohit Karlupia <rohitk@qubole.com>
>> wrote:
>>
>>> Thanks everyone!
>>> Please share how it works and how it doesn't. Both help.
>>>
>>> Fawaze, just made few changes to make this work with spark 1.6. Can you
>>> please try building from branch *spark_1.6*
>>>
>>> thanks,
>>> rohitk
>>>
>>>
>>>
>>> On Thu, Mar 22, 2018 at 10:18 AM, Fawze Abujaber <fawzeaj@gmail.com>
>>> wrote:
>>>
>>>> It's super amazing .... i see it was tested on spark 2.0.0 and above,
>>>> what about Spark 1.6 which is still part of Cloudera's main versions?
>>>>
>>>> We have a vast Spark applications with version 1.6.0
>>>>
>>>> On Thu, Mar 22, 2018 at 6:38 AM, Holden Karau <holden@pigscanfly.ca>
>>>> wrote:
>>>>
>>>>> Super exciting! I look forward to digging through it this weekend.
>>>>>
>>>>> On Wed, Mar 21, 2018 at 9:33 PM ☼ R Nair (रविशंकर नायर)
<
>>>>> ravishankar.nair@gmail.com> wrote:
>>>>>
>>>>>> Excellent. You filled a missing link.
>>>>>>
>>>>>> Best,
>>>>>> Passion
>>>>>>
>>>>>> On Wed, Mar 21, 2018 at 11:36 PM, Rohit Karlupia <rohitk@qubole.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> Happy to announce the availability of Sparklens as open source
>>>>>>> project. It helps in understanding the  scalability limits of
spark
>>>>>>> applications and can be a useful guide on the path towards tuning
>>>>>>> applications for lower runtime or cost.
>>>>>>>
>>>>>>> Please clone from here: https://github.com/qubole/sparklens
>>>>>>> Old blogpost: https://www.qubole.com/blog/introducing-quboles-sp
>>>>>>> ark-tuning-tool/
>>>>>>>
>>>>>>> thanks,
>>>>>>> rohitk
>>>>>>>
>>>>>>> PS: Thanks for the patience. It took couple of months to get
back on
>>>>>>> this.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> --
>>>>> Twitter: https://twitter.com/holdenkarau
>>>>>
>>>>
>>>>
>>>
>>
>>
>> --
>> Shmuel Blitz
>> Big Data Developer
>> Email: shmuel.blitz@similarweb.com
>> www.similarweb.com
>> <https://www.facebook.com/SimilarWeb/>
>> <https://www.linkedin.com/company/429838/>
>> <https://twitter.com/similarweb>
>>
>
>
>
> --
> Shmuel Blitz
> Big Data Developer
> Email: shmuel.blitz@similarweb.com
> www.similarweb.com
> <https://www.facebook.com/SimilarWeb/>
> <https://www.linkedin.com/company/429838/>
> <https://twitter.com/similarweb>
>

Mime
View raw message