spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Samarth Mailinglist <mailinglistsama...@gmail.com>
Subject Re: Scala vs Python performance differences
Date Wed, 12 Nov 2014 10:16:17 GMT
I was about to ask this question.

On Wed, Nov 12, 2014 at 3:42 PM, Andrew Ash <andrew@andrewash.com> wrote:

> Jeremy,
>
> Did you complete this benchmark in a way that's shareable with those
> interested here?
>
> Andrew
>
> On Tue, Apr 15, 2014 at 2:50 PM, Nicholas Chammas <
> nicholas.chammas@gmail.com> wrote:
>
>> I'd also be interested in seeing such a benchmark.
>>
>>
>> On Tue, Apr 15, 2014 at 9:25 AM, Ian Ferreira <ianferreira@hotmail.com>
>> wrote:
>>
>>> This would be super useful. Thanks.
>>>
>>> On 4/15/14, 1:30 AM, "Jeremy Freeman" <freeman.jeremy@gmail.com> wrote:
>>>
>>> >Hi Andrew,
>>> >
>>> >I'm putting together some benchmarks for PySpark vs Scala. I'm focusing
>>> on
>>> >ML algorithms, as I'm particularly curious about the relative
>>> performance
>>> >of
>>> >MLlib in Scala vs the Python MLlib API vs pure Python implementations.
>>> >
>>> >Will share real results as soon as I have them, but roughly, in our
>>> hands,
>>> >that 40% number is ballpark correct, at least for some basic operations
>>> >(e.g
>>> >textFile, count, reduce).
>>> >
>>> >-- Jeremy
>>> >
>>> >---------------------
>>> >Jeremy Freeman, PhD
>>> >Neuroscientist
>>> >@thefreemanlab
>>> >
>>> >
>>> >
>>> >--
>>> >View this message in context:
>>> >
>>> http://apache-spark-user-list.1001560.n3.nabble.com/Scala-vs-Python-perfor
>>> >mance-differences-tp4247p4261.html
>>> >Sent from the Apache Spark User List mailing list archive at Nabble.com.
>>>
>>>
>>>
>>
>

Mime
View raw message