spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Huang, Jie" <>
Subject RE: [SparkScore]Performance portal for Apache Spark - WW26
Date Fri, 26 Jun 2015 12:17:44 GMT
Correct. Your calculation is right!

We have been aware of that kmeans performance drop also. According to our observation, it
is caused by some unbalanced executions among different tasks. Even we used the same test
data between different versions (i.e., not caused by the data skew).

And the corresponding run time information has been shared with Xiangrui. Now he is also helping
to identify the root cause altogether.

Thank you && Best Regards,
Grace (Huang Jie)

From: Nan Zhu []
Sent: Friday, June 26, 2015 7:59 PM
To: Huang, Jie
Subject: Re: [SparkScore]Performance portal for Apache Spark - WW26

Hi, Jie,

Thank you very much for this work! Very helpful!

I just would like to confirm that I understand the numbers correctly: if we take the running
time of 1.2 release as 100s

9.1% - means the running time is 109.1 s?

-4% - means it comes 96s?

If that’s the true meaning of the numbers, what happened to k-means in HiBench?


Nan Zhu

On Friday, June 26, 2015 at 7:24 AM, Huang, Jie wrote:
Intel® Xeon® CPU E5-2697

View raw message