spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nan Zhu <zhunanmcg...@gmail.com>
Subject Re: [SparkScore]Performance portal for Apache Spark - WW26
Date Fri, 26 Jun 2015 12:21:11 GMT
Thank you, Jie! Very nice work!

--  
Nan Zhu
http://codingcat.me


On Friday, June 26, 2015 at 8:17 AM, Huang, Jie wrote:

> Correct. Your calculation is right!  
>   
> We have been aware of that kmeans performance drop also. According to our observation,
it is caused by some unbalanced executions among different tasks. Even we used the same test
data between different versions (i.e., not caused by the data skew).
>   
> And the corresponding run time information has been shared with Xiangrui. Now he is also
helping to identify the root cause altogether.  
>   
> Thank you && Best Regards,
> Grace (Huang Jie)
>   
> From: Nan Zhu [mailto:zhunanmcgill@gmail.com]  
> Sent: Friday, June 26, 2015 7:59 PM
> To: Huang, Jie
> Cc: user@spark.apache.org (mailto:user@spark.apache.org); dev@spark.apache.org (mailto:dev@spark.apache.org)
> Subject: Re: [SparkScore]Performance portal for Apache Spark - WW26  
>   
> Hi, Jie,  
>  
>   
>  
> Thank you very much for this work! Very helpful!
>  
>   
>  
> I just would like to confirm that I understand the numbers correctly: if we take the
running time of 1.2 release as 100s
>  
>   
>  
> 9.1% - means the running time is 109.1 s?
>  
>   
>  
> -4% - means it comes 96s?
>  
>   
>  
> If that’s the true meaning of the numbers, what happened to k-means in HiBench?
>  
>   
>  
> Best,
>  
>   
>  
> --  
>  
> Nan Zhu
>  
> http://codingcat.me
>  
>   
>  
>  
> On Friday, June 26, 2015 at 7:24 AM, Huang, Jie wrote:
> > Intel® Xeon® CPU E5-2697  
> >  
>  
>   
>  
>  
>  
>  



Mime
View raw message