spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yi Tian <tianyi.asiai...@gmail.com>
Subject Re: SparkSQL tasks spend too much time to finish.
Date Mon, 26 Jan 2015 10:22:11 GMT
Hi, San

You need to provide more information to diagnose this problem, like :

 1. What kind of SQL did you execute?
 2. If there are some |group| operation in this SQL, could you do some
    statistic about how many unique group keys in this case?

On 1/26/15 17:01, luohui20001@sina.com wrote:

> Hi there,
>
>        When running a sql query, i found abnormal time cost of tasks 
> like the attached pic. it runs fast in the first few tasks, but 
> extremely slow in later tasks, which speed 100X more time than early 
> ones.However,they are dealing with same size data at 128mb....
>
>        standalone & pseudo distributed cluster
>        executor memory:40g
>
>        driver momory:5g
>
> Any advices will be appreciated.
>
> --------------------------------
>
> Thanks&amp;Best regards!
> 罗辉 San.Luo
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org

​

Mime
View raw message