spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacek Laskowski <ja...@japila.pl>
Subject Re: Log messages for shuffle phase
Date Fri, 12 Aug 2016 03:15:28 GMT
Hi,

Have you looked at web UI? You should find such task metrics.

Jacek

On 11 Aug 2016 6:28 p.m., "Suman Somasundar" <suman.somasundar@oracle.com>
wrote:

> Hi,
>
>
>
> While going through the logs of an application, I noticed that I could not
> find any logs to dig deeper into any of the shuffle phases.
>
>
>
> I am interested in finding out time taken by each shuffle phase, the size
> of data spilled to disk if any, among other things.
>
>
>
> Does anyone know how I can do this?
>
>
>
> Thanks,
>
> Suman.
>

Mime
View raw message