While going through the logs of an application, I noticed that I could not find any logs to dig deeper into any of the shuffle phases.


I am interested in finding out time taken by each shuffle phase, the size of data spilled to disk if any, among other things.


Does anyone know how I can do this?