spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abdeali Kothari <abdealikoth...@gmail.com>
Subject Show function name in Logs for PythonUDFRunner
Date Thu, 22 Nov 2018 09:04:10 GMT
When I run Python UDFs with pyspark, I get multiple logs where it says:

18/11/22 01:51:59 INFO python.PythonUDFRunner: Times: total = 44, boot
= -25, init = 67, finish = 2


I am wondering if in these logs I can identify easily which of my
PythonUDFs this timing information is for (I have about a hundred) so it's
quite difficult for em to identify this easily ...

Mime
View raw message