spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vitaliy Pisarev <vitaliy.pisa...@biocatch.com>
Subject Debugging a local spark executor in pycharm
Date Sun, 11 Mar 2018 15:46:15 GMT
I want to step through the work of a spark executor running locally on my
machine, from Pycharm.

I am running explicit functionality, in the form of
dataset.foreachPartition(f) and I want to see what is going on inside f.

Is there a straightforward way to do it or do I need to resort to remote
debugging?
p.s

Posted this on SO
<https://stackoverflow.com/questions/49221733/debugging-a-local-spark-executor-in-pycharm>
as well.

Mime
View raw message