spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nicholas Chammas <nicholas.cham...@gmail.com>
Subject Running Spark through a debugger
Date Tue, 17 Dec 2019 05:26:52 GMT
I normally stick to the Python parts of Spark, but I am interested in
walking through the DSv2 code and understanding how it works. I tried
following the "IDE Setup" section of the developer tools
<https://spark.apache.org/developer-tools.html> page, but quickly hit
several problems loading the project.

Can anyone point to a tutorial or guide on running Spark through a
debugger, for folks like me who are not regular IntelliJ or Scala users?

My goal is just to be able to run something like
`spark.read.text(...).show()` and follow it step by step through the
codebase.

Nick

Mime
View raw message