spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Owen <>
Subject Re: Running Spark through a debugger
Date Tue, 17 Dec 2019 05:36:24 GMT
I just make a new test suite or something, set breakpoints, and
execute it in IJ. That generally works fine. You may need to set the
run configuration to have the right working dir (Spark project root),
and set the right system property to say 'this is running in a test'
in some cases. What are you having trouble with, does it build?

On Mon, Dec 16, 2019 at 11:27 PM Nicholas Chammas
<> wrote:
> I normally stick to the Python parts of Spark, but I am interested in walking through
the DSv2 code and understanding how it works. I tried following the "IDE Setup" section of
the developer tools page, but quickly hit several problems loading the project.
> Can anyone point to a tutorial or guide on running Spark through a debugger, for folks
like me who are not regular IntelliJ or Scala users?
> My goal is just to be able to run something like `` and follow
it step by step through the codebase.
> Nick

To unsubscribe e-mail:

View raw message