spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ganelin, Ilya" <Ilya.Gane...@capitalone.com>
Subject RE: Appropriate way to add a debug flag
Date Fri, 07 Nov 2014 18:04:58 GMT
That perspective - identifying the non serializable components - is actually very helpful.
I'll look into figuring out a way to do this. Thanks !

-----Original Message-----
From: Josh Rosen [rosenville@gmail.com<mailto:rosenville@gmail.com>]
Sent: Friday, November 07, 2014 12:51 PM Eastern Standard Time
To: Ganelin, Ilya; dev
Subject: Re: Appropriate way to add a debug flag

(Whoops, forgot to copy dev@ in my original reply; adding it back)

Yeah, the GraphViz part was mostly for fun and for understanding cyclic object graphs.  In
general, an object graph might contain cycles, so for understanding the overall structure
it's handy to have a picture.  The GraphViz thing is actually pretty fun to play with in an
interactive notebook environment, since even fairly simple programs can produce really interesting
object graphs.

For the purposes of debugging serialization errors, though, I guess you only need to know
about some path of non-transient fields that leads from the target object to an unserializable
object.  For that case, you might be able to add a try-catch block that performs an object
graph traversal to find a path to a non-serializable object if a serialization error occurs.
 Would logging this path be sufficient to debug the most common serialization issues, such
as unexpected over-capture of non-serializable objects in closures?

I guess that some users might also want a more general object graph printer / debugger to
help debug performance issues related to over-captures that do not lead to errors, but that
might be a lower priority / could happen in a separate PR.

Another option would be to do something like http://blog.crazybob.org/2007/02/debugging-serialization.html
to print a trace from the serializer's point of view, but the output from that might be hard
to understand since it could obscure the chain of references / fields that led to the error.

- Josh

On Thu, Nov 6, 2014 at 12:01 PM, Ganelin, Ilya <Ilya.Ganelin@capitalone.com<mailto:Ilya.Ganelin@capitalone.com>>
wrote:
Hi Josh – I think this could be useful for visualizing references in RDDs but I actually
wasn’t sure that this was that the original issue wanted in terms of a solution. I assumed
the the more useful output would be a string output. E.g.

RDD
 - Child 1
  — Child 1.1
  — Child 1.2
 - Child 2
 - Child 3

So that it’s readily integrated with the Spark logs. Would you agree?

I like the SparkConf idea, I will look into that.
From: Josh Rosen <rosenville@gmail.com<mailto:rosenville@gmail.com>>
Date: Thursday, November 6, 2014 at 2:42 PM
To: "Ganelin, Ilya" <ilya.ganelin@capitalone.com<mailto:ilya.ganelin@capitalone.com>>
Subject: Re: Appropriate way to add a debug flag

This is timely, since I’ve actually been hacking on some related stuff in order to debug
whether unexpected objects are being pulled into closures.  Here’s some code to print a
graphviz DOT file that shows the graph of non-transient, non-primitive objects reachable from
a given object: https://gist.github.com/JoshRosen/d6a8972c99992e97d040

For enabling / disabling automatic logging of this, I suppose that you could add a configuration
option to SparkConf.


On November 5, 2014 at 8:02:35 AM, Ganelin, Ilya (ilya.ganelin@capitalone.com<mailto:ilya.ganelin@capitalone.com>)
wrote:

Hello all – I am working on https://issues.apache.org/jira/browse/SPARK-3694 and would like
to understand the appropriate mechanism by which to check for a debug flag before printing
a graph traversal of dependencies of an RDD or Task. I understand that I can use the logging
utility and use logDebug to actually print the output but the graph traversal should not be
executed unless the debug output is enabled. The code changes I will be making will be in
the DAGScheduler and TaskSetManager classes.

Modifying the function interfaces does not seem like the appropriate approach . Is there an
existing debug flag that is set somehow within the spark config?
________________________________________________________

The information contained in this e-mail is confidential and/or proprietary to Capital One
and/or its affiliates. The information transmitted herewith is intended only for use by the
individual or entity to which it is addressed. If the reader of this message is not the intended
recipient, you are hereby notified that any review, retransmission, dissemination, distribution,
copying or other use of, or taking of any action in reliance upon this information is strictly
prohibited. If you have received this communication in error, please contact the sender and
delete the material from your computer.

________________________________

The information contained in this e-mail is confidential and/or proprietary to Capital One
and/or its affiliates. The information transmitted herewith is intended only for use by the
individual or entity to which it is addressed.  If the reader of this message is not the intended
recipient, you are hereby notified that any review, retransmission, dissemination, distribution,
copying or other use of, or taking of any action in reliance upon this information is strictly
prohibited. If you have received this communication in error, please contact the sender and
delete the material from your computer.

________________________________________________________

The information contained in this e-mail is confidential and/or proprietary to Capital One
and/or its affiliates. The information transmitted herewith is intended only for use by the
individual or entity to which it is addressed.  If the reader of this message is not the intended
recipient, you are hereby notified that any review, retransmission, dissemination, distribution,
copying or other use of, or taking of any action in reliance upon this information is strictly
prohibited. If you have received this communication in error, please contact the sender and
delete the material from your computer.
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message