spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Donovan <patrick.dono...@jadedpixel.com>
Subject pyspark python exceptions / py4j exceptions
Date Thu, 08 May 2014 19:52:05 GMT
Hello,

I'm trying to write a python function that does something like:

def foo(line):
    try:
        return stuff(line)
    except Exception:
        raise MoreInformativeException(line)

and then use it in a map like so:

rdd.map(foo)

and have my MoreInformativeException make it back if/when something goes
wrong, but all I can get back are errors like:

Py4JJavaError(u'An error occurred while calling o50.saveAsTextFile.\n',
JavaObject id=o51)

Is there any way to recover my MoreInformativeException back in python
land? If I reach into java via py4j can I get it back somehow? If so, how?

Thanks!

Mime
View raw message