spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Takeshi Yamamuro <linguin....@gmail.com>
Subject Re: Debugging a spark application in a none lazy mode
Date Mon, 12 Sep 2016 16:24:16 GMT
Hi,

Spark does not have such mode.
How about getting local arrays by `collect` methods for debugging?

// maropu

On Tue, Sep 13, 2016 at 12:44 AM, Hagai <hattias@akamai.com> wrote:

> Hi guys,
> Lately I was looking for a way to debug my spark application locally.
>
> However, since all transformations are actually being executed when the
> action is encountered, I have no way to look at the data after each
> transformation. Does spark support a non-lazy mode which enables to execute
> the transformations locally after each statement?
>
> Thanks,
> Hagai.
>
>
>
>
> --
> View this message in context: http://apache-spark-user-list.
> 1001560.n3.nabble.com/Debugging-a-spark-application-
> in-a-none-lazy-mode-tp27695.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>
>


-- 
---
Takeshi Yamamuro

Mime
View raw message