spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eugene Morozov <fathers...@list.ru>
Subject Re: Debugging Spark job in Eclipse
Date Wed, 05 Aug 2015 14:00:49 GMT
Deepesh, 

you have to call an action to start actual processing.
words.count() would do the trick.


On 05 Aug 2015, at 11:42, Deepesh Maheshwari <deepesh.maheshwari17@gmail.com> wrote:

> Hi,
> 
> As spark job is executed when you run start() method of JavaStreamingContext.
> All the job like map, flatMap is already defined earlier but even though you put breakpoints
in the function ,breakpoint doesn't stop there , then how can i debug the spark jobs.
> 
> JavaDStream<String> words=lines.flatMap(new FlatMapFunction<String, String>()
{
>             private static final long serialVersionUID = -2042174881679341118L;
> 
>             @Override
>             public Iterable<String> call(String t) throws Exception {
> 
> // Mark Debug Point here, it doesn't stop here.
> 
>                 return Lists.newArrayList(SPACE.split(t));
>             }
>         });
> 
> Please suggest how can i saw the in-between data values.
> 
> Regards,
> Deepesh

Eugene Morozov
fathersson@list.ru





Mime
View raw message