spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jay Vyas <jayunit100.apa...@gmail.com>
Subject Re: Unit testing and Spark Streaming
Date Fri, 12 Dec 2014 16:21:34 GMT
https://github.com/jayunit100/SparkStreamingCassandraDemo
 
On this note, I've built a framework which is mostly "pure" so that functional unit tests
can be run composing mock data for Twitter statuses, with just regular junit... That might
be relevant also.

I think at some point we should come up with a robust test driven  framework for building
stream apps... And the idea of Scala test with the injection and comparison you did might
be a good start.

Thanks for starting this dialogue!

> On Dec 12, 2014, at 9:18 AM, Emre Sevinc <emre.sevinc@gmail.com> wrote:
> 
> On Fri, Dec 12, 2014 at 2:17 PM, Eric Loots <eric.loots@gmail.com> wrote:
> > How can the log level in test mode be reduced (or extended when needed) ?
> 
> Hello Eric,
> 
> The following might be helpful for reducing the log messages during unit testing:
> 
>  http://stackoverflow.com/a/27360000/236007
> 
> --
> Emre Sevinç
> https://be.linkedin.com/in/emresevinc
> 

Mime
View raw message