I would also would like to participate on that.Greetings,JuanOn Fri, Dec 9, 2016 at 6:03 AM, Michael Stratton <michael.stratton@
komodohealth.com> wrote:That sounds great, please include me so I can get involved.On Fri, Dec 9, 2016 at 7:39 AM, Marco Mistroni <firstname.lastname@example.org> wrote:Me too as I spent most of my time writing unit/integ tests.... pls advise on where I can startKrOn 9 Dec 2016 12:15 am, "Miguel Morales" <email@example.com> wrote:I would be interested in contributing. Ive created my own library for this as well. In my blog post I talk about testing with Spark in RSpec style:
Sent from my iPhone
On Dec 8, 2016, at 4:09 PM, Holden Karau <firstname.lastname@example.org> wrote:There are also libraries designed to simplify testing Spark in the various platforms, spark-testing-base for Scala/Java/Python (& video https://www.youtube.com/watch?
v=f69gSGSLGrY), sscheck (scala focused property based), pyspark.test (python focused with py.test instead of unittest2) (& blog post from nextdoor https://engblog.nextd oor.com/unit-testing-apache-sp) ark-with-py-test-3b8970dc013b# .jw3bdcej9Good luck on your Spark Adventures :)P.S.If anyone is interested in helping improve spark testing libraries I'm always looking for more people to be involved with spark-testing-base because I'm lazy :pOn Thu, Dec 8, 2016 at 2:05 PM, Lars Albertsson <email@example.com> wrote:I wrote some advice in a previous post on the list:
It does not mention python, but the strategy advice is the same. Just
replace JUnit/Scalatest with pytest, unittest, or your favourite
python test framework.
I recently held a presentation on the subject. There is a video
recording at https://vimeo.com/192429554 and slides at
You can find more material on test strategies at
Data engineering consultant
+46 70 7687109
Calendar: https://goo.gl/6FBtlS, https://freebusy.io/lalle@mapf
On Thu, Dec 8, 2016 at 4:14 PM, pseudo oduesp <firstname.lastname@example.org> wrote:
> somone can tell me how i can make unit test on pyspark ?
> (book, tutorial ...)
To unsubscribe e-mail: email@example.com.