spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Naveen Madhire <vmadh...@umail.iu.edu>
Subject Re: Unit tests of spark application
Date Mon, 13 Jul 2015 17:04:15 GMT
Thanks. Spark-testing-base works pretty well.

On Fri, Jul 10, 2015 at 3:23 PM, Burak Yavuz <brkyvz@gmail.com> wrote:

> I can +1 Holden's spark-testing-base package.
>
> Burak
>
> On Fri, Jul 10, 2015 at 12:23 PM, Holden Karau <holden@pigscanfly.ca>
> wrote:
>
>> Somewhat biased of course, but you can also use spark-testing-base from
>> spark-packages.org as a basis for your unittests.
>>
>> On Fri, Jul 10, 2015 at 12:03 PM, Daniel Siegmann <
>> daniel.siegmann@teamaol.com> wrote:
>>
>>> On Fri, Jul 10, 2015 at 1:41 PM, Naveen Madhire <vmadhire@umail.iu.edu>
>>> wrote:
>>>
>>>> I want to write junit test cases in scala for testing spark
>>>> application. Is there any guide or link which I can refer.
>>>>
>>>
>>> https://spark.apache.org/docs/latest/programming-guide.html#unit-testing
>>>
>>> Typically I create test data using SparkContext.parallelize and then
>>> call RDD.collect to get the results to assert.
>>>
>>
>>
>>
>> --
>> Cell : 425-233-8271
>> Twitter: https://twitter.com/holdenkarau
>> Linked In: https://www.linkedin.com/in/holdenkarau
>>
>
>

Mime
View raw message