spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nicholas Chammas <>
Subject Re: Run a specific PySpark test or group of tests
Date Tue, 15 Aug 2017 12:49:31 GMT
Pytest does support unittest-based tests
<>, allowing for incremental
adoption. I'll see how convenient it is to use with our current test layout.

On Tue, Aug 15, 2017 at 1:03 AM Hyukjin Kwon <> wrote:

> For me, I would like this if this can be done with relatively small
> changes.
> How about adding more granular options, for example, specifying or
> filtering smaller set of test goals in the script?
> I think it'd be quite small change and we could roughly reach this goal if
> I understood correctly.
> 2017-08-15 3:06 GMT+09:00 Nicholas Chammas <>:
>> Say you’re working on something and you want to rerun the PySpark tests,
>> focusing on a specific test or group of tests. Is there a way to do that?
>> I know that you can test entire modules with this:
>> ./python/run-tests --modules pyspark-sql
>> But I’m looking for something more granular, like pytest’s -k option.
>> On that note, does anyone else think it would be valuable to use a test
>> runner like pytest to run our Python tests? The biggest benefits would be
>> the use of fixtures <>,
>> and more flexibility on test running and reporting. Just wondering if we’ve
>> already considered this.
>> Nick
>> ​

View raw message