spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Imran Rashid <iras...@cloudera.com.INVALID>
Subject Re: python tests: any reason for a huge tests.py?
Date Wed, 05 Sep 2018 18:27:47 GMT
I filed https://issues.apache.org/jira/browse/SPARK-25344

On Fri, Aug 24, 2018 at 11:57 AM Reynold Xin <rxin@databricks.com> wrote:

> We should break it.
>
> On Fri, Aug 24, 2018 at 9:53 AM Imran Rashid <irashid@cloudera.com.invalid>
> wrote:
>
>> Hi,
>>
>> another question from looking more at python recently.  Is there any
>> reason we've got a ton of tests in one humongous tests.py file, rather than
>> breaking it out into smaller files?
>>
>> Having one huge file doesn't seem great for code organization, and it
>> also makes the test parallelization in run-tests.py not work as well.  On
>> my laptop, tests.py takes 150s, and the next longest test file takes only
>> 20s.
>>
>> can we at least try to put new tests into smaller files?
>>
>> thanks,
>> Imran
>>
>

Mime
View raw message