spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Holden Karau <hol...@pigscanfly.ca>
Subject Re: [PYSPARK] Python tests organization
Date Thu, 12 Jan 2017 18:34:32 GMT
I'd be happy to help with reviewing Python test improvements. Maybe make an
umbrella JIRA and do one sub components at a time?

On Thu, Jan 12, 2017 at 12:20 PM Saikat Kanjilal <sxk1969@hotmail.com>
wrote:

>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Following up, any thoughts on next steps for this?
>
>
>
>
>
>
>
>
>
>
> ------------------------------
>
>
> *From:* Maciej Szymkiewicz <mszymkiewicz@gmail.com>
>
>
>
> *Sent:* Wednesday, January 11, 2017 10:14 AM
>
>
> *To:* Saikat Kanjilal
>
>
> *Subject:* Re: [PYSPARK] Python tests organization
>
>
> Not yet, I want to see if there is any consensus about it. It is a lot of
> tedious work and I would be shame if someone started working on this just
> to get it dropped.
>
>
>
>
>
>
>
> On 01/11/2017 06:44 PM, Saikat Kanjilal wrote:
>
>
>
>
>
>
>
>
> Hello Maciej,
>
>
> If there's a jira available for this I'd like to help get this moving, let
> me know next steps.
>
>
> Thanks in advance.
>
>
>
>
>
>
>
>
>
>
>
>
> ------------------------------
>
>
> *From:* Maciej Szymkiewicz
>
> <mszymkiewicz@gmail.com> <mszymkiewicz@gmail.com>
>
>
> *Sent:* Wednesday, January 11, 2017 4:18 AM
>
>
> *To:*
>
> dev@spark.apache.org
>
>
> *Subject:* [PYSPARK] Python tests organization
>
>
>
>
>
>
>
>
>
>
> Hi,
>
>
>
>
>
> I can't help but wonder if there is any practical reason for keeping
>
>
> monolithic test modules. These things are already pretty large (1500 -
>
>
> 2200 LOCs) and can only grow. Development aside, I assume that many
>
>
> users use tests the same way as me, to check the intended behavior, and
>
>
> largish loosely coupled modules make it harder than it should be.
>
>
>
>
>
> If there's no rationale for that it could be a good time start thinking
>
>
> about moving tests to packages and separating into modules reflecting
>
>
> project structure.
>
>
>
>
>
> --
>
>
> Best,
>
>
> Maciej
>
>
>
>
>
>
>
>
> ---------------------------------------------------------------------
>
>
> To unsubscribe e-mail:
>
> dev-unsubscribe@spark.apache.org
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> --
> Maciej Szymkiewicz
>
>
>
>

Mime
View raw message