spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hyukjin Kwon <gurwls...@gmail.com>
Subject New PySpark test style
Date Wed, 14 Nov 2018 06:57:24 GMT
Hi all,

Lately, https://github.com/apache/spark/pull/23021 is merged, which tries
to a big single file that contains all the tests into smaller files.

I picked up one example and follow, NumPy. Because the current style looks
closer to NumPy structure and looks easier to follow. Please see
https://github.com/numpy/numpy/tree/master/numpy.

I would like to say:

1. If you were working on PySpark changes, I am terribly sorry that I
caused some conflicts. Please take a look for the new styles and get used
it.

2. Probably, I rushed a bit. I was scared if it's going to get conflicts
again and again but please take a look and see if there's outstanding
structural issues. I am willing to fix or revert at worst case.

3. Now, PySpark's SQL test were newly split but other tests will be split
too in the same manner.

Thank you so much.

Mime
View raw message