This is an automated email from the ASF dual-hosted git repository.
eyal pushed a change to branch spark-tmp
in repository https://gitbox.apache.org/repos/asf/datafu.git.
from c33c63a Make tests pass for all Spark/Scala combinations and add build script
new 8a5ccc8 code reformatting according to Spark's scalastyle
new a410e1c rename example_tests
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../META-INF/services/datafu.spark.PythonResource | 2 +-
.../src/main/scala/datafu/spark/DataFrameOps.scala | 65 ++-
.../scala/datafu/spark/PythonPathsManager.scala | 141 +++---
.../scala/datafu/spark/ScalaPythonBridge.scala | 58 ++-
.../src/main/scala/datafu/spark/SparkDFUtils.scala | 408 +++++++++++-----
.../src/main/scala/datafu/spark/SparkUDAFs.scala | 168 ++++---
.../utils/overwrites/SparkOverwriteUDAFs.scala | 38 +-
.../spark/utils/overwrites/SparkPythonRunner.scala | 69 ++-
.../META-INF/services/datafu.spark.PythonResource | 2 +-
.../df_utils_tests.py | 0
.../{example_tests => python_tests}/pyfromscala.py | 4 +-
.../pyfromscala_with_error.py | 0
.../datafu/spark/PySparkLibTestResources.scala | 8 +-
.../scala/datafu/spark/TestScalaPythonBridge.scala | 74 +--
.../test/scala/datafu/spark/TestSparkDFUtils.scala | 528 ++++++++++++---------
.../test/scala/datafu/spark/TestSparkUDAFs.scala | 230 +++++----
16 files changed, 1138 insertions(+), 657 deletions(-)
rename datafu-spark/src/test/resources/{example_tests => python_tests}/df_utils_tests.py
(100%)
rename datafu-spark/src/test/resources/{example_tests => python_tests}/pyfromscala.py
(97%)
rename datafu-spark/src/test/resources/{example_tests => python_tests}/pyfromscala_with_error.py
(100%)
|