datafu-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From e...@apache.org
Subject [datafu] branch spark-tmp updated (c33c63a -> a410e1c)
Date Wed, 15 May 2019 14:13:35 GMT
This is an automated email from the ASF dual-hosted git repository.

eyal pushed a change to branch spark-tmp
in repository https://gitbox.apache.org/repos/asf/datafu.git.


    from c33c63a  Make tests pass for all Spark/Scala combinations and add build script
     new 8a5ccc8  code reformatting according to Spark's scalastyle
     new a410e1c  rename example_tests

The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 .../META-INF/services/datafu.spark.PythonResource  |   2 +-
 .../src/main/scala/datafu/spark/DataFrameOps.scala |  65 ++-
 .../scala/datafu/spark/PythonPathsManager.scala    | 141 +++---
 .../scala/datafu/spark/ScalaPythonBridge.scala     |  58 ++-
 .../src/main/scala/datafu/spark/SparkDFUtils.scala | 408 +++++++++++-----
 .../src/main/scala/datafu/spark/SparkUDAFs.scala   | 168 ++++---
 .../utils/overwrites/SparkOverwriteUDAFs.scala     |  38 +-
 .../spark/utils/overwrites/SparkPythonRunner.scala |  69 ++-
 .../META-INF/services/datafu.spark.PythonResource  |   2 +-
 .../df_utils_tests.py                              |   0
 .../{example_tests => python_tests}/pyfromscala.py |   4 +-
 .../pyfromscala_with_error.py                      |   0
 .../datafu/spark/PySparkLibTestResources.scala     |   8 +-
 .../scala/datafu/spark/TestScalaPythonBridge.scala |  74 +--
 .../test/scala/datafu/spark/TestSparkDFUtils.scala | 528 ++++++++++++---------
 .../test/scala/datafu/spark/TestSparkUDAFs.scala   | 230 +++++----
 16 files changed, 1138 insertions(+), 657 deletions(-)
 rename datafu-spark/src/test/resources/{example_tests => python_tests}/df_utils_tests.py
(100%)
 rename datafu-spark/src/test/resources/{example_tests => python_tests}/pyfromscala.py
(97%)
 rename datafu-spark/src/test/resources/{example_tests => python_tests}/pyfromscala_with_error.py
(100%)


Mime
View raw message