spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Maciej Szymkiewicz (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-7683) Confusing behavior of fold function of RDD in pyspark
Date Sat, 02 Jan 2016 20:03:39 GMT

    [ https://issues.apache.org/jira/browse/SPARK-7683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15076650#comment-15076650
] 

Maciej Szymkiewicz commented on SPARK-7683:
-------------------------------------------

[~srowen] Do you have any example how it could break existing code? In Scala it is pretty
obvious but it looks like the current implementation isolates Python RDDs from the effects
of modifying mutable elements in place. 

> Confusing behavior of fold function of RDD in pyspark
> -----------------------------------------------------
>
>                 Key: SPARK-7683
>                 URL: https://issues.apache.org/jira/browse/SPARK-7683
>             Project: Spark
>          Issue Type: Sub-task
>          Components: PySpark
>    Affects Versions: 1.3.1
>            Reporter: Ai He
>            Priority: Minor
>              Labels: releasenotes
>
> This will make the “fold” function consistent with the "fold" in rdd.scala and other
"aggregate" functions where “acc” goes first. Otherwise, users have to write a lambda
function like “lambda x, y: op(y, x)” if they want to use “zeroValue” to change the
result type.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message