spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matei Zaharia (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (SPARK-615) Add mapPartitionsWithIndex() to the Java API
Date Wed, 30 Apr 2014 00:41:14 GMT

     [ https://issues.apache.org/jira/browse/SPARK-615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Matei Zaharia resolved SPARK-615.
---------------------------------

       Resolution: Fixed
    Fix Version/s: 1.0.0

> Add mapPartitionsWithIndex() to the Java API
> --------------------------------------------
>
>                 Key: SPARK-615
>                 URL: https://issues.apache.org/jira/browse/SPARK-615
>             Project: Spark
>          Issue Type: New Feature
>          Components: Java API
>    Affects Versions: 0.6.0
>            Reporter: Josh Rosen
>            Assignee: Holden Karau
>            Priority: Minor
>              Labels: Starter
>             Fix For: 1.0.0
>
>
> We should add {{mapPartitionsWithIndex()}} to the Java API.
> What should the interface for this look like?  We could require the user to pass in a
{{FlatMapFunction[(Int, Iterator[T]))}}, but this requires them to unpack the tuple from Java.
 It would be nice if the UDF had a signature like {{f(int partition, Iterator[T] iterator)}},
but this will require defining a new set of {{Function}} classes.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message