spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Hamstra <m...@clearstorydata.com>
Subject Re: How to create RDD from Java in-memory data?
Date Tue, 11 Mar 2014 23:58:05 GMT
https://github.com/apache/incubator-spark/pull/421

Works pretty good, but really needs to be enhanced to work with
AsyncRDDActions.


On Tue, Mar 11, 2014 at 4:50 PM, wallacemann <wallace@bandpage.com> wrote:

> In a similar vein, it would be helpful to have an Iterable way to access
> the
> data inside an RDD.  The collect method takes everything in the RDD and
> puts
> in a list, but this blows up memory.  Since everything I want is already
> inside the RDD, it could be easy to iterate over the content without
> replicating the array.
>
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/How-to-create-RDD-from-Java-in-memory-data-tp2486p2568.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>

Mime
View raw message