spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Evo Eftimov" <evo.efti...@isecc.com>
Subject RE: AMP Lab Indexed RDD - question for Data Bricks AMP Labs
Date Thu, 16 Apr 2015 21:34:25 GMT
Thanks but we need a firm statement and preferably from somebody from the spark vendor Data
Bricks including answer to the specific question posed by me and assessment/confirmation whether
this is a production ready / quality library which can be used for general purpose RDDs not
just inside the context of graphx 

 

From: Koert Kuipers [mailto:koert@tresata.com] 
Sent: Thursday, April 16, 2015 10:31 PM
To: Evo Eftimov
Cc: user@spark.apache.org
Subject: Re: AMP Lab Indexed RDD - question for Data Bricks AMP Labs

 

i believe it is a generalization of some classes inside graphx, where there was/is a need
to keep stuff indexed for random access within each rdd partition

 

On Thu, Apr 16, 2015 at 5:00 PM, Evo Eftimov <evo.eftimov@isecc.com> wrote:

Can somebody from Data Briks sched more light on this Indexed RDD library

https://github.com/amplab/spark-indexedrdd

It seems to come from AMP Labs and most of the Data Bricks guys are from
there

What is especially interesting is whether the Point Lookup (and the other
primitives) can work from within a function (e.g. map) running on executors
on worker nodes



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/AMP-Lab-Indexed-RDD-question-for-Data-Bricks-AMP-Labs-tp22532.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org

 


Mime
View raw message