Thanks but we need a firm statement and preferably from somebody from the spark vendor Data
Bricks including answer to the specific question posed by me and assessment/confirmation whether
this is a production ready / quality library which can be used for general purpose RDDs not
just inside the context of graphx
From: Koert Kuipers [mailto:koert@tresata.com]
Sent: Thursday, April 16, 2015 10:31 PM
To: Evo Eftimov
Cc: user@spark.apache.org
Subject: Re: AMP Lab Indexed RDD - question for Data Bricks AMP Labs
i believe it is a generalization of some classes inside graphx, where there was/is a need
to keep stuff indexed for random access within each rdd partition
On Thu, Apr 16, 2015 at 5:00 PM, Evo Eftimov <evo.eftimov@isecc.com> wrote:
Can somebody from Data Briks sched more light on this Indexed RDD library
https://github.com/amplab/spark-indexedrdd
It seems to come from AMP Labs and most of the Data Bricks guys are from
there
What is especially interesting is whether the Point Lookup (and the other
primitives) can work from within a function (e.g. map) running on executors
on worker nodes
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/AMP-Lab-Indexed-RDD-question-for-Data-Bricks-AMP-Labs-tp22532.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
|