spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matei Zaharia <matei.zaha...@gmail.com>
Subject Re: Incrementally add/remove vertices in GraphX
Date Tue, 18 Mar 2014 22:30:59 GMT
I just meant that you call union() before creating the RDDs that you pass to new Graph(). If
you call it after it will produce other RDDs.

The Graph() constructor actually shuffles and “indexes” the data to make graph operations
efficient, so it’s not too easy to add elements after. You could access graph.vertices and
graph.edges to build new RDDs, and then call Graph() again to make a new graph. I’ve CCed
Joey and Ankur to see if they have further ideas on how to optimize this. It would be cool
to support more efficient union and subtracting of graphs once they’ve been partitioned
by GraphX.

Matei

On Mar 14, 2014, at 8:32 AM, alelulli <alessandro.lulli@gmail.com> wrote:

> Hi Matei,
> 
> Could you please clarify why i must call union before creating the graph?
> 
> What's the behavior if i call union / subtract after the creation? 
> Is the added /removed vertexes been processed?
> 
> For example if i'm implementing an iterative algorithm and at the 5th step i
> need to add some vertex / edge, can i call union / subtract on the
> VertexRDD, EdgeRDD and Triplets?
> 
> Thanks
> Alessandro
> 
> 
> 
> --
> View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Incrementally-add-remove-vertices-in-GraphX-tp2227p2695.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.


Mime
View raw message