spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Novak <>
Subject Re: Incrementally add/remove vertices in GraphX
Date Mon, 17 Mar 2014 23:38:42 GMT
I would assume that, regardless of the efficiency of such an operation, any
method of adding or removing vertices would need to result in a new graph,
since graphs in GraphX are supposed to be immutable.

It sounds like what you probably want is an efficient
union/subtract/whatever that operates on graphs, returning a graph modified
to include your new vertices or edges, or to remove the ones you wanted to
throw out. Because of the fancy indexing going on inside VertexRDD, I am
not sure how easy it would be to do this on vertices in an efficient way,
without having to rebuild the whole index.

Is the index used in a VertexRDD able to efficiently accommodate insertions?


On Mon, Mar 17, 2014 at 9:50 AM, Alessandro Lulli <> wrote:

> Hi All,
> Is somebody looking into this?
> I think this is correlated with the discussion "Are there any plans to
> develop Graphx Streaming?".
> Using union / subtract on VertexRDD or EdgeRDD leads on the creation of
> new RDD but NOT in the modification of the RDD in the graph.
> Is creating a new graph the only way to go to add /remove vertex or edge?
> Thanks
> Alessandro
> On Fri, Mar 14, 2014 at 4:32 PM, alelulli <>wrote:
>> Hi Matei,
>> Could you please clarify why i must call union before creating the graph?
>> What's the behavior if i call union / subtract after the creation?
>> Is the added /removed vertexes been processed?
>> For example if i'm implementing an iterative algorithm and at the 5th
>> step i
>> need to add some vertex / edge, can i call union / subtract on the
>> VertexRDD, EdgeRDD and Triplets?
>> Thanks
>> Alessandro
>> --
>> View this message in context:
>> Sent from the Apache Spark User List mailing list archive at

View raw message