spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ankur Dave <>
Subject Re: GraphX: New graph operator
Date Tue, 02 Jun 2015 06:55:00 GMT
I think it would be good to have more basic operators like union or
difference, as long as they have an efficient distributed implementation
and are plausibly useful.

If they can be written in terms of the existing GraphX API, it would be
best to put them into GraphOps to keep the core GraphX implementation
small. The `mask` operation should actually be in GraphOps -- it's only in
GraphImpl for historical reasons. On the other hand, `subgraph` needs to be
in GraphImpl for performance: it accesses EdgeRDDImpl#filter(epred, vpred),
which can't be a public EdgeRDD method because its semantics rely on an
implementation detail (vertex replication).

Ankur <>

On Mon, Jun 1, 2015 at 8:54 AM, Tarek Auel <> wrote:

> Hello,
> Someone proposed in a Jira issue to implement new graph operations. Sean
> Owen recommended to check first with the mailing list, if this is
> interesting or not.
> So I would like to know, if it is interesting for GraphX to implement the
> operators like:
> and/or
> If yes, should they be integrated into GraphImpl (like mask, subgraph
> etc.) or as external library? My feeling is that they are similar to mask.
> Because of consistency they should be part of the graph implementation
> itself.
> What do you guys think? I really would like to bring GraphX forward and
> help to implement some of these.
> Looking forward to hear your opinions
> Tarek

View raw message