spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gourav Sengupta <gourav.sengu...@gmail.com>
Subject Re: SPIP: DataFrame-based Property Graphs, Cypher Queries, and Algorithms
Date Sun, 20 Jan 2019 17:53:00 GMT
Hi,

this is fantastic and it will be great to have this. Also a place where we
could use graph frames is for data lineage. You will see a 100% adoption of
graph frames in case we can send data from catalyst to be stored somewhere
as graphs of dependencies.

In case you are including data lineage as well, please do let me know and I
will love to be a part of the testing as well.

Regards,
Gourav Sengupta

On Tue, Jan 15, 2019 at 4:53 PM Xiangrui Meng <mengxr@gmail.com> wrote:

> Hi all,
>
> I want to re-send the previous SPIP on introducing a DataFrame-based graph
> component to collect more feedback. It supports property graphs, Cypher
> graph queries, and graph algorithms built on top of the DataFrame API. If
> you are a GraphX user or your workload is essentially graph queries, please
> help review and check how it fits into your use cases. Your feedback would
> be greatly appreciated!
>
> # Links to SPIP and design sketch:
>
> * Jira issue for the SPIP:
> https://issues.apache.org/jira/browse/SPARK-25994
> * Google Doc:
> https://docs.google.com/document/d/1ljqVsAh2wxTZS8XqwDQgRT6i_mania3ffYSYpEgLx9k/edit?usp=sharing
> * Jira issue for a first design sketch:
> https://issues.apache.org/jira/browse/SPARK-26028
> * Google Doc:
> https://docs.google.com/document/d/1Wxzghj0PvpOVu7XD1iA8uonRYhexwn18utdcTxtkxlI/edit?usp=sharing
>
> # Sample code:
>
> ~~~
> val graph = ...
>
> // query
> val result = graph.cypher("""
>   MATCH (p:Person)-[r:STUDY_AT]->(u:University)
>   RETURN p.name, r.since, u.name
> """)
>
> // algorithms
> val ranks = graph.pageRank.run()
> ~~~
>
> Best,
> Xiangrui
>

Mime
View raw message