ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Denis Magda <dma...@apache.org>
Subject Re: Spark data frames integration merged
Date Fri, 29 Dec 2017 21:37:15 GMT
Great news,

Thanks Nikolay and Val!

Nikolay, could you document the feature before the release [1]? I’ve granted you required
permission.

More on the doc process can be found here [2].

[1] https://issues.apache.org/jira/browse/IGNITE-7345 <https://issues.apache.org/jira/browse/IGNITE-7345>
[2] https://cwiki.apache.org/confluence/display/IGNITE/How+to+Document <https://cwiki.apache.org/confluence/display/IGNITE/How+to+Document>

—
Denis

> On Dec 29, 2017, at 1:22 PM, Valentin Kulichenko <valentin.kulichenko@gmail.com>
wrote:
> 
> Igniters,
> 
> Great news! We completed and merged first part of integration with Spark data frames
[1]. It contains implementation of Spark data source which allows to use DataFrame API to
query Ignite data, as well as join it with other data frames originated from different sources.
> 
> Next planned steps are the following:
> - Implement custom execution strategy to avoid transferring data from Ignite to Spark
when possible [2]. This should give serious performance improvement in cases when only Ignite
tables participate in a query.
> - Implement ability to save a data frame into Ignite via DataFrameWrite API [3].
> 
> [1] https://issues.apache.org/jira/browse/IGNITE-3084 <https://issues.apache.org/jira/browse/IGNITE-3084>
> [2] https://issues.apache.org/jira/browse/IGNITE-7077 <https://issues.apache.org/jira/browse/IGNITE-7077>
> [3] https://issues.apache.org/jira/browse/IGNITE-7337 <https://issues.apache.org/jira/browse/IGNITE-7337>
> 
> Nikolay Izhikov, thanks for the contribution and for all the hard work!
> 
> -Val


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message