spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wenchen Fan <cloud0...@gmail.com>
Subject [VOTE] [SPIP] SPARK-15689: Data Source API V2
Date Thu, 17 Aug 2017 13:02:57 GMT
Hi all,

Following the SPIP process, I'm putting this SPIP up for a vote.

The current data source API doesn't work well because of some limitations
like: no partitioning/bucketing support, no columnar read, hard to support
more operator push down, etc.

I'm proposing a Data Source API V2 to address these problems, please read
the full document at
https://issues.apache.org/jira/secure/attachment/12882332/SPIP%20Data%20Source%20API%20V2.pdf

Since this SPIP is mostly about APIs, I also created a prototype and put
java docs on these interfaces, so that it's easier to review these
interfaces and discuss: https://github.com/cloud-fan/spark/pull/10/files

The vote will be up for the next 72 hours. Please reply with your vote:

+1: Yeah, let's go forward and implement the SPIP.
+0: Don't really care.
-1: I don't think this is a good idea because of the following
technical reasons.

Thanks!

Mime
View raw message