spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wenchen Fan <>
Subject [VOTE] [SPIP] SPARK-15689: Data Source API V2
Date Thu, 17 Aug 2017 13:02:57 GMT
Hi all,

Following the SPIP process, I'm putting this SPIP up for a vote.

The current data source API doesn't work well because of some limitations
like: no partitioning/bucketing support, no columnar read, hard to support
more operator push down, etc.

I'm proposing a Data Source API V2 to address these problems, please read
the full document at

Since this SPIP is mostly about APIs, I also created a prototype and put
java docs on these interfaces, so that it's easier to review these
interfaces and discuss:

The vote will be up for the next 72 hours. Please reply with your vote:

+1: Yeah, let's go forward and implement the SPIP.
+0: Don't really care.
-1: I don't think this is a good idea because of the following
technical reasons.


View raw message