spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Priyanka Gomatam <>
Subject RE: JDBC connector for DataSourceV2
Date Tue, 16 Jul 2019 00:16:09 GMT
I would have thought one of the most important goals would be pushing down limits since V2
supports it.

I am also interested in collaborating. Thanks!

Priyanka Gomatam

From: Shiv Prashant Sood <>
Sent: Monday, July 15, 2019 10:22 AM
To: Gabor Somogyi <>
Cc: Xianyin Xin <>; Ryan Blue <>;; Spark Dev List <>
Subject: Re: JDBC connector for DataSourceV2

Agree. Let's use SPARK-24907<>
as the JIRA for this work. Thanks for resolving SPARK-28380<>
as dupe of this.


On Mon, Jul 15, 2019 at 1:50 AM Gabor Somogyi <<>>
I've had a look at the jiras and seems like the intention is the same (correct me if I'm wrong).
I think one is enough and the rest can be closed with duplicate.
We should keep multiple jiras only when the intention is different.


On Mon, Jul 15, 2019 at 6:01 AM Xianyin Xin <<>>
There’s another pr<>
but which is based the old V2 APIs.

We’d better link the JIRAs, SPARK-24907<>,
and SPARK-28380<>
and finalize a plan.


From: Shiv Prashant Sood <<>>
Date: Sunday, July 14, 2019 at 2:59 AM
To: Gabor Somogyi <<>>
Cc: Xianyin Xin <<>>,
Ryan Blue <<>>, <<>>,
Spark Dev List <<>>
Subject: Re: JDBC connector for DataSourceV2

To me this looks like refactoring of DS1 JDBC to enable user provided connection factories.
In itself a good change, but IMO not DSV2 related.

I created a JIRA and added some goals. Please comments/add as relevant.<>

JIRA for DataSourceV2 API based JDBC connector.

Goals :

  *   Generic connector based on JDBC that supports all databases (min bar is support for
all V1 data bases).
  *   Reference implementation and Interface for any specialized JDBC connectors.


On Sat, Jul 13, 2019 at 2:17 AM Gabor Somogyi <<>>
Hi Guys,

Don't know what's the intention exactly here but there is such a PR:<>
If that's what we need maybe we can resurrect it. BTW, I'm also interested in...


On Sat, Jul 13, 2019 at 4:09 AM Shiv Prashant Sood <<>>
Thanks all. I can also contribute toward this effort.

Sent from my iPhone

On Jul 12, 2019, at 6:51 PM, Xianyin Xin <<>>
If there’s nobody working on that, I’d like to contribute.

Loop in @Gengliang Wang.


From: Ryan Blue <<>>
Reply-To: <<>>
Date: Saturday, July 13, 2019 at 6:54 AM
To: Shiv Prashant Sood <<>>
Cc: Spark Dev List <<>>
Subject: Re: JDBC connector for DataSourceV2

I'm not aware of a JDBC connector effort. It would be great to have someone build one!

On Fri, Jul 12, 2019 at 3:33 PM Shiv Prashant Sood <<>>
Can someone please help understand the current Status of DataSource V2 based JDBC connector?
I see connectors for various file formats in Master, but can't find a JDBC implementation
or related JIRA.

DatasourceV2 APIs to me look in good shape to attempt a JDBC connector for READ/WRITE path.
Thanks & Regards,

Ryan Blue
Software Engineer
View raw message