spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shiv Prashant Sood <shivprash...@gmail.com>
Subject Re: JDBC connector for DataSourceV2
Date Sat, 13 Jul 2019 18:59:40 GMT
To me this looks like refactoring of DS1 JDBC to enable user provided
connection factories. In itself a good change, but IMO not DSV2 related.

I created a JIRA and added some goals. Please comments/add as relevant.

https://issues.apache.org/jira/browse/SPARK-28380

JIRA for DataSourceV2 API based JDBC connector.

Goals :

   - Generic connector based on JDBC that supports all databases (min bar
   is support for all V1 data bases).
   - Reference implementation and Interface for any specialized JDBC
   connectors.


Regards,
Shiv

On Sat, Jul 13, 2019 at 2:17 AM Gabor Somogyi <gabor.g.somogyi@gmail.com>
wrote:

> Hi Guys,
>
> Don't know what's the intention exactly here but there is such a PR:
> https://github.com/apache/spark/pull/22560
> If that's what we need maybe we can resurrect it. BTW, I'm also interested
> in...
>
> BR,
> G
>
>
> On Sat, Jul 13, 2019 at 4:09 AM Shiv Prashant Sood <shivprashant@gmail.com>
> wrote:
>
>> Thanks all. I can also contribute toward this effort.
>>
>> Regards,
>> Shiv
>>
>> Sent from my iPhone
>>
>> On Jul 12, 2019, at 6:51 PM, Xianyin Xin <xianyin.xxy@alibaba-inc.com>
>> wrote:
>>
>> If there’s nobody working on that, I’d like to contribute.
>>
>>
>>
>> Loop in @Gengliang Wang.
>>
>>
>>
>> Xianyin
>>
>>
>>
>> *From: *Ryan Blue <rblue@netflix.com.INVALID>
>> *Reply-To: *<rblue@netflix.com>
>> *Date: *Saturday, July 13, 2019 at 6:54 AM
>> *To: *Shiv Prashant Sood <shivprashant@gmail.com>
>> *Cc: *Spark Dev List <dev@spark.apache.org>
>> *Subject: *Re: JDBC connector for DataSourceV2
>>
>>
>>
>> I'm not aware of a JDBC connector effort. It would be great to have
>> someone build one!
>>
>>
>>
>> On Fri, Jul 12, 2019 at 3:33 PM Shiv Prashant Sood <
>> shivprashant@gmail.com> wrote:
>>
>> Can someone please help understand the current Status of DataSource V2
>> based JDBC connector? I see connectors for various file formats in Master,
>> but can't find a JDBC implementation or related JIRA.
>>
>>
>>
>> DatasourceV2 APIs to me look in good shape to attempt a JDBC connector
>> for READ/WRITE path.
>>
>> Thanks & Regards,
>>
>> Shiv
>>
>>
>>
>>
>> --
>>
>> Ryan Blue
>>
>> Software Engineer
>>
>> Netflix
>>
>>

Mime
View raw message