spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rafael Kyrdan <rafaelkyr...@gmail.com>
Subject How to migrate DataSourceV2 into Spark 3.0.0
Date Mon, 17 Aug 2020 16:21:21 GMT
Hey guys,



I’m trying to migrate my package where I’m using DataSourceV2 into Spark
3.0.0

Unfortunately, neither migration guide nor JIRA ticket under which this API
was refactored says nothing about how to do it.

https://issues.apache.org/jira/browse/SPARK-25390 I was suggested to send
my question in the mailing list.



Here the gist file with an example which I will help to understand the
issue https://gist.github.com/rafaelkyrdan/2bea8385aadd71be5bf67cddeec59581

In general a few notes about what changes happened with DataSourceV2 into
Spark 3.0.0 and what interfaces we should use now would be helpful.



Thank you
-- 
Rafael Kyrdan

Skype:  raphael.kyrdan
Mob: +380633040707

Mime
View raw message