spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From yxchen <yxc...@linkedin.com>
Subject Is there a plan for official spark-avro/spark-orc read/write library using Data Source V2
Date Wed, 29 Aug 2018 20:59:24 GMT
Data Source V2 API is available in Spark 2.3. But currently there's no
official library using data source v2 api to read/write avro or orc files --
spark-avro and spark-orc are both using Data Source V1. I wonder if there's
a plan in the upstream to implement those readers/writers and make them
official libraries? Or maybe a Spark-Hive connector to read Avro/Orc tables
using Data Source V2?

Although it's doable to implement such readers/writers on our own, but
depending on the official library would make it easier to maintain the
project.



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org


Mime
View raw message