spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abhishek Somani <>
Subject New Spark Datasource for Hive ACID tables
Date Fri, 26 Jul 2019 12:37:55 GMT
Hi All,

We at Qubole <> have open sourced a datasource that
will enable users to work on their Hive ACID Transactional Tables
<> using


Hive ACID tables allow users to work on their data transactionally, and
also gives them the ability to Delete, Update and Merge data efficiently
without having to rewrite all of their data in a table, partition or file.
We believe that being able to work on these tables from Spark is a much
desired value add, as is also apparent in and with multiple people
looking for it. Currently the datasource supports reading from these ACID
tables only, and we are working on adding the ability to write into these
tables via Spark as well.

The datasource is also available as a spark package, and instructions on
how to use it are available on the Github page

We welcome your feedback and suggestions.

Abhishek Somani

View raw message