spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Abhishek Somani (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-15348) Hive ACID
Date Fri, 26 Jul 2019 12:48:00 GMT

    [ https://issues.apache.org/jira/browse/SPARK-15348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16893806#comment-16893806
] 

Abhishek Somani commented on SPARK-15348:
-----------------------------------------

We have worked on and open sourced a datasource that will enable users to work on their Hive
ACID Transactional tables using Spark. 
 
Github: [https://github.com/qubole/spark-acid]
 
It is available as a Spark package and instructions to use it are on the Github page. Currently
the datasource supports only reading from Hive ACID tables, and we are working on adding the
ability to write into these tables via Spark as well.
 
Feedback and suggestions are welcome!

> Hive ACID
> ---------
>
>                 Key: SPARK-15348
>                 URL: https://issues.apache.org/jira/browse/SPARK-15348
>             Project: Spark
>          Issue Type: New Feature
>          Components: SQL
>    Affects Versions: 1.6.3, 2.0.2, 2.1.2, 2.2.0, 2.3.0
>            Reporter: Ran Haim
>            Priority: Major
>
> Spark does not support any feature of hive's transnational tables,
> you cannot use spark to delete/update a table and it also has problems reading the aggregated
data when no compaction was done.
> Also it seems that compaction is not supported - alter table ... partition .... COMPACT
'major'



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message