spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nicolas Paris <>
Subject Re: SparkR integration with Hive 3 spark-r
Date Mon, 18 Nov 2019 18:53:22 GMT
Hi Alfredo

my 2 cents:
To my knowlegde and reading the spark3 pre-release note, it will handle
hive metastore 2.3.5 - no mention of hive 3 metastore. I made several
tests on this in the past[1] and it seems to handle any hive metastore

However spark cannot read hive managed table AKA transactional tables.
So I would say you should be able to read any hive 3 regular table with
any of spark, pyspark or sparkR.


On Mon, Nov 18, 2019 at 11:23:50AM -0600, Alfredo Marquez wrote:
> Hello,
> Our company is moving to Hive 3, and they are saying that there is no SparkR
> implementation in Spark 2.3.x + that will connect to Hive 3.  Is this true?
> If it is true, will this be addressed in the Spark 3 release?
> I don't use python, so losing SparkR to get work done on Hadoop is a huge loss.
> P.S. This is my first email to this community; if there is something I should
> do differently, please let me know.
> Thank you
> Alfredo


To unsubscribe e-mail:

View raw message