spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pradyumn Agrawal <>
Subject Re: Spark 3.0.1 not connecting with Hive 2.1.1
Date Sat, 09 Jan 2021 17:53:35 GMT
Hi Michael,
Thanks for references, although I had a hard time translating the 3rd one
as Google Translate
of the csdn blog, it didn't work correctly and already went through the 1st
and 2nd earlier.
But I can see the CDH distribution is different in my case, it is CDH-6.3.1.

[image: image.png]

As you can see here in the screenshot, it is saying that *Invalid Methods
Name: get_table_req*
I am guessing that CDH Distribution has some changes on Hive Metastore
Client which is conflicting with Shim implementations of Spark. Although, I
couldn't debug a lot, it's totally a guesswork.
Would certainly like to know your and other views on this?

Thanks and Regards
Pradyumn Agrawal (India)

On Sat, Jan 9, 2021 at 8:01 PM michael.yang <>

> Hi Pradyumn,
> We integrated Spark 3.0.1 with hive 2.1.1-cdh6.1.0 and it works fine to use
> spark-sql to query hive tables.
> Make sure you config spark-defaults.conf and well and copy
> hive/hadoop related config files to spark conf folder.
> You can refer to below refrences for detail.
> Thanks
> Michael Yang
> --
> Sent from:
> ---------------------------------------------------------------------
> To unsubscribe e-mail:

View raw message