spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mich Talebzadeh <mich.talebza...@gmail.com>
Subject Re: Hive on Spark engine
Date Sat, 26 Mar 2016 22:32:54 GMT
Thanks Jorn.

Just to be clear they get Hive working with Spark 1.6 out of the box
(binary download)? The usual work-around is to build your own package and
get the Hadoop-assembly jar file copied over to $HIVE_HOME/lib.


Cheers


Dr Mich Talebzadeh



LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*



http://talebzadehmich.wordpress.com



On 26 March 2016 at 22:08, Jörn Franke <jornfranke@gmail.com> wrote:

> If you check the newest Hortonworks distribution then you see that it
> generally works. Maybe you can borrow some of their packages. Alternatively
> it should be also available in other distributions.
>
> On 26 Mar 2016, at 22:47, Mich Talebzadeh <mich.talebzadeh@gmail.com>
> wrote:
>
> Hi,
>
> I am running Hive 2 and now Spark 1.6.1 but I still do not see any sign
> that Hive can utilise a Spark engine higher than 1.3.1
>
> My understanding was that there were miss-match on Hadoop assembly Jar
> files that cause Hive not being able to run on Spark using the binary
> downloads. I just tried Hive 2 on Spark 1.6 as the execution engine and it
> crashed.
>
> I do not know the development state of this cross-breed but will be very
> desirable if we could manage to sort out
> this spark-assembly-1.x.1-hadoop2.4.0.jar for once.
>
> Thanks
>
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>
>
>
> http://talebzadehmich.wordpress.com
>
>
>
>

Mime
View raw message