spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Hive on Spark engine
Date Sat, 26 Mar 2016 23:55:35 GMT
According to:
https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.4/bk_HDP_RelNotes/bk_HDP_RelNotes-20151221.pdf

Spark 1.5.2 comes out of box.

Suggest moving questions on HDP to Hortonworks forum.

Cheers

On Sat, Mar 26, 2016 at 3:32 PM, Mich Talebzadeh <mich.talebzadeh@gmail.com>
wrote:

> Thanks Jorn.
>
> Just to be clear they get Hive working with Spark 1.6 out of the box
> (binary download)? The usual work-around is to build your own package and
> get the Hadoop-assembly jar file copied over to $HIVE_HOME/lib.
>
>
> Cheers
>
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>
>
>
> http://talebzadehmich.wordpress.com
>
>
>
> On 26 March 2016 at 22:08, Jörn Franke <jornfranke@gmail.com> wrote:
>
>> If you check the newest Hortonworks distribution then you see that it
>> generally works. Maybe you can borrow some of their packages. Alternatively
>> it should be also available in other distributions.
>>
>> On 26 Mar 2016, at 22:47, Mich Talebzadeh <mich.talebzadeh@gmail.com>
>> wrote:
>>
>> Hi,
>>
>> I am running Hive 2 and now Spark 1.6.1 but I still do not see any sign
>> that Hive can utilise a Spark engine higher than 1.3.1
>>
>> My understanding was that there were miss-match on Hadoop assembly Jar
>> files that cause Hive not being able to run on Spark using the binary
>> downloads. I just tried Hive 2 on Spark 1.6 as the execution engine and it
>> crashed.
>>
>> I do not know the development state of this cross-breed but will be very
>> desirable if we could manage to sort out
>> this spark-assembly-1.x.1-hadoop2.4.0.jar for once.
>>
>> Thanks
>>
>>
>> Dr Mich Talebzadeh
>>
>>
>>
>> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>>
>>
>>
>> http://talebzadehmich.wordpress.com
>>
>>
>>
>>
>

Mime
View raw message