spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zhan Zhang <zzh...@hortonworks.com>
Subject Re: Support for Hive 0.14 in secure mode on hadoop 2.6.0
Date Fri, 27 Mar 2015 23:20:47 GMT
Hi Doug,

Spark-5111 is to make spark work with security hadoop cluster in 2.6. There is some compatibility
issue which need the fix Spark-5111 patch.
In insecure cluster, current spark can connect to hive-0.14 without problems.

By the way, I am really glad to hear that "an adaption layer in Spark SQL in 1.4” and "allows
Spark SQL to connect to arbitrary Hive version”

Thanks.

Zhan Zhang

On Mar 27, 2015, at 12:57 PM, Doug Balog <doug.sparkdev@dugos.com> wrote:

> Is there a JIRA for this adaption layer ? It sounds like a better long term solution.
> 
> If anybody knows what is require to get the current Shim layer working with Hive 0.14,
please post what you know.
> I’m willing to spend some time on it, but I’m still learning how things fit together
and it might take me a while.
> I’ve been looking at the pr associated with SPARK-5111 for hints.
> 
> Thanks,
> 
> Doug
> 
> 
>> On Mar 27, 2015, at 6:13 AM, Cheng Lian <lian.cs.zju@gmail.com> wrote:
>> 
>> We're planning to replace the current Hive version profiles and shim layer with an
adaption layer in Spark SQL in 1.4. This adaption layer allows Spark SQL to connect to arbitrary
Hive version greater than or equal to 0.12.0 (or maybe 0.13.1, not decided yet).
>> 
>> However, it's not a promise yet, since this requires major refactoring of the current
Spark SQL Hive support.
>> 
>> Cheng
>> 
>> On 3/27/15 4:48 PM, Doug Balog wrote:
>>> Hi,
>>>  I'm just wondering if anybody is working on supporting Hive 0.14 in secure mode
on hadoop 2.6.0 ?
>>> I see once Jira referring to it  https://issues.apache.org/jira/browse/SPARK-5111
>>> but it mentions no effort to move to 0.14.
>>> 
>>> Thanks,
>>> 
>>> Doug
>>> 
>>> 
>>> 
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
>>> For additional commands, e-mail: dev-help@spark.apache.org
>>> 
>>> 
>> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
> For additional commands, e-mail: dev-help@spark.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org


Mime
View raw message