ranger-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Madhan Neethiraj <mad...@apache.org>
Subject Re: Ranger/Hive Version mismatch in HDP 2.5 ?
Date Sat, 07 Jan 2017 20:29:14 GMT
Yan,

HDP-2.5.0 provides Hive-1.2.1 and a number of Apache patches – as detailed in HDP documentation.
The patch list includes HIVE-13418 and HIVE-13125. Please refer to HDP documentation for more
details. 

Hope this helps.

Madhan


On 1/6/17, 8:18 AM, "Yan Zhou" <yzhouopensource@gmail.com> wrote:

    My impressions are 1) HDP package needs to be modified too if it is now
    containing Hive 1.2 per its documentation; 2) as for the configuration, the
    real issue is the binary compatibility of different versions of the Hive
    libs. It might not be safe by just skipping a method call.
    
    
    Thanks,
    
    On Fri, Jan 6, 2017 at 12:19 AM, Madhan Neethiraj <madhan@apache.org> wrote:
    
    > HiveAuthzContext.getForwardedAddress() was added in HIVE-13418
    > <https://issues.apache.org/jira/browse/HIVE-13418>, which has been
    > committed only in Hive-2.1.0. Hence you see the issue with Hive-1.2.1.
    >
    >
    >
    > Also, support for column-masking and row-filtering was added in HIVE-13125
    > <https://issues.apache.org/jira/browse/HIVE-13125>, which is present only
    > in Hive-2.1.0. Since Ranger 0.6 supports policies for column-masking and
    > row-filtering, it would require Hive-2.1.0.
    >
    >
    >
    > Given above, I think the documentation should be updated to show that
    > Ranger 0.6 requires Hive-2.1.0 or higher.
    >
    >
    >
    > If support for Hive-1.2.1 is required, I would suggest to try the
    > following update to Ranger Hive plugin:
    >
    > -          Introduce a configuration in Ranger Hive plugin (like
    > ranger.plugin.hive.version-1.2.1); if this is set to true the plugin
    > should not call HiveAuthzContext.getForwardedAddress()
    >
    >
    >
    > Please note however that even with the above update, column-masking and
    > row-filter policies won’t be supported.
    >
    >
    >
    > Hope this helps.
    >
    >
    >
    > Madhan
    >
    >
    >
    >
    >
    > *From: *Yan Zhou <yzhouopensource@gmail.com>
    > *Reply-To: *"user@ranger.incubator.apache.org" <
    > user@ranger.incubator.apache.org>
    > *Date: *Thursday, January 5, 2017 at 10:50 PM
    > *To: *"user@ranger.incubator.apache.org" <user@ranger.incubator.apache.org>,
    > "dev-subscribe@ranger.incubator.apache.org" <dev-subscribe@ranger.
    > incubator.apache.org>
    >
    > *Subject: *Re: Ranger/Hive Version mismatch in HDP 2.5 ?
    >
    >
    >
    > When using Ranger 0.6.0 Hive Plugin with Hive 1.2.1, a beeline SQL command
    > throws the following exception:
    >
    >
    >
    > Exception in thread "HiveServer2-Handler-Pool: Thread-37"
    > java.lang.NoSuchMethodError: org.apache.hadoop.hive.ql.
    > security.authorization.plugin.HiveAuthzContext.
    > getForwardedAddresses()Ljava/util/List;
    >
    > at org.apache.ranger.authorization.hive.authorizer.
    > RangerHiveAccessRequest.<init>(RangerHiveAccessRequest.java:55)
    >
    > at org.apache.ranger.authorization.hive.authorizer.
    > RangerHiveAccessRequest.<init>(RangerHiveAccessRequest.java:72)
    >
    > at org.apache.ranger.authorization.hive.authorizer.RangerHiveAuthorizer.
    > checkPrivileges(RangerHiveAuthorizer.java:263)
    >
    > at org.apache.hadoop.hive.ql.Driver.doAuthorizationV2(Driver.java:779)
    >
    > at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:574)
    >
    > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:468)
    >
    > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:308)
    >
    > at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1122)
    >
    > at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1116)
    >
    > at org.apache.hive.service.cli.operation.SQLOperation.
    > prepare(SQLOperation.java:110)
    >
    > at org.apache.hive.service.cli.operation.SQLOperation.
    > runInternal(SQLOperation.java:181)
    >
    > at org.apache.hive.service.cli.operation.Operation.run(Operation.java:257)
    >
    > at org.apache.hive.service.cli.session.HiveSessionImpl.
    > executeStatementInternal(HiveSessionImpl.java:388)
    >
    > at org.apache.hive.service.cli.session.HiveSessionImpl.
    > executeStatementAsync(HiveSessionImpl.java:375)
    >
    > at org.apache.hive.service.cli.CLIService.executeStatementAsync(
    > CLIService.java:274)
    >
    > at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(
    > ThriftCLIService.java:486)
    >
    > at org.apache.hive.service.cli.thrift.TCLIService$Processor$
    > ExecuteStatement.getResult(TCLIService.java:1313)
    >
    > at org.apache.hive.service.cli.thrift.TCLIService$Processor$
    > ExecuteStatement.getResult(TCLIService.java:1298)
    >
    > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
    >
    > at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
    >
    > at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$
    > TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:692)
    >
    > at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(
    > TThreadPoolServer.java:285)
    >
    > at java.util.concurrent.ThreadPoolExecutor.runWorker(
    > ThreadPoolExecutor.java:1145)
    >
    > at java.util.concurrent.ThreadPoolExecutor$Worker.run(
    > ThreadPoolExecutor.java:615)
    >
    > at java.lang.Thread.run(Thread.java:745)
    >
    >
    >
    > The issue seems to be that HiveAuthzContext adds a
    > getForwardedAddresses(...) method in Hive 2.1,  which is called by Ranger
    > 0.6 Hive Plugin
    >
    > due to Ranger-962.
    >
    >
    >
    > I'm using Ranger 0.6 and Hive built from git.
    >
    >
    >
    > Thanks.
    >
    >
    >
    > On Thu, Jan 5, 2017 at 1:39 PM, Madhan Neethiraj <madhan@apache.org>
    > wrote:
    >
    > Yan,
    >
    > >> However Ranger 0.6.0 seems to work only with Hive 2.1.0
    > Can you please add details of the issues found while using Ranger 0.6 with
    > Hive 1.2?
    >
    > Thanks,
    > Madhan
    >
    >
    >
    >
    > On 1/5/17, 11:46 AM, "Don Bosco Durai" <bosco@apache.org> wrote:
    >
    >     Yan, you should reach out to someone within Hortonworks to answer your
    > HDP related question and their version numbering.
    >
    >
    >
    >     Thanks
    >
    >
    >
    >     Bosco
    >
    >
    >
    >
    >
    >     From: Yan Zhou <yzhouopensource@gmail.com>
    >     Reply-To: <user@ranger.incubator.apache.org>
    >     Date: Thursday, January 5, 2017 at 9:43 AM
    >     To: <user@ranger.incubator.apache.org>, <dev@ranger.incubator.apache.
    > org>
    >     Subject: Re: Ranger/Hive Version mismatch in HDP 2.5 ?
    >
    >
    >
    >     Can anyone help to clarify this version issue between Hive and Ranger
    > 0.6.0 in HDP 2.5? I'm now in a dilemma where Ranger 0.6.0 forces me to use
    > Hive 2.1 while HDP 2.5 only has Hive 1.2.1.
    >
    >
    >
    >     Many Thanks!
    >
    >
    >
    >     On Wed, Jan 4, 2017 at 1:08 PM, Yan Zhou <yzhouopensource@gmail.com>
    > wrote:
    >
    >     Ranger's pom.xml has the Hive version as 2.1.0.
    >
    >
    >
    >     Thanks
    >
    >
    >
    >     On Wed, Jan 4, 2017 at 12:15 PM, Don Bosco Durai <bosco@apache.org>
    > wrote:
    >
    >     I feel we should have a matrix for the component versions supported by
    > each Ranger release. Having it on the main page will be good.
    >
    >
    >
    >     https://cwiki.apache.org/confluence/display/RANGER/Index
    >
    >
    >
    >     One more thing I noticed is that the Installation and User guide is
    > still pointing to release 0.5. If there are no changes, then we should we
    > make a note of it some where.
    >
    >
    >
    >     Thanks
    >
    >
    >
    >     Bosco
    >
    >
    >
    >
    >
    >     From: Yan Zhou <yzhouopensource@gmail.com>
    >     Reply-To: <user@ranger.incubator.apache.org>
    >     Date: Wednesday, January 4, 2017 at 11:58 AM
    >     To: <user@ranger.incubator.apache.org>
    >     Subject: Ranger/Hive Version mismatch in HDP 2.5 ?
    >
    >
    >
    >     According to https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.0/
    > bk_release-notes/content/ch_relnotes_v250.html , HDP 2.5 contains Ranger
    > 0.6.0 and Hive 1.2.1. However Ranger 0.6.0 seems to work only with Hive
    > 2.1.0.
    >
    >
    >
    >     Please clarify what versions of Ranger and Hive will work well with
    > each other from HDP 2.5.
    >
    >
    >
    >     Thanks.
    >
    >
    >
    >
    >
    >
    >
    >
    >
    



Mime
View raw message