ranger-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kent Yao (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (RANGER-2128) Implement SparkSQL plugin
Date Mon, 25 Jun 2018 08:26:00 GMT

    [ https://issues.apache.org/jira/browse/RANGER-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16522003#comment-16522003

Kent Yao commented on RANGER-2128:

[~bosco] I had a quick look on SPARK-18127 and its related PR. It has exposed Parser/Analyzer/Optimizer/Planner,
which is so great for all users. It also makes it easier for users to call our plug-in.

Some existing issues may be considered before we start 
1. spark-authorizer is designed as a Optimize Rule for Spark SQL and executed after all other
default rules because rules, such as column pruning, projection push down, and so on,  should
be operated first.

2. spark-authorizer has to visit hive SessionState object which is not accessible for spark
context classloader because Spark use a isolated classloader to load hive client jars.
   2.1 spark-authorizer itself will rewrite SessionState object the first time to do privileges
   2.2 kyuubi hacks spark and turn off that classloader.

3. spark-authorizer reuses the ranger hive plugin(0.5)which contains incompatible jersey
dependencies with spark ones. 

And what are the steps I should follow to contribute Ranger?


> Implement SparkSQL plugin
> -------------------------
>                 Key: RANGER-2128
>                 URL: https://issues.apache.org/jira/browse/RANGER-2128
>             Project: Ranger
>          Issue Type: New Feature
>          Components: plugins, Ranger
>    Affects Versions: 1.1.0
>            Reporter: t oo
>            Priority: Major
>             Fix For: 1.1.0
> Implement SparkSQL plugin

This message was sent by Atlassian JIRA

View raw message