metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Miklavcic <michael.miklav...@gmail.com>
Subject Re: Authorization for Configuration
Date Mon, 03 Dec 2018 21:15:15 GMT
> @Mike Yeah, I agree. The Jira for that doesn't exist yet, pretty much
pending this exact conversation winding down.

Yup, understood.

On Fri, Nov 30, 2018 at 7:36 AM Justin Leet <justinjleet@gmail.com> wrote:

> To start with, I'm thinking just the configuration, in particular anything
> that touches the ConfigurationsUtils.  I think for the first pass, we could
> leave reading from configs out of it and focus on who can write configs
> (since that's more disruptive to the system).
>
> To the best of my knowledge, this is:
>
>    - Configuration through the Management UI
>    - Configuration through the CLI (e.g. the push script + Stellar + any
>    other scripts that have to do it).
>
> In terms of how fine-grained, I would essentially expect this to be at the
> users+groups level and the individual sensor / topology level (e.g. parsers
> can be authorized individually by users/group, indexing is on the whole).
> Essentially, I don't propose trying to rearchitect things to try to have
> multiple indexing topologies for complete separation, etc.  Just to take
> the existing setup and be able to apply authorization on top of it.
>
> I would expect anything covered by the ZK configs to be done as part of
> this effort (possibly in stages).  As noted, I would expect this to be a
> feature branch rather than piecemeal replacement.
>
> @Mike Yeah, I agree. The Jira for that doesn't exist yet, pretty much
> pending this exact conversation winding down.
>
> On Tue, Nov 20, 2018 at 12:08 PM Michael Miklavcic <
> michael.miklavcic@gmail.com> wrote:
>
> > Justin, It probably makes sense to provide a list of these configuration
> > items as subtasks on the FB Jira so that we can crosscheck what entry
> > points have been implemented against the test scripts. Do you think this
> > will impact streaming enrichments or the profiler at all? That is to say,
> > as Ali asked, just how far are you looking to take the fine grained auth
> > scope for this?
> >
> > M
> >
> > On Mon, Nov 19, 2018 at 11:37 PM Ali Nazemian <alinazemian@gmail.com>
> > wrote:
> >
> > > Hi Justin,
> > >
> > > By configuration do you mean the sensor related configurations only?
> Are
> > > you limiting the scope of this activity to the management-UI or also
> > > Alert-UI as well? For example, defining different roles (pre-defined
> > > or customizable) and the fine-grained integration with Ranger?
> > >
> > > Cheers,
> > > Ali
> > >
> > > On Wed, Nov 14, 2018 at 1:25 AM Justin Leet <justinjleet@gmail.com>
> > wrote:
> > >
> > > > Hi all,
> > > >
> > > > Right now, our various configs can be modified by anyone with access
> to
> > > the
> > > > various scripts. I'd like to start a discussion around building out
> > some
> > > > authorization to be able to add some more fine grained controls
> around
> > > > this.
> > > >
> > > > Other projects have some variants on how to accomplish this.
> > Typically,
> > > > this follows a pattern of calling out to a interface/class that takes
> > in
> > > > the operation and context (user and other info) and returns
> true/false
> > if
> > > > something is authorized.
> > > >
> > > > In my mind, what we would need out of this is
> > > >
> > > >    - Ability to apply fine grained permissions
> > > >    - The various scripts and UI should flow this authorization
> > framework.
> > > >    I believe most (if not all) of our configuration flows through
> > > >    ConfigurationsUtils.  Anything that doesn't should either be
> hooked
> > in
> > > > or
> > > >    refactored to flow through the same codepaths.
> > > >    - Pluggability. We shouldn't force only one authorizer.
> > > >
> > > > In particular, I'm proposing we use Apache Ranger
> > > > <https://ranger.apache.org/> as a supported authorization framework,
> > > > implementing it alongside the authorization framework to validate
> what
> > we
> > > > build. In my mind, the main catch with Ranger is that, based on my
> > > > understanding, we won't be able to restrict users with direct access
> to
> > > > ZooKeeper via it's CLI (e.g. Ranger can't mirror it's ACLs down into
> > ZK's
> > > > ACLs).  I believe this is a reasonable restriction, especially as the
> > > > management UI gets improved to handle more of the configuration
> burden
> > > and
> > > > the number of users with access to ZK CLI begins to decrease.  Users
> > can
> > > > still add ZK ACLs separately to enforce that access there.
> > > >
> > > > For anyone not familiar with Ranger, essentially you build a plugin
> > that
> > > > hooks into the existing component's authorization framework (e.g. for
> > > > Storm, the plugin runs through the IAuthorizer
> > > > <
> > > >
> > >
> >
> https://storm.apache.org/releases/1.2.2/javadocs/org/apache/storm/security/auth/IAuthorizer.html
> > > > >
> > > > interface, for Yarn it runs through YarnAuthorizationProvider
> > > > <
> > > >
> > >
> >
> http://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-common/apidocs/org/apache/hadoop/yarn/security/YarnAuthorizationProvider.html
> > > > >).
> > > > Additionally, Ranger provides auditing capabilities for this
> > > authorization
> > > > and has plugins for a good portion of our stack already (so users can
> > > > already setup ACLs on HDFS, Storm, etc.). Checkout the Ranger Github
> > > > <https://github.com/apache/ranger> for a list of the plugins they
> have
> > > > built in.
> > > >
> > > > What this means for Metron is building out an authorization setup
> > similar
> > > > to Storm or Yarn or whatever we choose. We'll want this anyway, to
> > allow
> > > > our solution to be pluggable.  At that point, we build an
> > implementation
> > > of
> > > > the authorizer compatible with Ranger along with the plugin.
> > > >
> > > > I think this could probably be a fairly small feature branch, which
> I'm
> > > > suggesting primarily to do the Ranger implementation alongside the
> > > general
> > > > authorization work to validate what's being built.  I think the main
> > > > tasking would be something similar to:
> > > >
> > > >    - Build out pluggable authorization for our configs.
> > > >    - This includes testing (and possibly doing something similar to
> > > Storm,
> > > >    where they have a some testing IAuthorizers, e.g. NoopAuthorizer,
> > > >    DenyAuthorizer, etc.)
> > > >    - Ensure that all the code paths consistently flow through this
> > > >    Authorization.
> > > >    - Build a Ranger compatible version of this.
> > > >    - Define the Ranger plugin for this.
> > > >    - Make sure auditing is defined.
> > > >    - Integration testing (particularly with Kerberos. After all, if
> > they
> > > >    want to do authorization and auditing, they're almost certainly
> > using
> > > >    Kerberos).
> > > >
> > > > Is there anything missing that we'd need or want for this?  Are there
> > any
> > > > other concerns we'd want to make sure are taken care of?
> > > >
> > >
> > >
> > > --
> > > A.Nazemian
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message