metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Casey Stella <ceste...@gmail.com>
Subject Re: [DISCUSS] Kerberos Support
Date Sat, 25 Mar 2017 16:14:05 GMT
Actually, METRON-797 (https://github.com/apache/incubator-metron/pull/490)
is inspired by that work, Dave.
Specifically here are the differences:

   - The mpack work is not included
   - It's based on 793, so it uses storm-kafka-client rather than
   storm-kafka
   - It adds a flag when starting the parsers to pass the security protocol
   and sets up the writers and the spout automatically
   - The core extension points (flux properties) are set up like in that
   PR, to make the follow-on work easier


What's left is to pull the great work on the mpack out of there and get it
working as a follow-on PR, which I created METRON-799 (
https://issues.apache.org/jira/browse/METRON-799) to accomplish.

Casey


On Sat, Mar 25, 2017 at 10:11 AM, David Lyle <dlyle65535@gmail.com> wrote:

> Sounds good to me. A few of us did some initial exploration on that topic a
> week or so back. The branch is here:
> https://github.com/dlyle65535/incubator-metron/tree/kerb-testing?files=1
>
> It contains a prototype quality version of everything you've identified
> except METRON-793 and the probes.
>
> If it's a good starting place, it'd probably be short work to get it to a
> PR-able form.
>
> Thoughts?
>
> -D...
>
> On Fri, Mar 24, 2017 at 23:03 Casey Stella <cestella@gmail.com> wrote:
>
> > Hi All,
> >
> > I'd like to talk and start to formulate a plan around supporting running
> > Metron on a kerberized cluster.  This is a big bundle of work and seems
> > dauntingly nebulous, so I wanted to have a chat and get a firm direction.
> > When initially contemplating the issue, it's apparent that there are a
> few
> > snags that need to be thought through:
> >
> >    - The kafka spout (storm-kafka) from apache does not support
> interacting
> >    with kerberized kafka
> >    - We need to ensure the kafka writers are passing along the security
> >    protocol
> >    - We need to ensure that the credentials are auto-renewed
> >
> > Since I knew this was necessary, I wanted to get over this initial
> barrier
> > and submitted a couple of PRs associated with the following JIRAs that
> are
> > in review now:
> >
> >    - METRON-793: Migrate to storm-kafka-client, a kafka spout which
> >    supports kerberized kafka
> >    - METRON-797: Pass along security protocols and set up autorenewal
> >
> > Between these, we get what I believe is the enrichment, profiler and
> parser
> > topologies and interactions with hbase and hdfs from them working along
> > with the MR jobs.  We still lack a few things:
> >
> >    - A document describing the process of kerberizing a vagrant
> environment
> >    to enable testing
> >    - Kerberos support for the librdkafka-based sensors (bro and fastcapa)
> >    - Kerberos support for the sensors that use the console-producer
> (snort
> >    and yaf)
> >       - This should be as easy as ensuring to pass the right params to
> the
> >       console producer, but still, we need to adjust the sensor stubs
> > to make the
> >       right call.
> >    - Kerberos support for the mpack
> >    - Kerberos support for the REST API
> >    - Kerberos support for the pycapa
> >
> > I'm tracking these as JIRAs with the label of "kerberos" (see
> >
> > https://issues.apache.org/jira/browse/METRON-802?jql=
> labels%20%3D%20kerberos%20AND%20project%20%3D%20Metron
> > )
> >
> > Please chime in if I missed something; I'll add it to the list.
> >
> > Best,
> >
> > Casey
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message