metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Casey Stella <ceste...@gmail.com>
Subject [DISCUSS] Kerberos Support
Date Sat, 25 Mar 2017 04:03:33 GMT
Hi All,

I'd like to talk and start to formulate a plan around supporting running
Metron on a kerberized cluster.  This is a big bundle of work and seems
dauntingly nebulous, so I wanted to have a chat and get a firm direction.
When initially contemplating the issue, it's apparent that there are a few
snags that need to be thought through:

   - The kafka spout (storm-kafka) from apache does not support interacting
   with kerberized kafka
   - We need to ensure the kafka writers are passing along the security
   protocol
   - We need to ensure that the credentials are auto-renewed

Since I knew this was necessary, I wanted to get over this initial barrier
and submitted a couple of PRs associated with the following JIRAs that are
in review now:

   - METRON-793: Migrate to storm-kafka-client, a kafka spout which
   supports kerberized kafka
   - METRON-797: Pass along security protocols and set up autorenewal

Between these, we get what I believe is the enrichment, profiler and parser
topologies and interactions with hbase and hdfs from them working along
with the MR jobs.  We still lack a few things:

   - A document describing the process of kerberizing a vagrant environment
   to enable testing
   - Kerberos support for the librdkafka-based sensors (bro and fastcapa)
   - Kerberos support for the sensors that use the console-producer (snort
   and yaf)
      - This should be as easy as ensuring to pass the right params to the
      console producer, but still, we need to adjust the sensor stubs
to make the
      right call.
   - Kerberos support for the mpack
   - Kerberos support for the REST API
   - Kerberos support for the pycapa

I'm tracking these as JIRAs with the label of "kerberos" (see
https://issues.apache.org/jira/browse/METRON-802?jql=labels%20%3D%20kerberos%20AND%20project%20%3D%20Metron
)

Please chime in if I missed something; I'll add it to the list.

Best,

Casey

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message