metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Casey Stella <ceste...@gmail.com>
Subject [DISCUSS] Ambari Integration
Date Thu, 15 Sep 2016 22:10:46 GMT
Hi Everyone,

I wanted to solicit some discussion around a feature that is fast
approaching.  A major pain point in using Metron is installation.  Thus far
our only approach to installation has been driven by the developer's needs
to construct a virtual environment to test out changes, which lead us to
either an ansible installation or a manual installation.

Because we want to make sure that the installation of Metron is as easy as
possible, we have had some great contributions of an additional approach,
installation via Apache Ambari directly.  Our ansible scripts currently
rely on Ambari blueprints to set up Hadoop on the cluster that it is
deploying on, so it is not a new dependency, but we're working toward a
full Ambari management pack
<https://cwiki.apache.org/confluence/display/AMBARI/Management+Packs>  that
will lay down the relevant topologies (parser, enrichment, indexing),
configs, bits and their infrastructural dependencies (ES and mysql) and
allow the topologies to be started and stopped as minimum viable product.

The beginnings of this have started with:

   - Ambari Service Definitions for the Parser topologies
   <https://github.com/apache/incubator-metron/pull/218>
   - Ambari Service Definition for the Indexing Topology
   <https://github.com/apache/incubator-metron/pull/222>
   - Ambari Service Definition for Elasticsearch
   <https://github.com/apache/incubator-metron/pull/223>

There will be more to come in the near-term to realize that vision, but we
wanted to get some reactions.  Past minimum viable product, what do you
guys think we should have and how should it look?

Currently we are treating the domain of the ambari installation as from
kafka to the indexes, which leaves the sensors unmanaged via ambari.  Is
that a good decision?

Are there other pain points that you have had around installation that
you'd like to see addressed?

The purpose of this discussion thread is to let you guys know that we will
soon have a new way to install metron, but also to understand what the
future requirements are so we, as a community, can address them.

Best,

Casey

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message