metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nick Allen <n...@nickallen.org>
Subject Re: [DISCUSS] Update Metron Elasticsearch index names to metron_
Date Wed, 24 Jan 2018 21:28:27 GMT
+1 to a standard prefix for all Metron indices.  I've had the same thought
myself and you laid out the advantages well.





On Wed, Jan 24, 2018 at 3:47 PM Zeolla@GMail.com <zeolla@gmail.com> wrote:

> I agree with having a metron_ prefix for ES indexes, and the timing.
>
> Jon
>
> On Wed, Jan 24, 2018 at 3:20 PM Michael Miklavcic <
> michael.miklavcic@gmail.com> wrote:
>
> > With the completion of https://github.com/apache/metron/pull/840
> > (METRON-939: Upgrade ElasticSearch and Kibana), we have the makings for a
> > major release rev of Metron in the upcoming release (currently slotted to
> > 0.4.3, I believe). Since there are non-backwards compatible changes
> > pertaining to ES indexing, it seems like a good opportunity to revisit
> our
> > index naming standards.
> >
> > I propose we add a simple prefix "metron_" to all Metron indexes. There
> are
> > numerous reasons for doing so
> >
> >    - removes the likelihood of index name collisions when we perform
> >    operations on index wildcard names, e.g. "enrichment_*, indexing_*,
> > etc.".
> >    - ie, this allows us to be more friendly in a multi-tenant ES
> >    environment for relatively low engineering cost.
> >    - simplifies the Kibana dashboard a bit. We currently needed to
> create a
> >    special index pattern in order to accommodate multi-index pattern
> > matching
> >    across all metron-specific indexes. Using metron_* would be much
> simpler
> >    and less prone to error.
> >    - easier for customers to debug and identify Metron-specific indexes
> and
> >    associated data
> >
> >
> > The reason for making these changes now is that we already have breaking
> > changes with ES. Leveraging existing indexed data rather than deleting
> > indexes and starting from scractch already requires a
> re-indexing/migration
> > step, so there is no additional effort on the part of users if they
> choose
> > to attempt a migration. It further makes sense with our current work
> > towards upgrading Solr.
> >
> > We already have a battery of integration and manual tests after the ES
> > upgrade work that can be leveraged to validate the changes.
> >
> > Mike Miklavcic
> >
>
>
> --
>
> Jon
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message