metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otto Fowler <ottobackwa...@gmail.com>
Subject Re: [DISCUSS] Dropping support for elastic 2.x
Date Wed, 04 Oct 2017 17:35:37 GMT
If we break out the indexing from the hdfswriting, we could just have two
different topologies to configure couldn’t we?



On October 4, 2017 at 13:20:19, Michael Miklavcic (
michael.miklavcic@gmail.com) wrote:

The question comes back to the DISCUSS I opened the other day about
upgrading ES. I believe we could theoretically maintain backwards
compatibility, but we'd have to keep the existing TransportClient. It's not
deprecated yet, but it will be. Keeping the ability to manage ES 2.x and
5.x+ via Ambari might be some work, but it looks like it's just calling
shell commands from python, e.g.
https://github.com/apache/metron/blob/master/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/ELASTICSEARCH/2.3.3/package/scripts/elastic_master.py#L43-L45


On Wed, Oct 4, 2017 at 11:08 AM, Casey Stella <cestella@gmail.com> wrote:

> Ok, so, whoever does this ES work, we should ensure the upgrade path is
at
> least spelled out in the Upgrade doc. This would also probably, IMO,
> necessitate a major version change in metron.
>
> On Wed, Oct 4, 2017 at 1:07 PM, Justin Leet <justinjleet@gmail.com>
wrote:
>
> > Forgot the link
> > https://www.elastic.co/guide/en/elasticsearch/reference/
> > current/setup-upgrade.html
> >
> > On Wed, Oct 4, 2017 at 1:07 PM, Simon Elliston Ball <
> > simon@simonellistonball.com> wrote:
> >
> > > The simplest option would probably be to upgrade the ES and then
> reindex
> > > from the HDFS store. Alternatively there are means to do inplace
> upgrades
> > > from 2.x to 5.x I believe.
> > >
> > > Simon
> > >
> > > > On 4 Oct 2017, at 18:05, Casey Stella <cestella@gmail.com> wrote:
> > > >
> > > > So, how would this work in an upgrade scenario that does not
involve
> > > losing
> > > > the existing indexed data?
> > > >
> > > > On Wed, Oct 4, 2017 at 12:55 PM, Michael Miklavcic <
> > > > michael.miklavcic@gmail.com> wrote:
> > > >
> > > >> The client I'm currently working on moving towards would *not* be
> > > backwards
> > > >> compatible.
> > > >> https://www.elastic.co/guide/en/elasticsearch/client/java-
> > > >> rest/current/java-rest-high-compatibility.html
> > > >>
> > > >> "
> > > >> The High Level Client is guaranteed to be able to communicate with
> any
> > > >> Elasticsearch node running on the same major version and greater
or
> > > equal
> > > >> minor version. It doesn’t need to be in the same minor version as
> the
> > > >> Elasticsearch nodes it communicates with, as it is forward
> compatible
> > > >> meaning that it supports communicating with later versions of
> > > Elasticsearch
> > > >> than the one it was developed for.
> > > >>
> > > >> The 5.6 client can communicate with any 5.6.x Elasticsearch node.
> > > Previous
> > > >> 5.x minor versions like 5.5.x, 5.4.x etc. are not (fully)
supported.
> > > >> "
> > > >>
> > > >> Best,
> > > >> Mike
> > > >>
> > > >>
> > > >> On Wed, Oct 4, 2017 at 10:45 AM, Simon Elliston Ball <
> > > >> simon@simonellistonball.com> wrote:
> > > >>
> > > >>> A number of people are currently working on upgrading the ES
> support
> > in
> > > >>> Metron to 5.x (including the clients, and the mpack managed
> install).
> > > >>>
> > > >>> Would anyone have any objections to dropping formal support for
2.x
> > as
> > > a
> > > >>> result of this work? In theory the clients should be backward
> > > compatible
> > > >>> against older data stores, so metron could be upgraded without
> > needing
> > > an
> > > >>> elastic upgrade.
> > > >>>
> > > >>> In practice, we would need to do pretty extensive testing and
I
> > > wouldn’t
> > > >>> want us to have to code around long term support on older clients
> if
> > > >> no-one
> > > >>> in the community cares enough about the older ES. Do we think
there
> > is
> > > a
> > > >>> case to be made for maintaining long term support for older
> clients?
> > > >>>
> > > >>> Simon
> > > >>
> > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message