metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Lyle <dlyle65...@gmail.com>
Subject Re: [DISCUSS] Ambari Metron Configuration Management consequences and call to action
Date Fri, 13 Jan 2017 15:46:32 GMT
To be clear- NOBODY is suggesting replacing Zookeeper with Ambari.

So, as a bit of a reset- here's what's being proposed:

 - Expand ambari to manage the remaining sensor-specific configs
 - Refactor the push calls to zookeeper (in ConfigurationUtils, I think)
   to push to ambari and take an Ambari user/pw and (optionally) reason
 - (Ambari can push to zookeeper, but it requires a service restart, so for
"live changes" you may
    want do both a rest call and zookeeper update from ConfigurationUtils)
    WAS
    Question remains about whether ambari can do the push to zookeeper
    or whetheror whether ConfigurationUtils has to push to zookeeper as
well as update
    ambari.
  - Refactor the middleware that Ryan submitted to have the API calls take
     an Ambari user/pw and (optionally) reason
  - Refactor the management UI to pass in an Ambari user/pw and
(optionally) reason
  - Refactor the Stellar Management functions CONFIG_PUT to accept an
Ambari user/pw and (optionally) reason

-D...



On Fri, Jan 13, 2017 at 10:42 AM, Ryan Merriman <merrimanr@gmail.com> wrote:

> The driver for using Zookeeper is that it is asynchronous and accepts
> callbacks.  Ambari would need to have that capability, otherwise we have to
> poll which is a deal breaker in my opinion.
>
> On Fri, Jan 13, 2017 at 9:28 AM, Casey Stella <cestella@gmail.com> wrote:
>
> > No, it was good to bring up, Nick.  I might have it wrong re: Ambari.
> >
> > Casey
> >
> > On Fri, Jan 13, 2017 at 10:27 AM, Nick Allen <nick@nickallen.org> wrote:
> >
> > > That makes sense.  I wasn't sure based on Matt's original
> > > suggestion/description of Ambari, whether that was something that
> Ambari
> > > had also designed for or not.
> > >
> > > On Fri, Jan 13, 2017 at 10:14 AM, Casey Stella <cestella@gmail.com>
> > wrote:
> > >
> > > > Polling the Ambari server via REST (or their API if they have one),
> > would
> > > > entail all workers hitting one server and create a single point of
> > > failure
> > > > (the ambari server is what serves up REST).  Zookeeper's intent is to
> > not
> > > > have a single point of failure like this and (one of its main)
> > use-cases
> > > is
> > > > to serve up configs in a distributed environment.
> > > >
> > > > Casey
> > > >
> > > > On Fri, Jan 13, 2017 at 9:55 AM, Nick Allen <nick@nickallen.org>
> > wrote:
> > > >
> > > > > Let me ask a stupid question.  What does Zookeeper do for us that
> > > Ambari
> > > > > cannot?  Why keep Zookeeper in the mix?
> > > > >
> > > > >
> > > > >
> > > > > On Fri, Jan 13, 2017 at 9:28 AM, David Lyle <dlyle65535@gmail.com>
> > > > wrote:
> > > > >
> > > > > > In the main yes- I've made some changes:
> > > > > >
> > > > > >  - Expand ambari to manage the remaining sensor-specific configs
> > > > > >  - Refactor the push calls to zookeeper (in ConfigurationUtils,
I
> > > > think)
> > > > > >    to push to ambari and take an Ambari user/pw and (optionally)
> > > reason
> > > > > >  - (Ambari can push to zookeeper, but it requires a service
> > restart,
> > > so
> > > > > for
> > > > > > "live changes" you may
> > > > > >     want do both a rest call and zookeeper update from
> > > > > ConfigurationUtils)
> > > > > >     WAS
> > > > > >     Question remains about whether ambari can do the push to
> > > zookeeper
> > > > > >     or whetheror whether ConfigurationUtils has to push to
> > zookeeper
> > > as
> > > > > > well as update
> > > > > >     ambari.
> > > > > >   - Refactor the middleware that Ryan submitted to have the
API
> > calls
> > > > > take
> > > > > >      an Ambari user/pw and (optionally) reason
> > > > > >   - Refactor the management UI to pass in an Ambari user/pw
and
> > > > > > (optionally) reason
> > > > > >   - Refactor the Stellar Management functions CONFIG_PUT to
> accept
> > an
> > > > > > Ambari user/pw and (optionally) reason
> > > > > >
> > > > > > I think we'd need to do some detailed design around how to handle
> > > what
> > > > we
> > > > > > expect to be dynamic configs, but the main principle should
(imo)
> > be
> > > to
> > > > > > always know who and why and make sure that Ambari is aware and
is
> > the
> > > > > > static backing store for Zookeeper.
> > > > > >
> > > > > > -D...
> > > > > >
> > > > > >
> > > > > > On Fri, Jan 13, 2017 at 9:19 AM, Casey Stella <
> cestella@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > > > So, basically, your proposed changes, broken into tangible
> > gobbets
> > > of
> > > > > > work:
> > > > > > >
> > > > > > >    - Expand ambari to manage the remaining sensor-specific
> > configs
> > > > > > >    - Refactor the push calls to zookeeper (in
> > ConfigurationUtils, I
> > > > > > think)
> > > > > > >    to push to ambari and take a reason
> > > > > > >       - Question remains about whether ambari can do the
push
> to
> > > > > > zookeeper
> > > > > > >       or whether ConfigurationUtils has to push to zookeeper
as
> > > well
> > > > as
> > > > > > > update
> > > > > > >       ambari.
> > > > > > >    - Refactor the middleware that Ryan submitted to have
the
> API
> > > > calls
> > > > > > take
> > > > > > >    a reason
> > > > > > >    - Refactor the management UI to pass in a reason
> > > > > > >    - Refactor the Stellar Management functions CONFIG_PUT
to
> > > accept a
> > > > > > > reason
> > > > > > >
> > > > > > > Just so we can evaluate it and I can ensure I haven't
> overlooked
> > > some
> > > > > > > important point.  Please tell me if Ambari cannot do the
things
> > > we're
> > > > > > > suggesting it can do.
> > > > > > >
> > > > > > > Casey
> > > > > > >
> > > > > > > On Fri, Jan 13, 2017 at 9:15 AM, David Lyle <
> > dlyle65535@gmail.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > > That's exactly correct, Casey. Basically, an expansion
of
> what
> > > > we're
> > > > > > > > currently doing with global.json, enrichment.properties
and
> > > > > > > > elasticsearch.properties.
> > > > > > > >
> > > > > > > > -D...
> > > > > > > >
> > > > > > > >
> > > > > > > > On Fri, Jan 13, 2017 at 9:12 AM, Casey Stella <
> > > cestella@gmail.com>
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > > I would suggest not having Ambari replace zookeeper.
 I
> think
> > > the
> > > > > > > > proposal
> > > > > > > > > is to have Ambari replace the editable store
(like the JSON
> > > files
> > > > > on
> > > > > > > > > disk).  Zookeeper woudl be the source of truth
for the
> > running
> > > > > > > topologies
> > > > > > > > > and ambari would be sync'd to it.
> > > > > > > > >
> > > > > > > > > Correct if I misspeak, dave or matt.
> > > > > > > > >
> > > > > > > > > Casey
> > > > > > > > >
> > > > > > > > > On Fri, Jan 13, 2017 at 9:09 AM, Nick Allen <
> > > nick@nickallen.org>
> > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Ambari seems like a logical choice.
> > > > > > > > > >
> > > > > > > > > > *>> It doesn’t natively integrate
Zookeeper storage of
> > > configs,
> > > > > but
> > > > > > > > there
> > > > > > > > > > is a natural place to specify copy to/from
Zookeeper for
> > the
> > > > > files
> > > > > > > > > > desired.*
> > > > > > > > > >
> > > > > > > > > > How would Ambari interact with Zookeeper
in this
> scenario?
> > > > Would
> > > > > > > > Ambari
> > > > > > > > > > replace Zookeeper completely? Or would Zookeeper
act as
> the
> > > > > > > persistence
> > > > > > > > > > tier under Ambari?
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Thu, Jan 12, 2017 at 9:24 PM, Matt Foley
<
> > > mattf@apache.org>
> > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Mike, could you try again on the image,
please, making
> > sure
> > > > it
> > > > > > is a
> > > > > > > > > > simple
> > > > > > > > > > > format (gif, png, or jpeg)?  It got
munched, at least
> in
> > my
> > > > > > viewer.
> > > > > > > > > > Thanks.
> > > > > > > > > > >
> > > > > > > > > > > Casey, responding to some of the questions
you raised:
> > > > > > > > > > >
> > > > > > > > > > > I’m going to make a rather strong
statement:  We
> already
> > > > have a
> > > > > > > > service
> > > > > > > > > > > “to intermediate and handle config
update/retrieval”.
> > > > > > > > > > > Furthermore, it:
> > > > > > > > > > > - Correctly handles the problems of
distributed
> services
> > > > > running
> > > > > > on
> > > > > > > > > > > multi-node clusters.  (That’s a HARD
problem, people,
> and
> > > we
> > > > > > > > shouldn’t
> > > > > > > > > > try
> > > > > > > > > > > to reinvent the wheel.)
> > > > > > > > > > > - Correctly handles Kerberos security.
(That’s kinda
> hard
> > > > too,
> > > > > or
> > > > > > > at
> > > > > > > > > > least
> > > > > > > > > > > a lot of work.)
> > > > > > > > > > > - It does automatic versioning of configurations,
and
> > > allows
> > > > > > > viewing,
> > > > > > > > > > > comparing, and reverting historical
configs
> > > > > > > > > > > - It has a capable REST API for all
those things.
> > > > > > > > > > > It doesn’t natively integrate Zookeeper
storage of
> > configs,
> > > > but
> > > > > > > there
> > > > > > > > > is
> > > > > > > > > > a
> > > > > > > > > > > natural place to specify copy to/from
Zookeeper for the
> > > files
> > > > > > > > desired.
> > > > > > > > > > >
> > > > > > > > > > > It is Ambari.  And we should commit
to it, rather than
> > try
> > > to
> > > > > > > > re-create
> > > > > > > > > > > such features.
> > > > > > > > > > > Because it has a good REST API, it
is perfectly
> feasible
> > to
> > > > > > > implement
> > > > > > > > > > > Stellar functions that call it.
> > > > > > > > > > > GUI configuration tools can also use
the Ambari APIs,
> or
> > > > better
> > > > > > yet
> > > > > > > > be
> > > > > > > > > > > integrated in an “Ambari View”.
(Eg, see the “Yarn
> > Capacity
> > > > > > > Scheduler
> > > > > > > > > > > Configuration Tool” example in the
Ambari
> documentation,
> > > > under
> > > > > > > “Using
> > > > > > > > > > > Ambari Views”.)
> > > > > > > > > > >
> > > > > > > > > > > Arguments are: Parsimony, Sufficiency,
Not reinventing
> > the
> > > > > wheel,
> > > > > > > and
> > > > > > > > > Not
> > > > > > > > > > > spending weeks and weeks of developer
time over the
> next
> > > year
> > > > > > > > > reinventing
> > > > > > > > > > > the wheel while getting details wrong
multiple times…
> > > > > > > > > > >
> > > > > > > > > > > Okay, off soapbox.
> > > > > > > > > > >
> > > > > > > > > > > Casey asked what the config update
behavior of Ambari
> is,
> > > and
> > > > > how
> > > > > > > it
> > > > > > > > > will
> > > > > > > > > > > interact with changes made from outside
Ambari.
> > > > > > > > > > > The following is from my experience
working with the
> > Ambari
> > > > > Mpack
> > > > > > > for
> > > > > > > > > > > Metron.  I am not otherwise an Ambari
expert, so
> tomorrow
> > > > I’ll
> > > > > > get
> > > > > > > it
> > > > > > > > > > > reviewed by an Ambari development engineer.
> > > > > > > > > > >
> > > > > > > > > > > Ambari-server runs on one node, and
Ambari-agent runs
> on
> > > each
> > > > > of
> > > > > > > all
> > > > > > > > > the
> > > > > > > > > > > nodes.
> > > > > > > > > > > Ambari-server has a private set of
py, xml, and
> template
> > > > files,
> > > > > > > which
> > > > > > > > > > > together are used both to generate
the Ambari
> > configuration
> > > > > GUI,
> > > > > > > with
> > > > > > > > > > > defaults, and to generate configuration
files (of any
> > > needed
> > > > > > > > filetype)
> > > > > > > > > > for
> > > > > > > > > > > the various Stack components.
> > > > > > > > > > > Ambari-server also has a database where
it stores the
> > > schema
> > > > > > > related
> > > > > > > > to
> > > > > > > > > > > these files, so even if you reach in
and edit Ambari’s
> > > files,
> > > > > it
> > > > > > > will
> > > > > > > > > > Error
> > > > > > > > > > > out if the set of parameters or parameter
names
> changes.
> > > The
> > > > > > > > > historical
> > > > > > > > > > > information about configuration changes
is also stored
> in
> > > the
> > > > > db.
> > > > > > > > > > > For each component (and in the case
of Metron, for each
> > > > > > topology),
> > > > > > > > > there
> > > > > > > > > > > is a python file which controls the
logic for these
> > > actions,
> > > > > > among
> > > > > > > > > > others:
> > > > > > > > > > > - Install
> > > > > > > > > > > - Start / stop / restart / status
> > > > > > > > > > > - Configure
> > > > > > > > > > >
> > > > > > > > > > > It is actually up to this python code
(which we wrote
> for
> > > the
> > > > > > > Metron
> > > > > > > > > > > Mpack) what happens in each of these
API calls.  But
> the
> > > > > current
> > > > > > > > code,
> > > > > > > > > > and
> > > > > > > > > > > I believe this is typical of Ambari-managed
components,
> > > > > performs
> > > > > > a
> > > > > > > > > > > “Configure” action whenever you
press the “Save” button
> > > after
> > > > > > > > changing
> > > > > > > > > a
> > > > > > > > > > > component config in Ambari, and also
on each Install
> and
> > > > Start
> > > > > or
> > > > > > > > > > Restart.
> > > > > > > > > > >
> > > > > > > > > > > The Configure action consists of approximately
the
> > > following
> > > > > > > sequence
> > > > > > > > > > (see
> > > > > > > > > > > disclaimer above :-)
> > > > > > > > > > > - Recreate the generated config files,
using the
> template
> > > > files
> > > > > > and
> > > > > > > > the
> > > > > > > > > > > actual configuration most recently
set in Ambari
> > > > > > > > > > > o Note this is also under the control
of python code
> that
> > > we
> > > > > > wrote,
> > > > > > > > and
> > > > > > > > > > > this is the appropriate place to push
to ZK if desired.
> > > > > > > > > > > - Propagate those config files to each
Ambari-agent,
> > with a
> > > > > > command
> > > > > > > > to
> > > > > > > > > > set
> > > > > > > > > > > them locally
> > > > > > > > > > > - The ambari-agents on each node receive
the files and
> > > write
> > > > > them
> > > > > > > to
> > > > > > > > > the
> > > > > > > > > > > specified locations on local storage
> > > > > > > > > > >
> > > > > > > > > > > Ambari-server then whines that the
updated services
> > should
> > > be
> > > > > > > > > restarted,
> > > > > > > > > > > but does not initiate that action itself
(unless of
> > course
> > > > the
> > > > > > > > > initiating
> > > > > > > > > > > action was a Start command from the
administrator).
> > > > > > > > > > >
> > > > > > > > > > > Make sense?  It’s all quite straightforward
in concept,
> > > > there’s
> > > > > > > just
> > > > > > > > an
> > > > > > > > > > > awful lot of stuff wrapped around that
to make it all
> go
> > > > > smoothly
> > > > > > > and
> > > > > > > > > > > handle the problems when it doesn’t.
> > > > > > > > > > >
> > > > > > > > > > > There’s additional complexity in
that the Ambari-agent
> > also
> > > > > > caches
> > > > > > > > (on
> > > > > > > > > > > each node) both the template files
and COMPILED forms
> of
> > > the
> > > > > > python
> > > > > > > > > files
> > > > > > > > > > > (.pyc) involved in transforming them.
 The pyc files
> > > > > incorporate
> > > > > > > some
> > > > > > > > > > > amount of additional info regarding
parameter values,
> but
> > > I’m
> > > > > not
> > > > > > > > sure
> > > > > > > > > of
> > > > > > > > > > > the form.  I don’t think that changes
the above in any
> > > > > practical
> > > > > > > way
> > > > > > > > > > unless
> > > > > > > > > > > you’re trying to cheat Ambari by
reaching in and
> editing
> > > its
> > > > > > files
> > > > > > > > > > > directly.  In that case, you also need
to whack the pyc
> > > files
> > > > > (on
> > > > > > > > each
> > > > > > > > > > > node) to force the data to be reloaded
from
> > Ambari-server.
> > > > > Best
> > > > > > > > > solution
> > > > > > > > > > > is don’t cheat.
> > > > > > > > > > >
> > > > > > > > > > > Also, there may be circumstances under
which the
> > > Ambari-agent
> > > > > > will
> > > > > > > > > detect
> > > > > > > > > > > changes and re-write the latest version
it knows of the
> > > > config
> > > > > > > files,
> > > > > > > > > > even
> > > > > > > > > > > without a Save or Start action at the
Ambari-server.
> I’m
> > > not
> > > > > > sure
> > > > > > > of
> > > > > > > > > > this
> > > > > > > > > > > and need to check with Ambari developers.
 It may no
> > longer
> > > > > > happen,
> > > > > > > > > altho
> > > > > > > > > > > I’m pretty sure change detection/reversion
was a
> feature
> > of
> > > > > early
> > > > > > > > > > versions
> > > > > > > > > > > of Ambari.
> > > > > > > > > > >
> > > > > > > > > > > Hope this helps,
> > > > > > > > > > > --Matt
> > > > > > > > > > >
> > > > > > > > > > > ================================================
> > > > > > > > > > > From: Michael Miklavcic <michael.miklavcic@gmail.com>
> > > > > > > > > > > Reply-To: "dev@metron.incubator.apache.org"
> > > > > > > > > > <dev@metron.incubator.apache.
> > > > > > > > > > > org>
> > > > > > > > > > > Date: Thursday, January 12, 2017 at
3:59 PM
> > > > > > > > > > > To: "dev@metron.incubator.apache.org"
> > > > > > > <dev@metron.incubator.apache.
> > > > > > > > org
> > > > > > > > > >
> > > > > > > > > > > Subject: Re: [DISCUSS] Ambari Metron
Configuration
> > > Management
> > > > > > > > > > consequences
> > > > > > > > > > > and call to action
> > > > > > > > > > >
> > > > > > > > > > > Hi Casey,
> > > > > > > > > > >
> > > > > > > > > > > Thanks for starting this thread. I
believe you are
> > correct
> > > in
> > > > > > your
> > > > > > > > > > > assessment of the 4 options for updating
configs in
> > Metron.
> > > > > When
> > > > > > > > using
> > > > > > > > > > more
> > > > > > > > > > > than one of these options we can get
into a split-brain
> > > > > > scenario. A
> > > > > > > > > basic
> > > > > > > > > > > example is updating the global config
on disk and using
> > the
> > > > > > > > > > > zk_load_configs.sh. Later, if a user
decides to restart
> > > > Ambari,
> > > > > > the
> > > > > > > > > > cached
> > > > > > > > > > > version stored by Ambari (it's in the
MySQL or other
> > > database
> > > > > > > backing
> > > > > > > > > > > Ambari) will be written out to disk
in the defined
> config
> > > > > > > directory,
> > > > > > > > > and
> > > > > > > > > > > subsequently loaded using the zk_load_configs.sh
under
> > the
> > > > > hood.
> > > > > > > Any
> > > > > > > > > > global
> > > > > > > > > > > configuration modified outside of Ambari
will be lost
> at
> > > this
> > > > > > > point.
> > > > > > > > > This
> > > > > > > > > > > is obviously undesirable, but I also
like the purpose
> and
> > > > > utility
> > > > > > > > > exposed
> > > > > > > > > > > by the multiple config management interfaces
we
> currently
> > > > have
> > > > > > > > > > available. I
> > > > > > > > > > > also agree that a service would be
best.
> > > > > > > > > > >
> > > > > > > > > > > For reference, here's my understanding
of the current
> > > > > > configuration
> > > > > > > > > > > loading mechanisms and their deps.
> > > > > > > > > > >
> > > > > > > > > > > <image>
> > > > > > > > > > >
> > > > > > > > > > > Mike
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > On Thu, Jan 12, 2017 at 3:08 PM, Casey
Stella <
> > > > > > cestella@gmail.com>
> > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > In the course of discussion on the
PR for METRON-652
> > > > > > > > > > > <https://github.com/apache/incubator-metron/pull/415>
> > > > > something
> > > > > > > > that I
> > > > > > > > > > > should definitely have understood better
came to light
> > and
> > > I
> > > > > > > thought
> > > > > > > > > that
> > > > > > > > > > > it was worth bringing to the attention
of the community
> > to
> > > > get
> > > > > > > > > > > clarification/discuss is just how we
manage configs.
> > > > > > > > > > >
> > > > > > > > > > > Currently (assuming the management
UI that Ryan
> Merriman
> > > > > > submitted)
> > > > > > > > > > configs
> > > > > > > > > > > are managed/adjusted via a couple of
different
> mechanism.
> > > > > > > > > > >
> > > > > > > > > > >    - zk_load_utils.sh: pushed and pulled
from disk to
> > > > zookeeper
> > > > > > > > > > >    - Stellar REPL: pushed and pulled
via the
> > > > > > CONFIG_GET/CONFIG_PUT
> > > > > > > > > > > functions
> > > > > > > > > > >    - Ambari: initialized via the zk_load_utils
script
> and
> > > > then
> > > > > > some
> > > > > > > > of
> > > > > > > > > > them
> > > > > > > > > > >    are managed directly (global config)
and some
> > indirectly
> > > > > > > > > > > (sensor-specific
> > > > > > > > > > >    configs).
> > > > > > > > > > >       - NOTE: Upon service restart,
it may or may not
> > > > overwrite
> > > > > > > > changes
> > > > > > > > > > on
> > > > > > > > > > >       disk or on zookeeper.  *Can someone
more
> > > knowledgeable
> > > > > than
> > > > > > > me
> > > > > > > > > > about
> > > > > > > > > > >       this describe precisely the semantics
that we can
> > > > expect
> > > > > on
> > > > > > > > > > > service restart
> > > > > > > > > > >       for Ambari? What gets overwritten
on disk and
> what
> > > gets
> > > > > > > updated
> > > > > > > > > > > in ambari?*
> > > > > > > > > > >    - The Management UI: manages some
of the configs.
> > *RYAN:
> > > > > Which
> > > > > > > > > configs
> > > > > > > > > > >    do we support here and which don't
we support here?*
> > > > > > > > > > >
> > > > > > > > > > > As you can see, we have a mishmash
of mechanisms to
> > update
> > > > and
> > > > > > > manage
> > > > > > > > > the
> > > > > > > > > > > configuration for Metron in zookeeper.
 In the
> beginning
> > > the
> > > > > > > approach
> > > > > > > > > was
> > > > > > > > > > > just to edit configs on disk and push/pull
them via
> > > > > > zk_load_utils.
> > > > > > > > > > Configs
> > > > > > > > > > > could be historically managed using
source control,
> etc.
> > > As
> > > > we
> > > > > > got
> > > > > > > > > more
> > > > > > > > > > > and more components managing the configs,
we haven't
> > taken
> > > > care
> > > > > > > that
> > > > > > > > > they
> > > > > > > > > > > they all work with each other in an
expected way (I
> > believe
> > > > > these
> > > > > > > are
> > > > > > > > > > > true..correct me if I'm wrong):
> > > > > > > > > > >
> > > > > > > > > > >    - If configs are modified in the
management UI or
> the
> > > > > Stellar
> > > > > > > REPL
> > > > > > > > > and
> > > > > > > > > > >    someone forgets to pull the configs
from zookeeper
> to
> > > > disk,
> > > > > > > before
> > > > > > > > > > they
> > > > > > > > > > > do
> > > > > > > > > > >    a push via zk_load_utils, they will
clobber the
> > configs
> > > in
> > > > > > > > zookeeper
> > > > > > > > > > > with
> > > > > > > > > > >    old configs.
> > > > > > > > > > >    - If the global config is changed
on disk and the
> > ambari
> > > > > > service
> > > > > > > > > > >    restarts, it'll get reset with the
original global
> > > config.
> > > > > > > > > > >    - *Ryan, in the management UI, if
someone changes
> the
> > > > > > zookeeper
> > > > > > > > > > configs
> > > > > > > > > > >    from outside, are those configs
reflected
> immediately
> > in
> > > > the
> > > > > > > UI?*
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > It seems to me that we have a couple
of options here:
> > > > > > > > > > >
> > > > > > > > > > >    - A service to intermediate and
handle config
> > > > > update/retrieval
> > > > > > > and
> > > > > > > > > > >    tracking historical changes so these
different
> > > mechanisms
> > > > > can
> > > > > > > use
> > > > > > > > a
> > > > > > > > > > > common
> > > > > > > > > > >    component for config management/tracking
and
> refactor
> > > the
> > > > > > > existing
> > > > > > > > > > >    mechanisms to use that service
> > > > > > > > > > >    - Standardize on exactly one component
to manage the
> > > > configs
> > > > > > and
> > > > > > > > > > regress
> > > > > > > > > > >    the others (that's a verb, right?
  nicer than
> > delete.)
> > > > > > > > > > >
> > > > > > > > > > > I happen to like the service approach,
myself, but I
> > wanted
> > > > to
> > > > > > put
> > > > > > > it
> > > > > > > > > up
> > > > > > > > > > > for discussion and hopefully someone
will volunteer to
> > > design
> > > > > > such
> > > > > > > a
> > > > > > > > > > thing.
> > > > > > > > > > >
> > > > > > > > > > > To frame the debate, I want us to keep
in mind a couple
> > of
> > > > > things
> > > > > > > > that
> > > > > > > > > > may
> > > > > > > > > > > or may not be relevant to the discussion:
> > > > > > > > > > >
> > > > > > > > > > >    - We will eventually be moving to
support kerberos
> so
> > > > there
> > > > > > > should
> > > > > > > > > at
> > > > > > > > > > >    least be a path to use kerberos
for any solution IMO
> > > > > > > > > > >    - There is value in each of the
different mechanisms
> > in
> > > > > place
> > > > > > > now.
> > > > > > > > > If
> > > > > > > > > > >    there weren't, then they wouldn't
have been created.
> > > > Before
> > > > > > we
> > > > > > > > try
> > > > > > > > > to
> > > > > > > > > > > make
> > > > > > > > > > >    this a "there can be only one" argument,
I'd like to
> > > hear
> > > > > very
> > > > > > > > good
> > > > > > > > > > >    arguments.
> > > > > > > > > > >
> > > > > > > > > > > Finally, I'd appreciate if some people
might answer the
> > > > > > questions I
> > > > > > > > > have
> > > > > > > > > > in
> > > > > > > > > > > bold there.  Hopefully this discussion,
if nothing else
> > > > > happens,
> > > > > > > will
> > > > > > > > > > > result in fodder for proper documentation
of the ins
> and
> > > outs
> > > > > of
> > > > > > > each
> > > > > > > > > of
> > > > > > > > > > > the components bulleted above.
> > > > > > > > > > >
> > > > > > > > > > > Best,
> > > > > > > > > > >
> > > > > > > > > > > Casey
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > > Nick Allen <nick@nickallen.org>
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Nick Allen <nick@nickallen.org>
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Nick Allen <nick@nickallen.org>
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message