qpid-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mick <mgoul...@redhat.com>
Subject Re: QIP proposal: Easy Broker Info
Date Thu, 13 Jan 2011 16:12:16 GMT
On Wed, 2011-01-12 at 15:31 -0500, Justin Ross wrote:
> On Wed, 12 Jan 2011, mick wrote:
> 
> I like this proposal.  It combines low ambition 

well crap, there goes my Nobel Prize  :-(

> and high utility, a good 
> combination!
> 
> From a QIP standpoint, however, I think the utility part needs more 
> explanation.  It's not hard for me to imagine uses of this, but it ought 
> to be explicit.  Testing in particular could gain from a simple protocol 
> like this, so I think such an example would be great.

roger that.  see below in "Rationale"


> 
> > Index
> >
> > Easy Broker Info Service
> >
> > Status
> >
> > Draft
> >
> >
> >
> > Summary
> >
> >        Brokers write simple name-value pairs, one to a line, into PID
> >        files in /var/run/qpid. The names are tree-structured. Files are
> >        updated periodically, and taken down when broker shuts down
> >        normally. This allows easy discovery of broker info by any
> >        script or program.
> 
> I might consider dropping the tree-structured names part.  Based on my 
> experience with java properties files, I find people tend to be confused 
> as to whether ".", for instance, is a word separator or a parent-child 
> operator.  The result is a big mess.
> 
> I see two options: decide up front that data with an implicit tree 
> structure is not necessary and remove the language about trees, or switch 
> to a format that truly supports nested data (json, yaml, etc.).


I replied to this point after Andrew's reply in a future email.

> 
> > Problem
> >
> >        I wrote a script in which I needed to know the port that my
> >        broker was listening on for both TCP, and for SSL. There was no
> >        way to get this info. Right now, we use a mechanism where the
> >        broker writes the port number to stdout that it has chosen
> >        ( when you use --port 0 ). You can tell it to report on a
> >        different transport's port (i.e. SSL ) by using the
> >        "--transport" flag. But what if you want to know several
> >        different pieces of info about the broker? What if you are not
> >        the script that started it, but are just some other program or
> >        script that is coming late to the party? There should be one
> >        central, easy way to discover all running brokers, discover
> >        which of them are clustered, etc. What ports they are listening
> >        on. Maybe even info that gets updated periodically like health
> >        information, throughput, etc.
> >
> >
> > Solution
> >
> >        Brokers write information about themselves to files in a
> >        well-known directory ( i.e. /var/run/qpid ). This allows any
> 
> On linux I think it would be /var/lib/qpid.  What else can you say about 
> how these files are namespaced?

Ah, I see.   
"/var/lib : Variable state information"
OK -- that's me!  ( That name with those semantics has never made the
least bit of sense to me. )

I was just imagining a single file for each broker at /var/lib/PID

That file contains all the current status info for the broker, one attr
value pair per line.


> 
> >        running program or script to easily discover what brokers are
> >        running, what ports they are listening to for which transports,
> >        and any other information that the brokers want to share. This
> >        is strictly broker-based, and works whether or not management is
> >        enabled. Brokers only write the info, and non-brokers only read
> >        it. The info is in a simple, easily grepped name-value format,
> >        whose names are tree-structured. For example:
> >        "transports_tcp_port 6666". There is a single name-value pair
> >        per line.
> >
> >
> > Rationale
> >
> >        I think this is the simplest possible solution for this problem.
> >        I want something that is strictly broker-based, i.e. not
> >        dependent on management, language-neutral, OS-neutral, easily
> >        greppable. At first I thought of /proc, but that's too hinky.



Here is an example of how this facility might be used in a script,
against brokers that are already running -- not started by this script.



for broker_path in  $QPID_INFO_ROOT/*
do
  broker_pid=`basename $broker_path`
  ssl_port=`cat ${broker_path} | grep transport_ssl | awk '{print $2}'`
  if [ $ssl_port ]
  then
    echo "I will connect to ssl port: $ssl_port on broker $broker_pid "
    break
  fi
done



> >
> >
> > Implementation Notes
> >
> >        I think this is dead easy, in its first impl. make a class
> >        called something like ... infoPublisher ... or something. Make a
> >        public instance of it in the Broker. Anything in the code that
> >        can reach the Broker can call broker->infoPublisher ( name,
> >        value ); And that pair will get written out to the appropriate
> >        file. You can make it either replace any previous instance of
> >        that attr-value pair in the file, or just append this latest
> >        line to the end of the file. This code would be in
> >        cpp/src/qpid/broker . /em>
> 
> I find the alternative append behavior surprising.  I tend to think it 
> should be simpler: there's a map in memory, and sometimes it gets written 
> to disk.
> 
> Assuming that plan, what triggers a write?  Can a record get written 
> mid-update, or is there some kind of locking that guarantees consistency?

i think i will indeed want an image in mem -- that what will be
maintained by the member strings of BrokerInfo of whatever the class is
called.  And then I will have to lock while overwriting the file.   This
will only happen when something changes in the BrokerInfo class.   

Maybe for some data, BrokerInfo could have timers that go off every 10
seconds or so when it goes and updates itself.  Like for
number-of-connections, or whatever.


> 
> >
> >
> > Consequences
> >
> >        There are consequences in the following areas:
> >
> >              * Documentation: We would need a list of all the
> >                attributes names, and the formats of their values (int,
> >                string)
> >
> > Contributor-in-Charge
> >
> >        Mick Goulish
> >
> >        mgoulish@redhat.com
> >
> >
> > Contributors
> >
> >        Mick Goulish mgoulish@redhat.com
> >
> >
> > Version
> >
> > 1.0
> >
> >
> 
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project:      http://qpid.apache.org
> Use/Interact: mailto:dev-subscribe@qpid.apache.org
> 



---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org


Mime
View raw message