qpid-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ken Giusti <kgiu...@redhat.com>
Subject Re: QMFv2 object update bug [WAS Re: Questions from a novice]
Date Fri, 03 May 2013 15:51:24 GMT
Bill, Fraser,

Thanks for the well thought out replies.

I want to step back a bit.  

Before I spend any more time on this problem, I want to know what the QPID project's plan
is for QMFv1/v2 support in the C++ broker going forward. 

If the consensus is that a 1.x C++ broker is going to support both QMFv1 and QMFv2, then simply
reverting all the changes I've made to qmf.console and reverting to the old behavior is probably
the least risky move.

On the other hand, if the consensus is to deprecate QMFv1 support prior to releasing a 1.0
C++ broker - or perhaps toss in the towel on QMFv2 - then the final fix for this needs to
take that plan into account.

So I'm going to call a discussion among the developers, proposing we vote on what QMF support
a 1.x C++ broker will provide.  

Once that's decided, we can determine the most appropriate fix.

As an aside - we've talked a lot about preserving behavior and functionality across 0.X releases
in this thread.  While nobody wants to gratuitously break stuff, this project has reserved
the right to remove and change functionality between 0.x releases in order to fix bugs and
provide new features.  Things may be different once we get to 1.x, but for 0.x, this has been
the case.


----- Original Message -----
> From: "Bill Freeman" <ke1g.nh@gmail.com>
> To: "Qpid Dev" <dev@qpid.apache.org>
> Sent: Monday, April 29, 2013 2:53:29 PM
> Subject: Fwd: QMFv2 object update bug [WAS Re: Questions from a novice]
> I messed up and only sent this tor Fraser originally, but intended it for
> all.
> Bill
> ---------- Forwarded message ----------
> From: Bill Freeman <ke1g.nh@gmail.com>
> Date: Mon, Apr 29, 2013 at 11:18 AM
> Subject: Re: QMFv2 object update bug [WAS Re: Questions from a novice]
> To: Fraser Adams <fraser.adams@blueyonder.co.uk>
> On Sat, Apr 27, 2013 at 5:45 AM, Fraser Adams <fraser.adams@blueyonder.co.uk
> > wrote:
> > On 26/04/13 17:18, Ken Giusti wrote:
> >
> >> Hey Bill,
> >>
> >>
> >>
> >> I first started to implement the additional objectUpdate callback as
> >> originally proposed.  Very easy to do.  But that additional api required
> >> other tools to be updated - and additional documentation changes, etc.
> >>
> > Hmm I remain a little baffled here. So as I understood it prior to any
> > fixing going on here there existed a bug that actually prevented QMF2
> > updates actually hitting callbacks on the asynchronous python API, that's
> > good IMHO because most clients of that API almost certainly were expecting
> > a single callback call for a given object.
> >
> > I'm kind of unclear about "other tools to be updated" 'cause I'd have
> > assumed given the previous QMF2 bug that no existing tools were actually
> > receiving v2 updates so wouldn't need "fixed". Sure they will need modified
> > to handle V2 if they want to consume both, but I'd argue that it might be
> > better to modify tools that need to handle both than to risk breaking
> > changes which will blat an indeterminate number of third party tools. It's
> > not ideal for sure, but a non-breaking API extension requiring an addition
> > to clients to use it feels better and TBH it's actually clearer IMHO
> > because there's some conscious choice being made.
> >
> >
> >
> >> After implementing this, I discovered a problem with the fix: tools that
> >> need to monitor both QMFv1 and QMFv2 agents starting reporting incorrect
> >> results (qpid-tool, to be specific).  These tools need to support both the
> >> old callbacks (for QMFv1 updates) and the new one (for QMFv2 updates), in
> >> order to see both QMFv1 and QMFv2 agents.
> >>
> > To be honest as I suggest above IMHO I actually think it's a good thing
> > for tools that wish to support both to be explicit. I think from the
> > various sections of this thread it's pretty clear that trying to crowbar
> > everything together is a *bad idea*. Sure QMF1 and QMF2 are essentially
> > representing equivalent information but there are sufficient differences
> > and subtleties to flash a great big red light. The reason it's good to have
> > a separate objectUpdate is because the QMF2 API *is* different at that
> > point because the protocol is sending what amounts to incompatible data.
> >
> > At that point there are two choices 1) an additional non-breaking API
> > callback or 2) a shim in the client side that makes things behave
> > identically.
> >
> > The latter may be a bad choice if both v1 and v2 updates arrive because
> > it'd need to apply a filter for the case of v1 plus v2 but it'd probably be
> > a good choice if it could select either v1 or v2 because then existing
> > clients wouldn't need to be modified at all.
> >
> >
> As long as I'm including the whole context, let me say that I think that
> Fraser is completely correct here.  To paraphrase from a perhaps different
> point of view: Existing tools should work as they did before any of these
> un-released fixes were made.  Having the previously undelivered v2 updates
> arrive at a callback that the existing tools didn't know about, and and,
> thus, for which they can't possibly be providing handlers, seems to fulfill
> this bill of particulars.
> One possible addition to make the old and new more similar would be to add
> a keyword argument to enable the first fix, meaning that tools which don't
> know bout that fix won't be consuming additional network bandwidth and
> compute cycles because of it.  (I'm not suggesting this, since I don't see
> the additional traffic and processing as onerous, but if there are enough
> queues in someone else's circumstance, it could be relevant.)  The
> corresponding change, a keyword to inhibit the reception of v1 updates, is
> at least as useful, but is riskier, or at least would require more thorough
> testing, since I'm not certain that the console whether  tool itself
> depends upon information that I only knows how to extract from a v1
> response.
> >
> >
> >
> >
> >> And that's when I realized that this fix was just a hack to work around
> >> the actual problem.  The real issue at hand is that a QMF agent must not
> >> transmit both QMFv1 and QMFv2 updates for the same object.
> >>
> >> That's really what's broken here, and the C++ broker is doing precisely
> >> that.
> >>
> > Whoa there :-) I think that the problem is that this is *exactly* what it
> > should be doing :-)
> >
> > OK what I mean is how does it know? Bear in mind that the updates causing
> > the fun here are *unsolicited* data pushes! These updates are being pushed
> > to a topic, so unless any client is subscribing to
> > qmf.default.topic/data.ind.# or whatever it isn't going to be getting v2
> > updates, similarly if it's not subscribing to the equivalent on
> > qpid.management/ it's not going to be getting v1 updates. OTOH if it's
> > subscribing to both that's exactly what it's getting.
> >
> > The key thing is there's a fairly large level of decoupling going on here,
> > so I don't believe that there's an especially easy way for the broker
> > ManagementAgent to work out who's asking for what and thus avoid
> > transmitting both v1 and v2.
> >
> >
> > Funnily enough in the v2 protocol and API there's actually no concept of
> > an *unsolicited* data push, so I'd argue that what the broker is currently
> > doing for the v2 updates is an "undocumented feature" :-), what V2 actually
> > talks about is the concept of "query subscriptions". With query
> > subscriptions the asynchronous pushes are actually *solicited* by the
> > client actually requesting some specified subset of the data, so the
> > Management Agent does actually have a mechanism to track what data any
> > given subscribers are actually interested in.
> >
> From the practical use case perspective, eliminating updates for queues
> which are not of interest, such as those involved in the implementation of
> QMF itself, sounds desirable (again, bandwidth and processing).  On the
> other hand, I would have had to enumerate the queues I care about.  Right
> now I'm using the cheap cop out that the names of all of "out" queues begin
> with a particular three character sequence.
> >
> > That's clearly a moot point, but it would help with this pickle.
> >
> >
> > I was musing over this problem and I believe that there is an option,
> > albeit a non-trivial one :-(
> >
> > So the *real* issue is that it's difficult to manage the carnage caused
> > when both v1 and v2 updates happen. You've previously suggested disabling
> > v1 by default, I've sort of got sympathy with that :-) However there are
> > two (I suspect significant) use cases that will be impacted by that choice:
> > 1) Any asynchronous console that relies on the existing v1 callbacks will
> > stop working - I suspect that in practice that'll mean every single darned
> > one of them except (at a push) qpid-tool. There may be a lot of third party
> > tools using this API so it may be an issue. At the very least there needs
> > to be some real good communication to the user list.
> > 2) If you disable v1 then any third party Agents will break too. Again
> > perhaps they need to change to v2 and this is a catalyst, but it just may
> > cause non-trivial business impact on users.
> >
> > Ultimately I think that it's very worth pushing out a survey on the user
> > list to gauge the potential impact.
> >
> I'd like to offer a caution about expecting this mailing list to inform
> users that a minor update (e.g.; 0.10 to 0.10.1 or even 0.11, as opposed
> to, for example, 0.10 to 1.0) will require using organizations to update
> custom tools.  I essentially had my code working (with v1) before ever
> subscribing here (to learn what promises there might be about string
> representations of object IDs uniqueness).  When this contract ends, I
> maybe back to doing Django, or *nix kernel internals, or even hardware
> development, and may not be able to justify the time to continue to read
> here.  I'm sure that my circumstance isn't that unusual.  So the client may
> no longer have someone following the list (let alone having invested the
> time to know what will impact them).
> > The non-trivial mitigation that I was musing over was whether there was a
> > possibility of constructing a v1->v2 bridge Agent? What I mean is that it
> > might be possible to get the broker ManagementAgent to only send v2 updates
> > if there's a mechanism for the broker to intercept v1 updates from third
> > party Agents and map those into v2 too, thus making every data push
> > consistently V2? I *think* this could work because I believe that v1
> > requires broker involvement in order to work (I think that was one of the
> > reasons for moving to v2).
> >
> > If it were possible to get to a clear point of only sending v2 updates
> > then it should be possible to have the asynchronous Console API shimmed in
> > such a way that it behaves *exactly* the same way with v2 updates as it did
> > with v1 updates so it's possible to mitigate the impact on third party
> > console applications (yay!).
> >
> > It has to be said It's be much better if users didn't have to rewrite
> > applications because of objectUpdate versus propUpdate/statUpdate but
> > clearly a shim would be required to make v2 updates behave correctly and
> > transparently for propUpdate/statUpdate callbacks.
> >
> > The result of the survey question "how many people have custom v1 Agents"
> > would help determine whether it's worth investing in a v1->v2 bridge Agent.
> >
> > A bridge ought to be possible, that's actually exactly what I wrote for
> > the Java Broker in the Qmf2ManagementAgent to get that to talk QMF2 (albeit
> > I was mapping from its internal model and not QMF1, but the principal is
> > the same - just don't ask me to do it in C++ though :-))
> >
> >
> >
> >
> >> Why is that a problem?  Because a console receiving these two updates
> >> cannot tell that they are for the same object.  To the console
> >> application,
> >> it appears as two separate objects.
> >>
> > As I mention above I don't believe that it's any easier for the broker to
> > do a "diff" given the unsolicited nature of these updates :-)
> >
> >  I'm proposing that we should solve this problem by disabling QMFv1
> >> updates coming from the C++ broker.  This should be the default for the
> >> next release (0.26).  It should still be possible to turn them on manually
> >> if desired, but the C++ broker should only transmit QMFv2 type updates
> >> going forward.
> >>
> > As I say above I've got a fair amount of sympathy with this, and
> > parochially if it means that by doing this you can make statUpdate and
> > propUpdate for v2 behave *exactly* the same as for v1 so clients don't need
> > to be rewritten for v2 then it'd be better for me 'cause I only ever use v2
> > aside from this.
> >
> > However as I said above the only reason that this Console API ever
> > supported both was to cater for the case where a user needed to handle both
> > v1 and v2 updates and that use case is actually third party v1 Agents.
> >
> > I think it's important to at least check for how much impact this is going
> > to cause users.
> >
> >
> >
> >
> >> We can minimize the impact to console applications by having the
> >> objectProps and objectStats callbacks be invoked when a QMFv2 object
> >> arrives instead of introducing a new callback. [Bill - I'll fix the
> >> console
> >> to not call objectProps when only stats are present].  I think this should
> >> eliminate the need to re-code console applications.
> >>
> > So I don't think it's about "minimising the impact" I think that the API
> > needs to behave *exactly* the same for this to be any use. Basically any
> > asynchronous client hit by only v2 updates should behave identically to how
> > they behaved prior to this update or it's still introducing a regression
> > into an indeterminate number of applications. I've no idea how hard it'd be
> > to achieve this nirvana though.
> >
> >
> >
> >> This will cause problems for folks that use an old python console against
> >> the next release of the broker.  These folks will have to upgrade the
> >> console as well.
> >>
> >> Does anyone have a better alternative?
> >>
> > Well I'm not necessarily saying that my suggestions above are *better* :-)
> > It's difficult for sure. I guess that my biggest concern is introducing
> > breaking changes against an unknown number of third party Consoles and
> > Agents.
> >
> > One other thing makes me nervous. I guess that there's the scenario where
> > users are working with a mixed economy of brokers at different versions or
> > who (like Bill) may be unable to alter the broker config to only push out a
> > particular version.
> >
> > All brokers from at least 0.8 can push out v2 so it *might* be safe to
> > have the console only subscribe to v2 updates (providing a switch to
> > specify v1). If you can make propUpdate and statUpdate behave consistently
> > then that only leaves the case where there are v1 Agents to support, which
> > is where the bridge Agent may come in.
> >
> > How much do you regret starting on this fix now Ken? :-D
> >
> >
> > It's worth pointing out that this is I suspect only the tip of the iceberg
> > for the sort of problems that might start to be encountered when migration
> > to AMQP 1.0 Management starts to gain traction. My strong hunch is that
> > bridge Agents/ManagementNodes are going to become pretty important friends
> > when it comes to trying to manage any transition. It's something that's
> > going to bite sooner or later and the C++, Java & Python communities are
> > going to have to be pretty joined up. I quite like QMF, but clearly some
> > mistakes were made along the way given the lack of support in the Java
> > community and frankly the fragmentation of APIs and lack of take up on the
> > "official" QMF2 API. We *really* need to try harder next time round and act
> > as an exemplar in the AMQP community.
> >
> > Frase
> >
> > Also, Ken previously mentioned, but Fraser didn't quote, that v1 and v2
> object identifiers are unrelated.  I would like to point out that it is
> possible to synthesize the string representation of a v2 objectId using the
> information in a v1 objectProps() call.  (After adding the first patch, and
> figuring out how to make things work again I was doing this to conjoin
> information from the v1 and v2 updates for a single queue.)  The
> correspondences needed probably fall into the undocumented and subject to
> change category, but I suspect inertial will keep them from changing before
> 1.0.
> Bill


To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org

View raw message