qpid-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robbie Gemmell <robbie.gemm...@gmail.com>
Subject Re: Address validation Queues vs Topics
Date Wed, 29 Feb 2012 23:08:17 GMT
Ok, so actually taking a closer look at the code (oh my eyes :p), and
having been reminded of prior discussions around what a nice
addressing implementation might look like, I may have moved a lot
toward the idea of redoing things more comprehensively than I was
thinking earlier on. With the aim of getting this stuff working in a
far more maintainable way and allowing us to actually carry it forward
to e.g a 1.0 client, I can see that would indeed warrant possibly
breaking things a little in order to do it properly. Consider me
onboard with dumping 'destination.' and shaking up the JNDI file as


On 29 February 2012 20:25, Robbie Gemmell <robbie.gemmell@gmail.com> wrote:
> I understand that use of amq.topic does not imply something is a
> Topic, we actually have lots of users using Queues on amq.topic.
> However, the codebase has in many areas implied such a link and the
> JNDI lookup is once such area; if you inspect the current code for the
> JNDI stuff I think you you will be hard pressed not to see where I was
> going with that comment. We have a load of existing users using that
> code, and I think we should continue to support them where it makes
> sense and it is easy to do so. For example, I see little sense
> significantly changing the behaviour of using Binding URLs at this
> point, we should just support the behaviour it has already, as far as
> I know use of Binding URLs doesnt usually result in creation of
> Destination objects that are neither a Queue or a Topic, which is the
> major issue we are really interested in tackling here for Address
> strings whilst cleaning things up in general.
> I continue to fail to see how a user having to specify a node type in
> an Address string is really any different than a user specifying an
> (identical?) address string in a property that has a key prefixed with
> 'queue.' or 'topic.' instead of 'destination.' to select the type of
> Destination we construct. By asking them to select a one or the other
> via key prefix we are in fact still forcing them to specify what it is
> they want, which they could easily do in a 'destination.' entry with a
> string that specifies whether it is a queue or a topic, i.e there is
> really no significant difference. I am not saying we dont encourage
> use of the queue and topic prefixes for Address Strings, but I dont
> see the need to remove it and thus affect all the existing BURL users
> and anyone specifying an explicitly typed Address string for no good
> reason.
> I dont see that there is much *extra* scope for simplification than
> exists in general that would require forcing alteration of the
> existing behaviour of the JNDI file for pretty much every user who has
> ever used Qpid via a Binding URL or an Address string. The only people
> we really *need* to change their behaviour are those currently
> specifying an address string without node type into a 'destination.'
> entry. I think we can and should add the new behaviour of specifying
> Address strings via 'queue.' and 'topic.' prefixes, and fix it so
> people cant get anonymous Destinations when specifying Address strings
> via 'destination.' entries, all whilst maintaining the other
> behaviours for existing users. I dont particularly think we need to
> worry about BURL users being confused by whether they are using Queues
> or Topics given the way they are usually specified (i.e exhange and
> routing key, which is the same whether the thing on the other end is
> actually a Queue or a Topic) and constructed/used (again, check the
> JNDI related code), I think we simply leave the behaviour of those
> things untouched and rely on the fact it has been working the way it
> has for anything up to 6+ years or now and most people using them thus
> know how that works already.
> Robbie
> On 29 February 2012 19:22, Rajith Attapattu <rajith77@gmail.com> wrote:
>> On Wed, Feb 29, 2012 at 12:49 PM, Robbie Gemmell
>> <robbie.gemmell@gmail.com> wrote:
>>> Just to be clear, I have never been suggesting we remove
>>> 'destination.' entries from the equation. I think we should keep
>>> 'destination.' as we do have users already using it as its the only
>>> way to specify ADDRs in there,
>> The changes I'm proposing includes allowing both ADDR or BURL to be
>> specified for "queue" and "topics".
>> This will result in a concrete implementation of javax.jms.Queue or
>> javax.jms.Topic being returned.
>> As Rob points out, I think the pain in changing an existing
>> "destination" entry into a distinct "queue" or "topic" would force an
>> administrator to really think through the implications of the address
>> string entry.
>> We have lots of users who are confused with addressing and forcing
>> them to specify their address as a "queue" or a "topic" is going to
>> reduce a lot of pain for them down the line.
>> A huge plus is the simplicity we achieve in our code base by doing this.
>>>> and its also still an important entry
>>> point for people using BURLs without the arbitary assignment to
>>> amq.topic or amq.direct exchanges. I just think we should fix it.
>> I'm not exactly sure what you meant here.
>> But to clarify, amq.topic doesn't imply Topics any more than
>> amq.direct implies Queue.
>> We could do Topic style messaging using amq.topic, amq.direct,
>> amq.fanout or any custom exchange.
>> The existing implementation of AMQQueue and AMQTopic is very limited
>> in that it assume amq.topic for Topics and amq.direct for Queues.
>>> We should make people using it with ADDRs specify what type of node
>>> they desire, but theres no reason not to let any users who are already
>>> doing that or anyone using BURLs just continue to have their code keep
>>> working. Only the users using ADDRs and 'destination.' who currently
>>> arent setting a node type would need to adjust, which they would all
>>> have to anyway if we ever removed it in favour of the other two
>>> varieties.
>> As for backwards compatibility,
>> Well for ADDR we can throw an exception unless a type is specified.
>> How do you propose we tackle BURL ?
>> Choose Queue or Topic based on the type of exchange being used ?
>> What if they use a custom exchange? how would we determine the type ?
>> This amounts to the magic we are already doing with address string
>> albeit with not so stellar results with TCK failures etc....
>> Instead of us trying to solve those headaches in the code,  lets
>> simplify it by asking the administrator to make a decision.
>> Even if we allow "destination" for backwards compatibility, I'm
>> strongly in favour of deprecating "destination" and making it quite
>> clear in release notes and documentation.
>> Again the pain of having to deal with Queue vs Topic is best handled
>> at configuration time by an administrator than us trying to do the
>> magic in the code.
>> Regards,
>> Rajith
>>> Robbie
>>> On 29 February 2012 15:46, Rajith Attapattu <rajith77@gmail.com> wrote:
>>>> Robbie,
>>>> My preference is also to just use "queue" and "topic" qualifiers and
>>>> deprecate "destination" , hence listed it as option #1 in a previous
>>>> email.
>>>> I agree with you, Rob and Gordon that the above approach is simple,
>>>> easy and more importantly less buggy.
>>>> The reason I had "destination" in my summary is due to backwards compatibility.
>>>> However I agree with Rob that the pain of deprecating the
>>>> "destination"  qualifier is much less than having to deal with the
>>>> issues at the code level and the runtime issues a user might
>>>> encounter.
>>>> I will post the code soon for review.
>>>> Gentlemen, Thanks again for sharing your thoughts on this.
>>>> Regards,
>>>> Rajith
>>>> On Wed, Feb 29, 2012 at 7:46 AM, Robbie Gemmell
>>>> <robbie.gemmell@gmail.com> wrote:
>>>>> On 28 February 2012 17:35, Rajith Attapattu <rajith77@gmail.com>
>>>>>> Based on the discussion I would like to outline the following proposal.
>>>>>> I believe it reflects the consensus so far. Please correct me if
>>>>>> anything is amiss.
>>>>>> 1. If the user wants to use the specialized interfaces (JMS 1.0)
>>>>>> pass in either a Queue or a Topic, then they should be using "queue"
>>>>>> or "topic" in the jndi file.
>>>>>>   - This will result in a Destination impl being returned, that
>>>>>> implements either javax.jms.Queue or javax.jms.Topic depending on
>>>>>> qualifier used.
>>>>>>   - These destinations can obviously be used with the JMS 1.1 methods
>>>>>> which expects the generic javax.jms.Destination.
>>>>>>   - If a "type" is specified and is different from the qualifier
>>>>>> being used, we will raise an exception highlighting the discrepancy.
>>>>>>         Ex. topic.my-topic="my-queue; {type : queue}"
>>>>> This seems reasonable yes.
>>>>>> 2. If "destination.<jndi-name>=<address>" is used,
>>>>>>   - This will return a Destination impl that only implements the
>>>>>> javax.jms.Destination.
>>>>>>   -  ** If this destination is used with JMS 1.0 methods, it will
>>>>>> result in a class cast exception.**
>>>>>>   - We will not attempt to determine if the address used here is
>>>>>> JMS Topic or a JMS Queue.
>>>>>>   - If a "type" is specified with the address we will use that as
>>>>>> hint when trying to check the presence of the node in the broker.
>>>>>>     Ex if hello; {type=topic}, for 0-10 we will attempt to see
>>>>>> there is an exchange in the broker named "hello" and throw an
>>>>>> exception if no create instructions are given.
>>>>> This one I am less keen on. If a user specifies a type in their
>>>>> address string, then I think thats the type of object they should get,
>>>>> it shouldnt just be a hint used at some random point later when the
>>>>> Destination is used and we perform magic. If someone defined a
>>>>> 'destination.' entry saying that it is a queue, I would for example
>>>>> expect that to work when trying to create a QueueBrowser rather than
>>>>> throwing a ClassCastException. We had a user submit a patch just weeks
>>>>> ago to fix that exact use case.
>>>>> I think the scope for anyone getting a Destination object that didnt
>>>>> actually form a concreate Queue or Topic implementation should be
>>>>> absolutely minimal. I'd really rather prefer it didnt happen as I dont
>>>>> think it should ever be necessary for current use cases, and I'm a big
>>>>> fan of users actually having to ask for exactly what they want and
>>>>> then us trying to give them what they asked for. If what they asked
>>>>> for isnt compatible with whats found on the broker then I think thats
>>>>> when we throw an Exception telling them so, not when we start deciding
>>>>> things for them.
>>>>> Although JMS defines 'Destination', I really dont think it ever
>>>>> expects there to be direct implementations of it as it lists no
>>>>> methods, not even the mandated toString() 'representation of
>>>>> destination' as found on Queue and Topic. Destination was added to
>>>>> allow the new cross-domain Session objects in 1.1 to use of both
>>>>> existing Queue and Topic objects on a single session at the same time;
>>>>> there was no session.createDestination() method added along with it.
>>>>> The spec still deals with two domains, P2P and Pub/Sub, via the same
>>>>> old Queue and Topic types that now happen to also be Destinations.
>>>>> Do we know of any other providers who provide their users Destination
>>>>> objects which arent implementations of Queue or Topic? The idea had
>>>>> never entered my head previous to these Addressing discussions, so I
>>>>> am genuinely interested to know if there are.
>>>>> Robbie
>>>>>> Does this sound reasonable ? Please feel free to add/change anything
>>>>>> have missed.
>>>>>> Regards,
>>>>>> Rajith
>>>>>> On Tue, Feb 28, 2012 at 9:53 AM, Rajith Attapattu <rajith77@gmail.com>
>>>>>>> Rob,
>>>>>>> Addressing is indeed a pain point and most of it is due to grey
>>>>>>> causing undesirable side effects.
>>>>>>> I've got some work that I'm hoping to post today.
>>>>>>> Let me first check that into a branch and then I will post a
>>>>>>> outline of the design and the code in review board.
>>>>>>> I'm hoping to wrap this up in next 2 weeks.
>>>>>>> Regards,
>>>>>>> Rajith
>>>>>>> On Tue, Feb 28, 2012 at 4:01 AM, Rob Godfrey <rob.j.godfrey@gmail.com>
>>>>>>>> As an aside, I have been labeling the open Java Client JIRAs
so its
>>>>>>>> easier to pick out clusters of JIRAs that can be worked on
together /
>>>>>>>> see where the real pain points are.  A quick report on open
JIRAs per
>>>>>>>> label:
>>>>>>>> 17 - addressing
>>>>>>>>  9 - failover
>>>>>>>>  9 - exception-handling
>>>>>>>>  4 - deadlock
>>>>>>>>  3 - timestamp
>>>>>>>>  3 - possibly_complete
>>>>>>>>  3 - message-credit
>>>>>>>>  3 - jms-compliance
>>>>>>>>  2 - serialization
>>>>>>>>  2 - documentation
>>>>>>>>  1 - examples
>>>>>>>>  1 - browsing
>>>>>>>>  1 - amqp_compliance
>>>>>>>> Addressing covers anything to do with Destinations (ADDR
or BURL) but
>>>>>>>> is clearly the major pain point... Rajith - I know you were
working on
>>>>>>>> a patch for this... what is the status of this work?
>>>>>>>> Cheers,
>>>>>>>> Rob
>>>>>>>> On 28 February 2012 09:19, Rob Godfrey <rob.j.godfrey@gmail.com>
>>>>>>>>> On 28 February 2012 05:37, Rajith Attapattu <rajith77@gmail.com>
>>>>>>>>>> If the "queue" and "topic" qualifiers are used then
I guess it makes
>>>>>>>>>> it really easy for us to work out the validation.
>>>>>>>>>> What are we going to do with the "destination" qualifier
>>>>>>>>>> Ex destination.my-dest=<address-string>
>>>>>>>>>> 1. We deprecate this and get qpid users to use one
of "queue" or
>>>>>>>>>> "topic" as the administrator who writes the jndi
file surely knows
>>>>>>>>>> what it's going to be.
>>>>>>>>>> 2. We create the destination but not allow it to
be used with any
>>>>>>>>>> methods that require the Queue or Topic interface.
>>>>>>>>> ^^ This - it should be created as a Destination that
>>>>>>>>> neither Queue nor Topic.
>>>>>>>>>> 3. Attempt to figure out if the address is a Topic
or a Queue based on
>>>>>>>>>> the current behaviour (as described in my first email)
and then
>>>>>>>>>> convert it a Queue or Topic if the Destination object
is passed to any
>>>>>>>>>> methods that require a Queue/Topic interface.
>>>>>>>>> As per my previous comment on the JIRA, I think it's
not actually
>>>>>>>>> possible to determine from an address string what is
a "topic" and
>>>>>>>>> what is a "queue".  I can define a "queue" which distributes
>>>>>>>>> messages it hold to every consumer, and removes messages
only when
>>>>>>>>> every current consumer has irrevocably passed that message...
is this
>>>>>>>>> a "queue" or a "topic"?  From a JMS perspective it behaves
exactly as
>>>>>>>>> you would expect from a topic (especially in an AMQP
1-0 scenario
>>>>>>>>> where you can create durable subscribers with durable
links).  However
>>>>>>>>> from an AMQP 0-x perspective this looks like a "queue".
 (Indeed on my
>>>>>>>>> AMQP 1-0 branch I have implemented exactly this type
of "queue" in the
>>>>>>>>> Java broker... in a class called "Topic" :-) ). Conversely
I could
>>>>>>>>> define an exchange type which for any given message will
route to *at
>>>>>>>>> most one* bound queue... this "work sharing exchange"
has many of the
>>>>>>>>> properties of a "queue" in JMS semantics, but looks like
how we
>>>>>>>>> currently implement "topics" in AMQP 0-x.
>>>>>>>>> Given the above I think it is fruitless and indeed even
incorrect to
>>>>>>>>> attempt to determine whether a given address satisfies
JMS "Queue" or
>>>>>>>>> JMS "Topic" semantics based on the address itself.
>>>>>>>>> -- Rob
>>>>>>>>>> Regards,
>>>>>>>>>> Rajith
>>>>>>>>>> On Mon, Feb 27, 2012 at 11:30 PM, Rajith Attapattu
<rajith77@gmail.com> wrote:
>>>>>>>>>>> As per the discussion on QPID-792, I'm moving
the discussion onto the
>>>>>>>>>>> dev list under.
>>>>>>>>>>> I have attempted to summarise the current behaviour
and some of the
>>>>>>>>>>> comments expressed by Rob and Robbie.
>>>>>>>>>>> Currently the clients (C++, python and JMS) resolves
an address (with
>>>>>>>>>>> the 0-10 protocol) as follows.
>>>>>>>>>>> 1. If the name resolves to a queue, we treat
it as a Queue
>>>>>>>>>>> 2. If the name resolves to an exchange, we treat
is a Topic
>>>>>>>>>>> 3. If it doesn't resolve to either, we treat
it as a Queue.
>>>>>>>>>>> Rob made the following comments
>>>>>>>>>>> "I don't think that we should be trying to do
this because I'm pretty
>>>>>>>>>>> sure that it is impossible to determine what
is a Queue and what is a
>>>>>>>>>>> Topic.
>>>>>>>>>>> I think the closest we can come is to say that
an address that says
>>>>>>>>>>> you have to create a new temporary auto-delete
exclusive queue for
>>>>>>>>>>> every consumer should be treated as a topic...
but the converse is not
>>>>>>>>>>> true. As far as I am concerned the distinction
between Queue and Topic
>>>>>>>>>>> is something that only the "administrator" can
determine, and the code
>>>>>>>>>>> cannot determine dynamically."
>>>>>>>>>>> Robbie also expressed the following,
>>>>>>>>>>> "I also think that the (Java) client shouldnt
be making gueses as to
>>>>>>>>>>> whether something is a Queue or a Topic, as I'm
sure was fairly
>>>>>>>>>>> evident from previous mailings on the subject
last year. If that
>>>>>>>>>>> questionable behaviour is causing pain, then
we should at least
>>>>>>>>>>> consider simply not doing it. Destination is
itself only the parent
>>>>>>>>>>> interface of Queue and Topic, it doesnt actually
offer any methods
>>>>>>>>>>> (even the toString, though for backwards compatibility
>>>>>>>>>>> admitedly) and really only serves to allow creating
Topic and Queue
>>>>>>>>>>> consumers etc without having to have a specific
Session type. I
>>>>>>>>>>> realise forcing users to specify queue or topic
in the address string
>>>>>>>>>>> wouldnt be consistent with the other clients,
but I do think its worth
>>>>>>>>>>> noting that the Java client isn't entirely consistent
with the other
>>>>>>>>>>> clients for obvious reasons and trying to make
it more so isnt
>>>>>>>>>>> necessarily always going to be a helpful or useful
>>>>>>>>>>> Rob, further states that we could utilize the
queue and topic
>>>>>>>>>>> qualifiers that is currently present in our JNDI
>>>>>>>>>>> "I don't think the queue/topic distinction even
needs to go into the
>>>>>>>>>>> address - it should only needs to be defined
some way in the JNDI
>>>>>>>>>>> source
>>>>>>>>>>> e.g. in a properties file then things that begin
>>>>>>>>>>> queue.<NAME>=<address string>
>>>>>>>>>>> would be queues, and
>>>>>>>>>>> topic.<NAME>=<address string>
>>>>>>>>>>> would be topics"
>>>>>>>>>>> Regards,
>>>>>>>>>>> Rajith
>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>> Apache Qpid - AMQP Messaging Implementation
>>>>>>>>>> Project:      http://qpid.apache.org
>>>>>>>>>> Use/Interact: mailto:dev-subscribe@qpid.apache.org
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>> Apache Qpid - AMQP Messaging Implementation
>>>>>>>> Project:      http://qpid.apache.org
>>>>>>>> Use/Interact: mailto:dev-subscribe@qpid.apache.org
>>>>>> ---------------------------------------------------------------------
>>>>>> Apache Qpid - AMQP Messaging Implementation
>>>>>> Project:      http://qpid.apache.org
>>>>>> Use/Interact: mailto:dev-subscribe@qpid.apache.org
>>>>> ---------------------------------------------------------------------
>>>>> Apache Qpid - AMQP Messaging Implementation
>>>>> Project:      http://qpid.apache.org
>>>>> Use/Interact: mailto:dev-subscribe@qpid.apache.org
>>>> ---------------------------------------------------------------------
>>>> Apache Qpid - AMQP Messaging Implementation
>>>> Project:      http://qpid.apache.org
>>>> Use/Interact: mailto:dev-subscribe@qpid.apache.org
>>> ---------------------------------------------------------------------
>>> Apache Qpid - AMQP Messaging Implementation
>>> Project:      http://qpid.apache.org
>>> Use/Interact: mailto:dev-subscribe@qpid.apache.org
>> ---------------------------------------------------------------------
>> Apache Qpid - AMQP Messaging Implementation
>> Project:      http://qpid.apache.org
>> Use/Interact: mailto:dev-subscribe@qpid.apache.org

Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org

View raw message