qpid-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rajith Attapattu <rajit...@gmail.com>
Subject Re: Address validation Queues vs Topics
Date Wed, 29 Feb 2012 23:13:46 GMT
On Wed, Feb 29, 2012 at 6:08 PM, Robbie Gemmell
<robbie.gemmell@gmail.com> wrote:
> Ok, so actually taking a closer look at the code (oh my eyes :p), and
> having been reminded of prior discussions around what a nice
> addressing implementation might look like, I may have moved a lot
> toward the idea of redoing things more comprehensively than I was
> thinking earlier on. With the aim of getting this stuff working in a
> far more maintainable way and allowing us to actually carry it forward
> to e.g a 1.0 client, I can see that would indeed warrant possibly
> breaking things a little in order to do it properly. Consider me
> onboard with dumping 'destination.' and shaking up the JNDI file as
> necessary.

Thanks you sir !
Let me post the code into a branch asap.

Rajith

> Robbie
>
> On 29 February 2012 20:25, Robbie Gemmell <robbie.gemmell@gmail.com> wrote:
>> I understand that use of amq.topic does not imply something is a
>> Topic, we actually have lots of users using Queues on amq.topic.
>> However, the codebase has in many areas implied such a link and the
>> JNDI lookup is once such area; if you inspect the current code for the
>> JNDI stuff I think you you will be hard pressed not to see where I was
>> going with that comment. We have a load of existing users using that
>> code, and I think we should continue to support them where it makes
>> sense and it is easy to do so. For example, I see little sense
>> significantly changing the behaviour of using Binding URLs at this
>> point, we should just support the behaviour it has already, as far as
>> I know use of Binding URLs doesnt usually result in creation of
>> Destination objects that are neither a Queue or a Topic, which is the
>> major issue we are really interested in tackling here for Address
>> strings whilst cleaning things up in general.
>>
>> I continue to fail to see how a user having to specify a node type in
>> an Address string is really any different than a user specifying an
>> (identical?) address string in a property that has a key prefixed with
>> 'queue.' or 'topic.' instead of 'destination.' to select the type of
>> Destination we construct. By asking them to select a one or the other
>> via key prefix we are in fact still forcing them to specify what it is
>> they want, which they could easily do in a 'destination.' entry with a
>> string that specifies whether it is a queue or a topic, i.e there is
>> really no significant difference. I am not saying we dont encourage
>> use of the queue and topic prefixes for Address Strings, but I dont
>> see the need to remove it and thus affect all the existing BURL users
>> and anyone specifying an explicitly typed Address string for no good
>> reason.
>>
>> I dont see that there is much *extra* scope for simplification than
>> exists in general that would require forcing alteration of the
>> existing behaviour of the JNDI file for pretty much every user who has
>> ever used Qpid via a Binding URL or an Address string. The only people
>> we really *need* to change their behaviour are those currently
>> specifying an address string without node type into a 'destination.'
>> entry. I think we can and should add the new behaviour of specifying
>> Address strings via 'queue.' and 'topic.' prefixes, and fix it so
>> people cant get anonymous Destinations when specifying Address strings
>> via 'destination.' entries, all whilst maintaining the other
>> behaviours for existing users. I dont particularly think we need to
>> worry about BURL users being confused by whether they are using Queues
>> or Topics given the way they are usually specified (i.e exhange and
>> routing key, which is the same whether the thing on the other end is
>> actually a Queue or a Topic) and constructed/used (again, check the
>> JNDI related code), I think we simply leave the behaviour of those
>> things untouched and rely on the fact it has been working the way it
>> has for anything up to 6+ years or now and most people using them thus
>> know how that works already.
>>
>> Robbie
>>
>> On 29 February 2012 19:22, Rajith Attapattu <rajith77@gmail.com> wrote:
>>> On Wed, Feb 29, 2012 at 12:49 PM, Robbie Gemmell
>>> <robbie.gemmell@gmail.com> wrote:
>>>> Just to be clear, I have never been suggesting we remove
>>>> 'destination.' entries from the equation. I think we should keep
>>>> 'destination.' as we do have users already using it as its the only
>>>> way to specify ADDRs in there,
>>>
>>> The changes I'm proposing includes allowing both ADDR or BURL to be
>>> specified for "queue" and "topics".
>>> This will result in a concrete implementation of javax.jms.Queue or
>>> javax.jms.Topic being returned.
>>>
>>> As Rob points out, I think the pain in changing an existing
>>> "destination" entry into a distinct "queue" or "topic" would force an
>>> administrator to really think through the implications of the address
>>> string entry.
>>> We have lots of users who are confused with addressing and forcing
>>> them to specify their address as a "queue" or a "topic" is going to
>>> reduce a lot of pain for them down the line.
>>>
>>> A huge plus is the simplicity we achieve in our code base by doing this.
>>>>> and its also still an important entry
>>>> point for people using BURLs without the arbitary assignment to
>>>> amq.topic or amq.direct exchanges. I just think we should fix it.
>>>
>>> I'm not exactly sure what you meant here.
>>> But to clarify, amq.topic doesn't imply Topics any more than
>>> amq.direct implies Queue.
>>> We could do Topic style messaging using amq.topic, amq.direct,
>>> amq.fanout or any custom exchange.
>>>
>>> The existing implementation of AMQQueue and AMQTopic is very limited
>>> in that it assume amq.topic for Topics and amq.direct for Queues.
>>>
>>>> We should make people using it with ADDRs specify what type of node
>>>> they desire, but theres no reason not to let any users who are already
>>>> doing that or anyone using BURLs just continue to have their code keep
>>>> working. Only the users using ADDRs and 'destination.' who currently
>>>> arent setting a node type would need to adjust, which they would all
>>>> have to anyway if we ever removed it in favour of the other two
>>>> varieties.
>>>
>>> As for backwards compatibility,
>>> Well for ADDR we can throw an exception unless a type is specified.
>>>
>>> How do you propose we tackle BURL ?
>>> Choose Queue or Topic based on the type of exchange being used ?
>>> What if they use a custom exchange? how would we determine the type ?
>>> This amounts to the magic we are already doing with address string
>>> albeit with not so stellar results with TCK failures etc....
>>> Instead of us trying to solve those headaches in the code,  lets
>>> simplify it by asking the administrator to make a decision.
>>>
>>> Even if we allow "destination" for backwards compatibility, I'm
>>> strongly in favour of deprecating "destination" and making it quite
>>> clear in release notes and documentation.
>>>
>>> Again the pain of having to deal with Queue vs Topic is best handled
>>> at configuration time by an administrator than us trying to do the
>>> magic in the code.
>>>
>>> Regards,
>>>
>>> Rajith
>>>
>>>> Robbie
>>>>
>>>> On 29 February 2012 15:46, Rajith Attapattu <rajith77@gmail.com> wrote:
>>>>> Robbie,
>>>>>
>>>>> My preference is also to just use "queue" and "topic" qualifiers and
>>>>> deprecate "destination" , hence listed it as option #1 in a previous
>>>>> email.
>>>>> I agree with you, Rob and Gordon that the above approach is simple,
>>>>> easy and more importantly less buggy.
>>>>>
>>>>> The reason I had "destination" in my summary is due to backwards compatibility.
>>>>> However I agree with Rob that the pain of deprecating the
>>>>> "destination"  qualifier is much less than having to deal with the
>>>>> issues at the code level and the runtime issues a user might
>>>>> encounter.
>>>>> I will post the code soon for review.
>>>>>
>>>>> Gentlemen, Thanks again for sharing your thoughts on this.
>>>>>
>>>>> Regards,
>>>>>
>>>>> Rajith
>>>>>
>>>>> On Wed, Feb 29, 2012 at 7:46 AM, Robbie Gemmell
>>>>> <robbie.gemmell@gmail.com> wrote:
>>>>>> On 28 February 2012 17:35, Rajith Attapattu <rajith77@gmail.com>
wrote:
>>>>>>> Based on the discussion I would like to outline the following
proposal.
>>>>>>> I believe it reflects the consensus so far. Please correct me
if
>>>>>>> anything is amiss.
>>>>>>>
>>>>>>> 1. If the user wants to use the specialized interfaces (JMS 1.0)
and
>>>>>>> pass in either a Queue or a Topic, then they should be using
"queue"
>>>>>>> or "topic" in the jndi file.
>>>>>>>
>>>>>>>   - This will result in a Destination impl being returned, that
>>>>>>> implements either javax.jms.Queue or javax.jms.Topic depending
on the
>>>>>>> qualifier used.
>>>>>>>   - These destinations can obviously be used with the JMS 1.1
methods
>>>>>>> which expects the generic javax.jms.Destination.
>>>>>>>   - If a "type" is specified and is different from the qualifier
>>>>>>> being used, we will raise an exception highlighting the discrepancy.
>>>>>>>         Ex. topic.my-topic="my-queue; {type : queue}"
>>>>>>
>>>>>>
>>>>>> This seems reasonable yes.
>>>>>>
>>>>>>>
>>>>>>> 2. If "destination.<jndi-name>=<address>" is used,
>>>>>>>
>>>>>>>   - This will return a Destination impl that only implements
the
>>>>>>> javax.jms.Destination.
>>>>>>>   -  ** If this destination is used with JMS 1.0 methods, it
will
>>>>>>> result in a class cast exception.**
>>>>>>>   - We will not attempt to determine if the address used here
is a
>>>>>>> JMS Topic or a JMS Queue.
>>>>>>>   - If a "type" is specified with the address we will use that
as a
>>>>>>> hint when trying to check the presence of the node in the broker.
>>>>>>>     Ex if hello; {type=topic}, for 0-10 we will attempt to
see if
>>>>>>> there is an exchange in the broker named "hello" and throw an
>>>>>>> exception if no create instructions are given.
>>>>>>>
>>>>>>
>>>>>> This one I am less keen on. If a user specifies a type in their
>>>>>> address string, then I think thats the type of object they should
get,
>>>>>> it shouldnt just be a hint used at some random point later when the
>>>>>> Destination is used and we perform magic. If someone defined a
>>>>>> 'destination.' entry saying that it is a queue, I would for example
>>>>>> expect that to work when trying to create a QueueBrowser rather than
>>>>>> throwing a ClassCastException. We had a user submit a patch just
weeks
>>>>>> ago to fix that exact use case.
>>>>>>
>>>>>> I think the scope for anyone getting a Destination object that didnt
>>>>>> actually form a concreate Queue or Topic implementation should be
>>>>>> absolutely minimal. I'd really rather prefer it didnt happen as I
dont
>>>>>> think it should ever be necessary for current use cases, and I'm
a big
>>>>>> fan of users actually having to ask for exactly what they want and
>>>>>> then us trying to give them what they asked for. If what they asked
>>>>>> for isnt compatible with whats found on the broker then I think thats
>>>>>> when we throw an Exception telling them so, not when we start deciding
>>>>>> things for them.
>>>>>>
>>>>>> Although JMS defines 'Destination', I really dont think it ever
>>>>>> expects there to be direct implementations of it as it lists no
>>>>>> methods, not even the mandated toString() 'representation of
>>>>>> destination' as found on Queue and Topic. Destination was added to
>>>>>> allow the new cross-domain Session objects in 1.1 to use of both
>>>>>> existing Queue and Topic objects on a single session at the same
time;
>>>>>> there was no session.createDestination() method added along with
it.
>>>>>> The spec still deals with two domains, P2P and Pub/Sub, via the same
>>>>>> old Queue and Topic types that now happen to also be Destinations.
>>>>>>
>>>>>> Do we know of any other providers who provide their users Destination
>>>>>> objects which arent implementations of Queue or Topic? The idea had
>>>>>> never entered my head previous to these Addressing discussions, so
I
>>>>>> am genuinely interested to know if there are.
>>>>>>
>>>>>> Robbie
>>>>>>
>>>>>>> Does this sound reasonable ? Please feel free to add/change anything
I
>>>>>>> have missed.
>>>>>>>
>>>>>>> Regards,
>>>>>>>
>>>>>>> Rajith
>>>>>>>
>>>>>>> On Tue, Feb 28, 2012 at 9:53 AM, Rajith Attapattu <rajith77@gmail.com>
wrote:
>>>>>>>> Rob,
>>>>>>>>
>>>>>>>> Addressing is indeed a pain point and most of it is due to
grey areas
>>>>>>>> causing undesirable side effects.
>>>>>>>> I've got some work that I'm hoping to post today.
>>>>>>>>
>>>>>>>> Let me first check that into a branch and then I will post
a brief
>>>>>>>> outline of the design and the code in review board.
>>>>>>>> I'm hoping to wrap this up in next 2 weeks.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>>
>>>>>>>> Rajith
>>>>>>>>
>>>>>>>> On Tue, Feb 28, 2012 at 4:01 AM, Rob Godfrey <rob.j.godfrey@gmail.com>
wrote:
>>>>>>>>> As an aside, I have been labeling the open Java Client
JIRAs so its
>>>>>>>>> easier to pick out clusters of JIRAs that can be worked
on together /
>>>>>>>>> see where the real pain points are.  A quick report
on open JIRAs per
>>>>>>>>> label:
>>>>>>>>>
>>>>>>>>> 17 - addressing
>>>>>>>>>  9 - failover
>>>>>>>>>  9 - exception-handling
>>>>>>>>>  4 - deadlock
>>>>>>>>>  3 - timestamp
>>>>>>>>>  3 - possibly_complete
>>>>>>>>>  3 - message-credit
>>>>>>>>>  3 - jms-compliance
>>>>>>>>>  2 - serialization
>>>>>>>>>  2 - documentation
>>>>>>>>>  1 - examples
>>>>>>>>>  1 - browsing
>>>>>>>>>  1 - amqp_compliance
>>>>>>>>>
>>>>>>>>> Addressing covers anything to do with Destinations (ADDR
or BURL) but
>>>>>>>>> is clearly the major pain point... Rajith - I know you
were working on
>>>>>>>>> a patch for this... what is the status of this work?
>>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>> Rob
>>>>>>>>>
>>>>>>>>> On 28 February 2012 09:19, Rob Godfrey <rob.j.godfrey@gmail.com>
wrote:
>>>>>>>>>> On 28 February 2012 05:37, Rajith Attapattu <rajith77@gmail.com>
wrote:
>>>>>>>>>>> If the "queue" and "topic" qualifiers are used
then I guess it makes
>>>>>>>>>>> it really easy for us to work out the validation.
>>>>>>>>>>>
>>>>>>>>>>> What are we going to do with the "destination"
qualifier ?
>>>>>>>>>>> Ex destination.my-dest=<address-string>
>>>>>>>>>>>
>>>>>>>>>>> 1. We deprecate this and get qpid users to use
one of "queue" or
>>>>>>>>>>> "topic" as the administrator who writes the jndi
file surely knows
>>>>>>>>>>> what it's going to be.
>>>>>>>>>>>
>>>>>>>>>>> 2. We create the destination but not allow it
to be used with any
>>>>>>>>>>> methods that require the Queue or Topic interface.
>>>>>>>>>>
>>>>>>>>>> ^^ This - it should be created as a Destination that
implements
>>>>>>>>>> neither Queue nor Topic.
>>>>>>>>>>
>>>>>>>>>>> 3. Attempt to figure out if the address is a
Topic or a Queue based on
>>>>>>>>>>> the current behaviour (as described in my first
email) and then
>>>>>>>>>>> convert it a Queue or Topic if the Destination
object is passed to any
>>>>>>>>>>> methods that require a Queue/Topic interface.
>>>>>>>>>>
>>>>>>>>>> As per my previous comment on the JIRA, I think it's
not actually
>>>>>>>>>> possible to determine from an address string what
is a "topic" and
>>>>>>>>>> what is a "queue".  I can define a "queue" which
distributes the
>>>>>>>>>> messages it hold to every consumer, and removes messages
only when
>>>>>>>>>> every current consumer has irrevocably passed that
message... is this
>>>>>>>>>> a "queue" or a "topic"?  From a JMS perspective
it behaves exactly as
>>>>>>>>>> you would expect from a topic (especially in an AMQP
1-0 scenario
>>>>>>>>>> where you can create durable subscribers with durable
links).  However
>>>>>>>>>> from an AMQP 0-x perspective this looks like a "queue".
 (Indeed on my
>>>>>>>>>> AMQP 1-0 branch I have implemented exactly this type
of "queue" in the
>>>>>>>>>> Java broker... in a class called "Topic" :-) ). Conversely
I could
>>>>>>>>>> define an exchange type which for any given message
will route to *at
>>>>>>>>>> most one* bound queue... this "work sharing exchange"
has many of the
>>>>>>>>>> properties of a "queue" in JMS semantics, but looks
like how we
>>>>>>>>>> currently implement "topics" in AMQP 0-x.
>>>>>>>>>>
>>>>>>>>>> Given the above I think it is fruitless and indeed
even incorrect to
>>>>>>>>>> attempt to determine whether a given address satisfies
JMS "Queue" or
>>>>>>>>>> JMS "Topic" semantics based on the address itself.
>>>>>>>>>>
>>>>>>>>>> -- Rob
>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>>
>>>>>>>>>>> Rajith
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Feb 27, 2012 at 11:30 PM, Rajith Attapattu
<rajith77@gmail.com> wrote:
>>>>>>>>>>>> As per the discussion on QPID-792, I'm moving
the discussion onto the
>>>>>>>>>>>> dev list under.
>>>>>>>>>>>> I have attempted to summarise the current
behaviour and some of the
>>>>>>>>>>>> comments expressed by Rob and Robbie.
>>>>>>>>>>>>
>>>>>>>>>>>> Currently the clients (C++, python and JMS)
resolves an address (with
>>>>>>>>>>>> the 0-10 protocol) as follows.
>>>>>>>>>>>>
>>>>>>>>>>>> 1. If the name resolves to a queue, we treat
it as a Queue
>>>>>>>>>>>> 2. If the name resolves to an exchange, we
treat is a Topic
>>>>>>>>>>>> 3. If it doesn't resolve to either, we treat
it as a Queue.
>>>>>>>>>>>>
>>>>>>>>>>>> Rob made the following comments
>>>>>>>>>>>> "I don't think that we should be trying to
do this because I'm pretty
>>>>>>>>>>>> sure that it is impossible to determine what
is a Queue and what is a
>>>>>>>>>>>> Topic.
>>>>>>>>>>>>
>>>>>>>>>>>> I think the closest we can come is to say
that an address that says
>>>>>>>>>>>> you have to create a new temporary auto-delete
exclusive queue for
>>>>>>>>>>>> every consumer should be treated as a topic...
but the converse is not
>>>>>>>>>>>> true. As far as I am concerned the distinction
between Queue and Topic
>>>>>>>>>>>> is something that only the "administrator"
can determine, and the code
>>>>>>>>>>>> cannot determine dynamically."
>>>>>>>>>>>>
>>>>>>>>>>>> Robbie also expressed the following,
>>>>>>>>>>>> "I also think that the (Java) client shouldnt
be making gueses as to
>>>>>>>>>>>> whether something is a Queue or a Topic,
as I'm sure was fairly
>>>>>>>>>>>> evident from previous mailings on the subject
last year. If that
>>>>>>>>>>>> questionable behaviour is causing pain, then
we should at least
>>>>>>>>>>>> consider simply not doing it. Destination
is itself only the parent
>>>>>>>>>>>> interface of Queue and Topic, it doesnt actually
offer any methods
>>>>>>>>>>>> (even the toString, though for backwards
compatibility reasons
>>>>>>>>>>>> admitedly) and really only serves to allow
creating Topic and Queue
>>>>>>>>>>>> consumers etc without having to have a specific
Session type. I
>>>>>>>>>>>> realise forcing users to specify queue or
topic in the address string
>>>>>>>>>>>> wouldnt be consistent with the other clients,
but I do think its worth
>>>>>>>>>>>> noting that the Java client isn't entirely
consistent with the other
>>>>>>>>>>>> clients for obvious reasons and trying to
make it more so isnt
>>>>>>>>>>>> necessarily always going to be a helpful
or useful thing."
>>>>>>>>>>>>
>>>>>>>>>>>> Rob, further states that we could utilize
the queue and topic
>>>>>>>>>>>> qualifiers that is currently present in our
JNDI mechanism.
>>>>>>>>>>>> "I don't think the queue/topic distinction
even needs to go into the
>>>>>>>>>>>> address - it should only needs to be defined
some way in the JNDI
>>>>>>>>>>>> source
>>>>>>>>>>>>
>>>>>>>>>>>> e.g. in a properties file then things that
begin
>>>>>>>>>>>>
>>>>>>>>>>>> queue.<NAME>=<address string>
>>>>>>>>>>>>
>>>>>>>>>>>> would be queues, and
>>>>>>>>>>>>
>>>>>>>>>>>> topic.<NAME>=<address string>
>>>>>>>>>>>>
>>>>>>>>>>>> would be topics"
>>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>>
>>>>>>>>>>>> Rajith
>>>>>>>>>>>
>>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>> Apache Qpid - AMQP Messaging Implementation
>>>>>>>>>>> Project:      http://qpid.apache.org
>>>>>>>>>>> Use/Interact: mailto:dev-subscribe@qpid.apache.org
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>> Apache Qpid - AMQP Messaging Implementation
>>>>>>>>> Project:      http://qpid.apache.org
>>>>>>>>> Use/Interact: mailto:dev-subscribe@qpid.apache.org
>>>>>>>>>
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> Apache Qpid - AMQP Messaging Implementation
>>>>>>> Project:      http://qpid.apache.org
>>>>>>> Use/Interact: mailto:dev-subscribe@qpid.apache.org
>>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> Apache Qpid - AMQP Messaging Implementation
>>>>>> Project:      http://qpid.apache.org
>>>>>> Use/Interact: mailto:dev-subscribe@qpid.apache.org
>>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> Apache Qpid - AMQP Messaging Implementation
>>>>> Project:      http://qpid.apache.org
>>>>> Use/Interact: mailto:dev-subscribe@qpid.apache.org
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> Apache Qpid - AMQP Messaging Implementation
>>>> Project:      http://qpid.apache.org
>>>> Use/Interact: mailto:dev-subscribe@qpid.apache.org
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> Apache Qpid - AMQP Messaging Implementation
>>> Project:      http://qpid.apache.org
>>> Use/Interact: mailto:dev-subscribe@qpid.apache.org
>>>
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project:      http://qpid.apache.org
> Use/Interact: mailto:dev-subscribe@qpid.apache.org
>

---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org


Mime
View raw message