qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From HADI Ali <Ali.H...@murex.com>
Subject RE: Dispatch Router prefetch
Date Tue, 12 Mar 2019 13:22:46 GMT
Hello,

In our use case we have polling consumers with a prefetch policy of zero that issues one credit
at a time every few seconds. Between two receive, the consumer will be attached with zero
credit.
Thus, not considering a consumer to be a routable destination until it issues initial credit
would address the problem only for the first message, because the dispatch will still prefetch
possibly expired messages as soon as the destination is considered routable.

In this use case we are consuming a few messages per minutes and TTLs are between 2 to 5 seconds.
Concerning the granularity, one second should be sufficient for us.

We also noticed that the broker is not forwarding the TTL set at the level of the queue. Is
this an expected behavior?

Thanks,
Ali

-----Original Message-----
From: Ted Ross <tross@redhat.com>
Sent: lundi 11 mars 2019 15:32
To: users@qpid.apache.org
Subject: Re: Dispatch Router prefetch

On Fri, Mar 8, 2019 at 9:19 AM Gordon Sim <gsim@redhat.com> wrote:

> On 08/03/2019 2:12 pm, Gordon Sim wrote:
> > On 08/03/2019 12:59 pm, HADI Ali wrote:
> >> Hello,
> >>
> >> We are actually using in our cluster multiple brokers and thus we
> >> need to define the same address on multiple brokers.
> >> For this, we cannot use linkroutes as suggested, but we still need
> >> to have the correct behavior of the TTL in our cluster.
> >>
> >> Is it an option to manage the TTL of the message at the level of
> >> the dispatch router since we have all of the information needed in
> >> the message headers?
> >
> > It doesn't do that at present, but it doesn't seem like an
> > reasonable enhancement to me.
>
> Sorry, meant to say it doesn't seem like an *un*reasonable enhancement!
>

I'd like to better understand the use case here.  We've avoided adding any kind of TTL support
in Dispatch Router up to this point.

I assume, based on the fact that prefetch-1 didn't solve your problem, that you have consumers
that are attached but don't issue credit for long periods of time.  Is this accurate?

What is the pattern of your consumers?  Do they attach, then later issue credit to process
a message?  How many messages per second/minute/hour do your consumers handle?  Do they issue
one credit at a time?

What are the typical TTLs in your messages?  How granular does the expiration need to be (i.e.
how accurate of a timer would need to be used to tag each incoming delivery)?  Would one-second
granularity be sufficient, or do you need milliseconds?

An alternate approach would be to not consider a consumer to be a routable destination until
it issues initial credit.  Would this address your problem?


>
> >> In Internet Protocol, ipv4 for example, the routers manage the TTL
> >> and discard any expired messages.
> >>
> >> Or make it feasible to have the autolinks propagate the credit
> >> directly from consumers?
> >
> > This isn't really possible when you have autolinks for same address
> > to multiple brokers. If the consumer gives 10 credits, how do you
> > propagate that to two brokers?  5 each? What if they don't both have 5 messages?
> > 10 each? Then you are back to the situation where you have more
> > credit issued at source than the consumer has granted.
> >
> > --------------------------------------------------------------------
> > - To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org For
> > additional commands, e-mail: users-help@qpid.apache.org
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org For
> additional commands, e-mail: users-help@qpid.apache.org
>
>
*******************************
This e-mail contains information for the intended recipient only. It may contain proprietary
material or confidential information. If you are not the intended recipient you are not authorized
to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that
it is virus free and accepts no responsibility for any loss or damage arising from its use.
If you have received this e-mail in error please notify immediately the sender and delete
the original email received, any attachments and all copies from your system.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org

Mime
View raw message