qpid-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeff Stein (JIRA)" <qpid-...@incubator.apache.org>
Subject [jira] Updated: (QPID-1769) 64 kilobyte messages not dequeued immediately when messageConsumer.receive is called
Date Mon, 23 Mar 2009 21:29:50 GMT

     [ https://issues.apache.org/jira/browse/QPID-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jeff Stein updated QPID-1769:
-----------------------------

    Description: 
I'm running into a bug where, when I send messages 64 kilobytes long via a JMS producer, and
retrieve them via a JMS consumer, it appears they are not dequeued until much later (even
though the consumer is somehow still reading the messages).  It is probably dequeueing finally
when connection.close() or ctx.close() is called.  I've concluded this is the situation, because:

(A) The message number that overflows the queue is the same as the queue size divided by the
message size (i.e., all the messages are still in the queue when the overflow happens).

(B) The qpid-queue-stats program shows no dequeueing occuring.

(C) When I make a simple consumer to run against the 64k message producer, it receives the
messages, despite no actual dequeueing occuring in the queue.  The last thing it does is hang
on messageConsumer.receive(), and the read messages are never dequeued.

(D) When I modify the simple consumer from (C) to timeout after 30 seconds (messageConsumer.receive(30000)),
and it reaches the end of the program by timing out, the dequeues occurs all at once suddenly.

(E) This occurs even when I take it down to about 50 messages per second--no dequeueing occurs
until after the timeout mentioned in (D).

This has the effect of causing my queue to fill up.  Note that I do not have this problem
when sending messages that are 32 kilobytes long and smaller--messages dequeue normally at
those sizes.

I tried to replicate this behavior in the Python client, but that seemed to work without any
problems.

Note that I am running against the C++ broker and my queue size limit is 100 megabytes.

  was:
I'm running into a bug where, when I send messages 64 kilobytes long via a JMS producer, and
retrieve them via a JMS consumer, it appears they are not dequeued until much later (even
though the consumer is somehow still reading the messages).  It is probably dequeueing finally
when connection.close() or ctx.close() is called.  I've concluded this is the situation, because:

(A) The message number that overflows the queue is the same as the queue size divided by the
message size (i.e., all the messages are still in the queue when the overflow happens).

(B) The qpid-queue-stats program shows no dequeueing occuring.

(C) When I make a simple consumer to run against the 64k message producer, it receives the
messages, despite no actual dequeueing occuring in the queue.  The last thing it does is hang
on messageConsumer.receive(), and the read messages are never dequeued.

(D) When I modify the simple consumer from (C) to timeout after 30 seconds (messageConsumer.receive(30000)),
and it reaches the end of the program by timing out, the dequeues occurs all at once suddenly.

(E) This occurs even when I take it down to about 50 messages per second--no dequeueing occurs
until after the timeout mentioned in (D).

This has the effect of causing my queue to fill up.  Note that I do not have this problem
when sending messages that are 32 kilobytes long and smaller--messages dequeue normally at
those sizes.

I tried to replicate this behavior in the Python client, but that seemed to work without any
problems.

Note that I am running against the C++ broker and my queue size limit is 100 megabytes.

-------------------------
You can see this behavior by modifying the direct Producer/Consumer example for JMS like so:

Modifying the Producer:

1) Set numMessages to 2000 instead of 10, so that we reach the queue capacity (assuming 100
Megabytes maximum in the queue).

2) Add this private method:

private String makeString(int length) {
	String r = "r";
	for (int i=0;i<length;i++) {
		r+="r";
	}
	return r;
}

3) Add this line to the first line of the runTest() function:

String bigString = makeString(65536);

4) Change this line in the runTest() function...

message = session.createTextMessage("Message " + i);

...to this:

message = session.createTextMessage("Message " + i + bigString);

Modifying the Consumer:

1) Change this line in runTest()...

System.out.println(CLASS + ": Received message: " + text);

...to this:

System.out.println(CLASS + ": Received message: " + text.substring(0,20));

This last step is so that your console output doesn't get flooded with those "r" characters
that compose most of the message.


> 64 kilobyte messages not dequeued immediately when messageConsumer.receive is called
> ------------------------------------------------------------------------------------
>
>                 Key: QPID-1769
>                 URL: https://issues.apache.org/jira/browse/QPID-1769
>             Project: Qpid
>          Issue Type: Bug
>          Components: Java Client
>    Affects Versions: M4
>         Environment: Redhat
>            Reporter: Jeff Stein
>         Attachments: Consumer.java, Producer.java
>
>
> I'm running into a bug where, when I send messages 64 kilobytes long via a JMS producer,
and retrieve them via a JMS consumer, it appears they are not dequeued until much later (even
though the consumer is somehow still reading the messages).  It is probably dequeueing finally
when connection.close() or ctx.close() is called.  I've concluded this is the situation, because:
> (A) The message number that overflows the queue is the same as the queue size divided
by the message size (i.e., all the messages are still in the queue when the overflow happens).
> (B) The qpid-queue-stats program shows no dequeueing occuring.
> (C) When I make a simple consumer to run against the 64k message producer, it receives
the messages, despite no actual dequeueing occuring in the queue.  The last thing it does
is hang on messageConsumer.receive(), and the read messages are never dequeued.
> (D) When I modify the simple consumer from (C) to timeout after 30 seconds (messageConsumer.receive(30000)),
and it reaches the end of the program by timing out, the dequeues occurs all at once suddenly.
> (E) This occurs even when I take it down to about 50 messages per second--no dequeueing
occurs until after the timeout mentioned in (D).
> This has the effect of causing my queue to fill up.  Note that I do not have this problem
when sending messages that are 32 kilobytes long and smaller--messages dequeue normally at
those sizes.
> I tried to replicate this behavior in the Python client, but that seemed to work without
any problems.
> Note that I am running against the C++ broker and my queue size limit is 100 megabytes.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org


Mime
View raw message