tomee-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andy Gumbrecht <agumbre...@tomitribe.com>
Subject Re: need advice for single queue configuration
Date Tue, 13 Oct 2015 09:11:44 GMT
If you need this blocking on the consumer MDB then restrict the number of
consumers so you don't starve the MDB pool


@ActivationConfigProperty(propertyName = "maxSessions", propertyValue =
"5"),
@ActivationConfigProperty(propertyName = "maxMessagesPerSessions",
propertyValue = "1"),

So that would be 5 beans consuming 1 message, blocking until space is
available (ie. on success or failure).

Andy.


On 9 October 2015 at 16:25, Leonardo K. Shikida <shikida@gmail.com> wrote:

> Hi Andy
>
> I think I am trying to use activeMQ as the workmanager here. My consumers
> are MDBs, so I can control the pool size via tomee.xml and the
> transactional nature of JMS is exactly what I was counting with. If a
> message delivery fails, then it automatically returns to the queue (because
> it's was not consumed - consumed means processed here).
>
> Your suggestion if I've understood it well, is to detach the processing
> from the message consuming. The problem here is that I'd have to implement
> the same kind of control that activemq was supposed to provide, which is a
> consuming from a queue inside boundaries so resources (such as memory
> mainly) won't exhaust. If I detach, I'll be just moving the problem from
> the message consuming (which will be fast) to the workmanager (the new
> consumer here, that must now be controlled not to throw too many
> long-running threads because message consuming now is fast).
>
>
>
> []
>
> Leo
>
> On Fri, Oct 9, 2015 at 10:46 AM, agumbrecht@tomitribe.com <
> agumbrecht@tomitribe.com> wrote:
>
> > Ouch, I'll try and point you in a better direction. A message consumer
> > should never block longer than at maximum a few seconds, else you are
> going
> > to starve the pool in no time. You should only be checking that the
> message
> > payload is valid. If you need a long running task then hand the message
> off
> > to a workmanager. If the task fails, then simply resend the message to
> the
> > queue and let the next consumer try again. If you do that then be sure to
> > add some message property to keep count of how many times it's failed, so
> > the task doesn't keep sending it.
> >
> > Andy.
> >
> > agumbrecht@tomitribe.com
> > http://www.tomitribe.com
> >
> > Sent from my mobile device.
> >
> > ----- Reply message -----
> > From: "Tim Bain" <tbain@alumni.duke.edu>
> > To: "ActiveMQ Users" <users@activemq.apache.org>
> > Cc: <users@tomee.apache.org>
> > Subject: need advice for single queue configuration
> > Date: Fri, Oct 9, 2015 15:20
> >
> > If you want redelivery to other consumers rather than just the one to
> which
> > the message was first delivered (which is how I interpreted your
> paragraph
> > about message-driven beans) and you can live with out-of-order delivery,
> > I'd think you'd want to configure broker redelivery as described at the
> > bottom of
> > http://activemq.apache.org/message-redelivery-and-dlq-handling.html.
> >
> > When TomEE "simply stops consuming the queue", have you taken a thread
> dump
> > to see what its threads are doing?  Is it possible that every MDB is in
> the
> > middle of processing a 1-hour message?
> >
> > Have you used a JMX viewer such as JConsole to examine the subscriptions
> on
> > your destination on the broker when TomEE is having problems, to see if
> > it's still subscribed?
> >
> > Tim
> > On Sep 11, 2015 11:34 AM, "Leonardo K. Shikida" <shikida@gmail.com>
> wrote:
> >
> > > Hi
> > >
> > > I need a simple queue.
> > >
> > > This queue will be persistent and it will use a database in the backend
> > to
> > > store the messages.
> > >
> > > It will receive produced messages small in size (just one or two string
> > > attributes) in bursts of 10~100.
> > >
> > > It will consume messages by a pool of 100~300 message driven beans
> > (tomee).
> > > Each message will be consumed and will trigger a long-running job that
> > may
> > > take from 1 minute to 1 hour to complete. If the job fails, I want the
> > > message to return to the queue so it can be consumed again, no matter
> how
> > > many retries.
> > >
> > > I don't want to send invalid messages to a "poisoned messages queue".
> > >
> > > Architecture is simple. The queue will run in a single activeMQ
> instance
> > > and consumers are MDBs from a single TomEE+ web application.
> > >
> > > So, here's what I have so far
> > >
> > > activemq.xml=========================================
> > >
> > >     <broker xmlns="http://activemq.apache.org/schema/core"
> > > brokerName="localhost" dataDirectory="${activemq.data}">
> > >         <destinationPolicy>
> > >             <policyMap>
> > >                 <policyEntries>
> > >                     <policyEntry topic=">" producerFlowControl="true">
> > >                         <pendingMessageLimitStrategy>
> > >                             <constantPendingMessageLimitStrategy
> > > limit="1000" />
> > >                         </pendingMessageLimitStrategy>
> > >                     </policyEntry>
> > >                     <policyEntry queue=">" producerFlowControl="true"
> > > memoryLimit="1mb">
> > >                     </policyEntry>
> > >                 </policyEntries>
> > >             </policyMap>
> > >         </destinationPolicy>
> > >
> > >         <persistenceAdapter>
> > >             <jdbcPersistenceAdapter dataSource="#oracle-ds" />
> > >         </persistenceAdapter>
> > >
> > >         <systemUsage>
> > >             <systemUsage>
> > >                 <memoryUsage>
> > >                     <memoryUsage limit="1 gb" />
> > >                 </memoryUsage>
> > >                 <storeUsage>
> > >                     <storeUsage limit="100 gb" />
> > >                 </storeUsage>
> > >                 <tempUsage>
> > >                     <tempUsage limit="50 gb" />
> > >                 </tempUsage>
> > >             </systemUsage>
> > >         </systemUsage>
> > >
> > >         <transportConnectors>
> > >             <transportConnector name="tcp" uri="tcp://0.0.0.0:61616"
> />
> > >         </transportConnectors>
> > >     </broker>
> > >
> > >     <bean id="oracle-ds"
> class="org.apache.commons.dbcp.BasicDataSource"
> > > destroy-method="close">
> > >         <property name="driverClassName"
> value="oracle.jdbc.OracleDriver"
> > > />
> > >         <property name="url" value="jdbc:oracle:thin:@localhost
> :1521:XE"
> > > />
> > >         <property name="username" value="*****" />
> > >         <property name="password" value="******" />
> > >         <property name="poolPreparedStatements" value="true" />
> > >         <property name="maxActive" value="100" />
> > >     </bean>
> > >
> > > tomee.xml=========================================
> > >
> > >     <Resource id="Default JMS Resource Adapter"
> > > type="ActiveMQResourceAdapter">
> > >         BrokerXmlConfig =  broker:(tcp://localhost:61616)
> > >         ServerUrl = tcp://localhost:61616
> > >         ThreadPoolSize = 200
> > >     </Resource>
> > >
> > >     <Resource id="MyJmsConnectionFactory"
> > > type="javax.jms.ConnectionFactory">
> > >     ResourceAdapter = Default\ JMS\ Resource\ Adapter
> > >     PoolMaxSize = 200
> > >     </Resource>
> > >
> > >     <Resource id="JobQueue" type="javax.jms.Queue" />
> > >
> > > MDB=========================================
> > >
> > > @MessageDriven(activationConfig = {
> > >         @ActivationConfigProperty(propertyName = "destinationType",
> > > propertyValue = "javax.jms.Queue"),
> > >         @ActivationConfigProperty(propertyName = "destination",
> > > propertyValue = "JobQueue"),
> > >         @ActivationConfigProperty(propertyName =
> > "maxMessagesPerSessions",
> > > propertyValue = "1"),
> > >         @ActivationConfigProperty(propertyName = "maxSessions",
> > > propertyValue = "200"),
> > >         @ActivationConfigProperty(propertyName = "acknowledgeMode",
> > > propertyValue = "Auto-acknowledge") })
> > >
> > > =========================================
> > >
> > > With this configuration, I am still facing some interruptions.
> > >
> > > Sometimes, TomEE simply stops consuming the queue (once or twice per
> > week),
> > > so I have to restart it. I am still checking the logs for anything
> > > suspicious.
> > >
> > > I am using TomEE+ 1.6.0 stable (quite old, I know) and a standalone AMQ
> > > 5.10 stable in a linux environment and oracle JDK 7.
> > >
> > > But I'd like some advice on how to properly set up activemq / tomee in
> > this
> > > scenario (that's why I am sending this email to both activemq and tomee
> > > lists)
> > >
> > > Thank you very much, any help is welcome.
> > >
> > > []
> > >
> > > Leo
> > >
> >
>



-- 
  Andy Gumbrecht
  https://twitter.com/AndyGeeDe
  http://www.tomitribe.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message