tomee-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andy Gumbrecht <agumbre...@tomitribe.com>
Subject Re: need advice for single queue configuration
Date Tue, 13 Oct 2015 09:17:53 GMT
You can tune this so that all your MDBs accumulated maxSessions are less
than the configured MDB pool size (or equal to the size, but I'd leave at
least one spare for thread swapping on accept)

On 13 October 2015 at 11:11, Andy Gumbrecht <agumbrecht@tomitribe.com>
wrote:

> If you need this blocking on the consumer MDB then restrict the number of
> consumers so you don't starve the MDB pool
>
>
> @ActivationConfigProperty(propertyName = "maxSessions", propertyValue =
> "5"),
> @ActivationConfigProperty(propertyName = "maxMessagesPerSessions",
> propertyValue = "1"),
>
> So that would be 5 beans consuming 1 message, blocking until space is
> available (ie. on success or failure).
>
> Andy.
>
>
> On 9 October 2015 at 16:25, Leonardo K. Shikida <shikida@gmail.com> wrote:
>
>> Hi Andy
>>
>> I think I am trying to use activeMQ as the workmanager here. My consumers
>> are MDBs, so I can control the pool size via tomee.xml and the
>> transactional nature of JMS is exactly what I was counting with. If a
>> message delivery fails, then it automatically returns to the queue
>> (because
>> it's was not consumed - consumed means processed here).
>>
>> Your suggestion if I've understood it well, is to detach the processing
>> from the message consuming. The problem here is that I'd have to implement
>> the same kind of control that activemq was supposed to provide, which is a
>> consuming from a queue inside boundaries so resources (such as memory
>> mainly) won't exhaust. If I detach, I'll be just moving the problem from
>> the message consuming (which will be fast) to the workmanager (the new
>> consumer here, that must now be controlled not to throw too many
>> long-running threads because message consuming now is fast).
>>
>>
>>
>> []
>>
>> Leo
>>
>> On Fri, Oct 9, 2015 at 10:46 AM, agumbrecht@tomitribe.com <
>> agumbrecht@tomitribe.com> wrote:
>>
>> > Ouch, I'll try and point you in a better direction. A message consumer
>> > should never block longer than at maximum a few seconds, else you are
>> going
>> > to starve the pool in no time. You should only be checking that the
>> message
>> > payload is valid. If you need a long running task then hand the message
>> off
>> > to a workmanager. If the task fails, then simply resend the message to
>> the
>> > queue and let the next consumer try again. If you do that then be sure
>> to
>> > add some message property to keep count of how many times it's failed,
>> so
>> > the task doesn't keep sending it.
>> >
>> > Andy.
>> >
>> > agumbrecht@tomitribe.com
>> > http://www.tomitribe.com
>> >
>> > Sent from my mobile device.
>> >
>> > ----- Reply message -----
>> > From: "Tim Bain" <tbain@alumni.duke.edu>
>> > To: "ActiveMQ Users" <users@activemq.apache.org>
>> > Cc: <users@tomee.apache.org>
>> > Subject: need advice for single queue configuration
>> > Date: Fri, Oct 9, 2015 15:20
>> >
>> > If you want redelivery to other consumers rather than just the one to
>> which
>> > the message was first delivered (which is how I interpreted your
>> paragraph
>> > about message-driven beans) and you can live with out-of-order delivery,
>> > I'd think you'd want to configure broker redelivery as described at the
>> > bottom of
>> > http://activemq.apache.org/message-redelivery-and-dlq-handling.html.
>> >
>> > When TomEE "simply stops consuming the queue", have you taken a thread
>> dump
>> > to see what its threads are doing?  Is it possible that every MDB is in
>> the
>> > middle of processing a 1-hour message?
>> >
>> > Have you used a JMX viewer such as JConsole to examine the
>> subscriptions on
>> > your destination on the broker when TomEE is having problems, to see if
>> > it's still subscribed?
>> >
>> > Tim
>> > On Sep 11, 2015 11:34 AM, "Leonardo K. Shikida" <shikida@gmail.com>
>> wrote:
>> >
>> > > Hi
>> > >
>> > > I need a simple queue.
>> > >
>> > > This queue will be persistent and it will use a database in the
>> backend
>> > to
>> > > store the messages.
>> > >
>> > > It will receive produced messages small in size (just one or two
>> string
>> > > attributes) in bursts of 10~100.
>> > >
>> > > It will consume messages by a pool of 100~300 message driven beans
>> > (tomee).
>> > > Each message will be consumed and will trigger a long-running job that
>> > may
>> > > take from 1 minute to 1 hour to complete. If the job fails, I want the
>> > > message to return to the queue so it can be consumed again, no matter
>> how
>> > > many retries.
>> > >
>> > > I don't want to send invalid messages to a "poisoned messages queue".
>> > >
>> > > Architecture is simple. The queue will run in a single activeMQ
>> instance
>> > > and consumers are MDBs from a single TomEE+ web application.
>> > >
>> > > So, here's what I have so far
>> > >
>> > > activemq.xml=========================================
>> > >
>> > >     <broker xmlns="http://activemq.apache.org/schema/core"
>> > > brokerName="localhost" dataDirectory="${activemq.data}">
>> > >         <destinationPolicy>
>> > >             <policyMap>
>> > >                 <policyEntries>
>> > >                     <policyEntry topic=">" producerFlowControl="true">
>> > >                         <pendingMessageLimitStrategy>
>> > >                             <constantPendingMessageLimitStrategy
>> > > limit="1000" />
>> > >                         </pendingMessageLimitStrategy>
>> > >                     </policyEntry>
>> > >                     <policyEntry queue=">" producerFlowControl="true"
>> > > memoryLimit="1mb">
>> > >                     </policyEntry>
>> > >                 </policyEntries>
>> > >             </policyMap>
>> > >         </destinationPolicy>
>> > >
>> > >         <persistenceAdapter>
>> > >             <jdbcPersistenceAdapter dataSource="#oracle-ds" />
>> > >         </persistenceAdapter>
>> > >
>> > >         <systemUsage>
>> > >             <systemUsage>
>> > >                 <memoryUsage>
>> > >                     <memoryUsage limit="1 gb" />
>> > >                 </memoryUsage>
>> > >                 <storeUsage>
>> > >                     <storeUsage limit="100 gb" />
>> > >                 </storeUsage>
>> > >                 <tempUsage>
>> > >                     <tempUsage limit="50 gb" />
>> > >                 </tempUsage>
>> > >             </systemUsage>
>> > >         </systemUsage>
>> > >
>> > >         <transportConnectors>
>> > >             <transportConnector name="tcp" uri="tcp://0.0.0.0:61616"
>> />
>> > >         </transportConnectors>
>> > >     </broker>
>> > >
>> > >     <bean id="oracle-ds"
>> class="org.apache.commons.dbcp.BasicDataSource"
>> > > destroy-method="close">
>> > >         <property name="driverClassName"
>> value="oracle.jdbc.OracleDriver"
>> > > />
>> > >         <property name="url" value="jdbc:oracle:thin:@localhost
>> :1521:XE"
>> > > />
>> > >         <property name="username" value="*****" />
>> > >         <property name="password" value="******" />
>> > >         <property name="poolPreparedStatements" value="true" />
>> > >         <property name="maxActive" value="100" />
>> > >     </bean>
>> > >
>> > > tomee.xml=========================================
>> > >
>> > >     <Resource id="Default JMS Resource Adapter"
>> > > type="ActiveMQResourceAdapter">
>> > >         BrokerXmlConfig =  broker:(tcp://localhost:61616)
>> > >         ServerUrl = tcp://localhost:61616
>> > >         ThreadPoolSize = 200
>> > >     </Resource>
>> > >
>> > >     <Resource id="MyJmsConnectionFactory"
>> > > type="javax.jms.ConnectionFactory">
>> > >     ResourceAdapter = Default\ JMS\ Resource\ Adapter
>> > >     PoolMaxSize = 200
>> > >     </Resource>
>> > >
>> > >     <Resource id="JobQueue" type="javax.jms.Queue" />
>> > >
>> > > MDB=========================================
>> > >
>> > > @MessageDriven(activationConfig = {
>> > >         @ActivationConfigProperty(propertyName = "destinationType",
>> > > propertyValue = "javax.jms.Queue"),
>> > >         @ActivationConfigProperty(propertyName = "destination",
>> > > propertyValue = "JobQueue"),
>> > >         @ActivationConfigProperty(propertyName =
>> > "maxMessagesPerSessions",
>> > > propertyValue = "1"),
>> > >         @ActivationConfigProperty(propertyName = "maxSessions",
>> > > propertyValue = "200"),
>> > >         @ActivationConfigProperty(propertyName = "acknowledgeMode",
>> > > propertyValue = "Auto-acknowledge") })
>> > >
>> > > =========================================
>> > >
>> > > With this configuration, I am still facing some interruptions.
>> > >
>> > > Sometimes, TomEE simply stops consuming the queue (once or twice per
>> > week),
>> > > so I have to restart it. I am still checking the logs for anything
>> > > suspicious.
>> > >
>> > > I am using TomEE+ 1.6.0 stable (quite old, I know) and a standalone
>> AMQ
>> > > 5.10 stable in a linux environment and oracle JDK 7.
>> > >
>> > > But I'd like some advice on how to properly set up activemq / tomee in
>> > this
>> > > scenario (that's why I am sending this email to both activemq and
>> tomee
>> > > lists)
>> > >
>> > > Thank you very much, any help is welcome.
>> > >
>> > > []
>> > >
>> > > Leo
>> > >
>> >
>>
>
>
>
> --
>   Andy Gumbrecht
>   https://twitter.com/AndyGeeDe
>   http://www.tomitribe.com
>



-- 
  Andy Gumbrecht
  https://twitter.com/AndyGeeDe
  http://www.tomitribe.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message