james-server-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Norman Maurer <nor...@apache.org>
Subject Re: OutOfMemory exception
Date Fri, 02 Apr 2010 19:04:45 GMT
Hi Eric,

I forgot to mention that one cause of the OOM could be the current
usage of InMemoryMail. At the moment James load the whole MimeMessage
into memory while processing the camel routes. This is really a bad
practice in terms of performance and could maybe cause a OOM if you
processing many big messages which are bigger (the sumary) as the
avaible memory.
Could this be the problem on your side ?

I'm currently workin on storing the MimeMessage on an external storage
and just stream in on demand.. So only the "meta-data" is stored in
jms and the rest in a db / jcr etc.. I have a proof-of-concept workin
here. But I need a bit more time to see if its the way to go..

I'm also evolating the use of BlobMessages with ActiveMQ.

Bye,
Norman


2010/4/2 Norman Maurer <norman@apache.org>:
> Hi Eric
>
> so at least the files not leak anymore, unfortunaly this was the less
> critical problem.
> About the errors in pop3server I will have a look and see what could
> cause this. I'm 100 % sure it's not related to the OOM.
>
> Any more infos related to the OOM are welcome.
>
> Bye
> Norman
>
> 2010/4/2, Eric Charles <eric.charles@u-mangate.com>:
>> nop. OutOfMemoryError is back.
>>
>> I will double check my server, jdk and environment config, but other
>> java processes are running on the same machine without problem.
>> I will also have a stress scenario on my local dev PC that runs fine and
>> after, I will rerun it on my production server.
>> Dump analysis and full profiling are also to do.
>> Some stuff are difficult to simulate : See for example (occurs regulary)
>> ERROR 16:47:23,508 | james.pop3server | WriteRequest holds not a an
>> valid Object but class java.lang.String
>>
>> There are no more .m64 file in /tmp
>>
>> Tks,
>> Eric
>>
>>
>> ERROR 16:24:53,636 | james.pop3server | WriteRequest holds not a an
>> valid Object but class java.lang.String
>> INFO  16:24:58,061 | james.smtpserver | Successfully spooled mail
>> Mail1270218298023-9820 from jewishnessesyh2@ecs-solar.com on
>> 68.62.205.100 for [ric.charles@u-mangate.org, ric.charles@u-mangate.com]
>> INFO  16:25:00,097 | org.apache.activemq.store.kahadb.MessageDatabase |
>> Slow KahaDB access: Journal append took: 989 ms, Index update took 0 ms
>> INFO  16:25:00,099 | james.mailetcontext | Storing mail
>> Mail1270218298023-9820-!226071 in file://var/mail/address-error/
>> INFO  16:25:07,699 | james.smtpserver | Successfully spooled mail
>> Mail1270218307697-9822 from oursu31@stahelin.com on 68.62.205.100 for
>> [ric.charles@u-mangate.be]
>> INFO  16:25:08,723 | james.mailetcontext | Storing mail
>> Mail1270218307697-9822-!70969 in file://var/mail/address-error/
>> INFO  16:25:09,908 | org.apache.activemq.store.kahadb.MessageDatabase |
>> Slow KahaDB access: Journal append took: 1184 ms, Index update took 0 ms
>> ERROR 16:25:35,144 | james.pop3server | WriteRequest holds not a an
>> valid Object but class java.lang.String
>> INFO  16:25:41,212 | org.apache.activemq.store.kahadb.MessageDatabase |
>> Slow KahaDB access: cleanup took 7274
>> INFO  16:25:55,612 | org.apache.activemq.store.kahadb.MessageDatabase |
>> Slow KahaDB access: cleanup took 1141
>> ERROR 16:26:42,059 | james.pop3server | WriteRequest holds not a an
>> valid Object but class java.lang.String
>> INFO  16:27:00,003 | org.apache.activemq.store.kahadb.MessageDatabase |
>> Slow KahaDB access: cleanup took 58100
>> INFO  16:27:00,003 | org.apache.activemq.store.kahadb.MessageDatabase |
>> Slow KahaDB access: Journal append took: 10032 ms, Index update took
>> 52100 ms
>> INFO  16:27:02,183 | james.smtpserver | Successfully spooled mail
>> Mail1270218355606-9824 from inexperienceMO@toutgratuit.com on
>> 213.163.116.78 for [angelo.biddau@u-mangate.net]
>> WARN  16:36:27,919 |
>> org.springframework.jms.listener.DefaultMessageListenerContainer | Setup
>> of JMS message listener invoker failed for destination
>> 'processor.local-address-error' - trying to recover. Cause: Java heap space
>> ERROR 16:43:58,281 | james.pop3server | WriteRequest holds not a an
>> valid Object but class java.lang.String
>> WARN  16:44:23,231 |
>> org.springframework.jms.listener.DefaultMessageListenerContainer | Setup
>> of JMS message listener invoker failed for destination 'processor.spam'
>> - trying to recover. Cause: Java heap space
>> ERROR 16:47:23,508 | james.pop3server | WriteRequest holds not a an
>> valid Object but class java.lang.String
>> ERROR 16:48:37,497 | james.pop3server | WriteRequest holds not a an
>> valid Object but class java.lang.String
>> ERROR 16:49:53,684 | james.pop3server | WriteRequest holds not a an
>> valid Object but class java.lang.String
>> ERROR 16:52:17,970 | org.apache.activemq.ActiveMQConnection | Caught an
>> exception trying to create a JMSException for
>> java.lang.OutOfMemoryError: Java heap space
>> java.lang.OutOfMemoryError: Java heap space
>> ERROR 16:58:05,554 | james.pop3server | WriteRequest holds not a an
>> valid Object but class java.lang.String
>> ERROR 17:04:31,427 | james.pop3server | WriteRequest holds not a an
>> valid Object but class java.lang.String
>> WARN  17:09:11,055 |
>> org.springframework.jms.listener.DefaultMessageListenerContainer | Setup
>> of JMS message listener invoker failed for destination
>> 'processor.local-address-error' - trying to recover. Cause: Java heap space
>> ERROR 17:09:35,017 |
>> org.apache.james.transport.camel.JMSSelectorPollingConsumer | Consumer
>> Consumer[pollingjms://queue?consumer.endpointUri=activemq%3Aqueue%3Aoutgoing.retry&delay=30000]
>> could not poll endpoint:
>> pollingjms://queue?consumer.endpointUri=activemq%3Aqueue%3Aoutgoing.retry&delay=30000
>> caused by: Java heap space
>> java.lang.OutOfMemoryError: Java heap space
>> WARN  17:09:55,361 |
>> org.springframework.jms.listener.DefaultMessageListenerContainer | Setup
>> of JMS message listener invoker failed for destination 'processor.spam'
>> - trying to recover. Cause: Java heap space
>> WARN  17:09:57,557 |
>> org.springframework.jms.listener.DefaultMessageListenerContainer | Setup
>> of JMS message listener invoker failed for destination 'processor.root'
>> - trying to recover. Cause: Java heap space
>> WARN  17:11:00,695 |
>> org.springframework.jms.listener.DefaultMessageListenerContainer | Setup
>> of JMS message listener invoker failed for destination
>> 'processor.local-address-error' - trying to recover. Cause: Java heap space
>> WARN  17:22:19,706 |
>> org.springframework.jms.listener.DefaultMessageListenerContainer | Setup
>> of JMS message listener invoker failed for destination
>> 'processor.bounces' - trying to recover. Cause: Java heap space
>> WARN  17:22:42,422 |
>> org.springframework.jms.listener.DefaultMessageListenerContainer | Setup
>> of JMS message listener invoker failed for destination
>> 'processor.transport' - trying to recover. Cause: Java heap space
>>
>>
>> On 04/02/2010 09:00 AM, Norman Maurer wrote:
>>> Hi Eric,
>>>
>>> this sounds promising.... Looking forward for your next report.
>>>
>>> Bye
>>> Norman
>>>
>>> 2010/4/2, Eric Charles<eric.charles@u-mangate.com>:
>>>
>>>> James survived the night.
>>>> 300MB (exactly the same as when it started).
>>>> No m64 file in tmp.
>>>>
>>>> I will post another update this evening.
>>>>
>>>> Tks,
>>>>
>>>> Eric
>>>>
>>>>
>>>> On 04/01/2010 07:06 PM, Norman Maurer wrote:
>>>>
>>>>> Yeah thats right.. just checout server/trunk.
>>>>>
>>>>> I'm looking forward to hear your feedback ;)
>>>>>
>>>>> Bye,
>>>>> Norman
>>>>>
>>>>>
>>>>> 2010/4/1 Eric Charles<eric.charles@u-mangate.com>:
>>>>>
>>>>>
>>>>>> Hello Norman,
>>>>>> Done :)
>>>>>> Tks, Eric
>>>>>>
>>>>>> PS1 : I didn't change the config, except for the JDCBDomainList I
am
>>>>>> using.
>>>>>> PS2 : I only checkout
>>>>>> http://svn.apache.org/repos/asf/james/server/trunk
>>>>>> and
>>>>>> not the other projects. From what I read from the poms, snapshot
>>>>>> version
>>>>>> of
>>>>>> imap-mailbox, jsieve,... are taken from the central maven repository
>>>>>> and
>>>>>> I
>>>>>> don't have to rebuild those projects.
>>>>>>
>>>>>>
>>>>>>
>>>>>>>    Hi Eric,
>>>>>>>
>>>>>>>    its me again ;) Could you please svn up to latest trunk
and try
>>>>>>> again
>>>>>>> ?
>>>>>>>
>>>>>>>    Thx,
>>>>>>>    Norman
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> 2010/4/1 Norman Maurer<norman.mau...@googlemail.com>:
>>>>>>
>>>>>>
>>>>>>>>    Hi Eric,
>>>>>>>>
>>>>>>>>    I did a stress test in the last few hours (running in
the
>>>>>>>> background
>>>>>>>>    here) and I was not able to get a OOM yet with
>>>>>>>>    "?broker.useJmx=false&amp;jms.prefetchPolicy.all=1"
for the
>>>>>>>>    jmsConnectionFactory. I sent 40000 emails with 100k
to it and 3000
>>>>>>>>    with 5 mb.
>>>>>>>>
>>>>>>>>    I'm using -Xmx512m and memory usage is not getting higher
then
>>>>>>>> 700m.
>>>>>>>>
>>>>>>>>    Bye.
>>>>>>>>    Norman
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>>>>>> For additional commands, e-mail: server-dev-help@james.apache.org
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>>>>> For additional commands, e-mail: server-dev-help@james.apache.org
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>>>> For additional commands, e-mail: server-dev-help@james.apache.org
>>>>
>>>>
>>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>>> For additional commands, e-mail: server-dev-help@james.apache.org
>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>> For additional commands, e-mail: server-dev-help@james.apache.org
>>
>>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org


Mime
View raw message