james-server-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Norman Maurer <norman.mau...@googlemail.com>
Subject Re: OutOfMemory exception
Date Tue, 06 Apr 2010 17:46:03 GMT
Hi Eric,

I was refering to expunge while using imap...

Bye,
Norman

2010/4/6 Eric Charles <eric.charles@u-mangate.com>:
> Hi Norman,
>
> Strange, POP3 is now working correctly.
> Regarding expunge, It is disabled by default on thunderbird.
> If I enable it, it seems to remove all mails from the folder, even if I
> don't delete them.
> Remembering the qmail/courrier/... server I was running a few years ago, I
> had a transparent behaviour between pop/imap.
> But I have to take more time to make some tests.
>
> Tks,
> Eric
>
> On 04/06/2010 07:09 PM, Norman Maurer wrote:
>>
>> Hi Eric,
>>
>> I could only of one change which could maybe improve things and so you
>> don't see the OOM anymore. The patch introduced in revision r930727.
>> This patch changed the behavoir of pop3servers RETR and TOP handler to
>> not load the email in the memory while return it to the client.
>>
>> Do you know which revision you used before ?
>>
>>
>> Some more info would be interesting. Could you tell me after which
>> POP3 command the Mailformed Packets appeared ?
>>
>>
>> Thx,
>> Norman
>>
>> 2010/4/6 Eric Charles<eric.charles@u-mangate.com>:
>>
>>>
>>> Hi Norman,
>>>
>>> james revision 930727 (mina RC1, jdk 1.6.0_18) up-and-running since 1,5
>>> day
>>> in prod.
>>>
>>> jmap -dump:file=... (this morning and right now) shows
>>> org.apache.derby.impl.services.cache.Clock with 15% of Heap, less for the
>>> rest.
>>>
>>> However, I had to redefine all accounts as IMAP in thunderbird to be able
>>> to
>>> read mails:
>>> - the pop3 was working fine, and at a time-being, it hanged.
>>> - Since there, all pop3 requests hang (thunderbird gives me a timeout
>>> after
>>> 2 minutes), simply saying "downloading message 1 of ..." (nothing in the
>>> log
>>> files)
>>> - wireshark shows a correct conversation at the beginning, and afterwards
>>> many "Malformed Packets".
>>> Maybe I could restart james, but I prefer let him work to have a better
>>> view
>>> on the OOM.
>>>
>>> Tks,
>>>
>>> Eric
>>>
>>>
>>> On 04/02/2010 09:04 PM, Norman Maurer wrote:
>>>
>>>>
>>>> Hi Eric,
>>>>
>>>> I forgot to mention that one cause of the OOM could be the current
>>>> usage of InMemoryMail. At the moment James load the whole MimeMessage
>>>> into memory while processing the camel routes. This is really a bad
>>>> practice in terms of performance and could maybe cause a OOM if you
>>>> processing many big messages which are bigger (the sumary) as the
>>>> avaible memory.
>>>> Could this be the problem on your side ?
>>>>
>>>> I'm currently workin on storing the MimeMessage on an external storage
>>>> and just stream in on demand.. So only the "meta-data" is stored in
>>>> jms and the rest in a db / jcr etc.. I have a proof-of-concept workin
>>>> here. But I need a bit more time to see if its the way to go..
>>>>
>>>> I'm also evolating the use of BlobMessages with ActiveMQ.
>>>>
>>>> Bye,
>>>> Norman
>>>>
>>>>
>>>> 2010/4/2 Norman Maurer<norman@apache.org>:
>>>>
>>>>
>>>>>
>>>>> Hi Eric
>>>>>
>>>>> so at least the files not leak anymore, unfortunaly this was the less
>>>>> critical problem.
>>>>> About the errors in pop3server I will have a look and see what could
>>>>> cause this. I'm 100 % sure it's not related to the OOM.
>>>>>
>>>>> Any more infos related to the OOM are welcome.
>>>>>
>>>>> Bye
>>>>> Norman
>>>>>
>>>>> 2010/4/2, Eric Charles<eric.charles@u-mangate.com>:
>>>>>
>>>>>
>>>>>>
>>>>>> nop. OutOfMemoryError is back.
>>>>>>
>>>>>> I will double check my server, jdk and environment config, but other
>>>>>> java processes are running on the same machine without problem.
>>>>>> I will also have a stress scenario on my local dev PC that runs fine
>>>>>> and
>>>>>> after, I will rerun it on my production server.
>>>>>> Dump analysis and full profiling are also to do.
>>>>>> Some stuff are difficult to simulate : See for example (occurs
>>>>>> regulary)
>>>>>> ERROR 16:47:23,508 | james.pop3server | WriteRequest holds not a
an
>>>>>> valid Object but class java.lang.String
>>>>>>
>>>>>> There are no more .m64 file in /tmp
>>>>>>
>>>>>> Tks,
>>>>>> Eric
>>>>>>
>>>>>>
>>>>>> ERROR 16:24:53,636 | james.pop3server | WriteRequest holds not a
an
>>>>>> valid Object but class java.lang.String
>>>>>> INFO  16:24:58,061 | james.smtpserver | Successfully spooled mail
>>>>>> Mail1270218298023-9820 from jewishnessesyh2@ecs-solar.com on
>>>>>> 68.62.205.100 for [ric.charles@u-mangate.org,
>>>>>> ric.charles@u-mangate.com]
>>>>>> INFO  16:25:00,097 | org.apache.activemq.store.kahadb.MessageDatabase
>>>>>> |
>>>>>> Slow KahaDB access: Journal append took: 989 ms, Index update took
0
>>>>>> ms
>>>>>> INFO  16:25:00,099 | james.mailetcontext | Storing mail
>>>>>> Mail1270218298023-9820-!226071 in file://var/mail/address-error/
>>>>>> INFO  16:25:07,699 | james.smtpserver | Successfully spooled mail
>>>>>> Mail1270218307697-9822 from oursu31@stahelin.com on 68.62.205.100
for
>>>>>> [ric.charles@u-mangate.be]
>>>>>> INFO  16:25:08,723 | james.mailetcontext | Storing mail
>>>>>> Mail1270218307697-9822-!70969 in file://var/mail/address-error/
>>>>>> INFO  16:25:09,908 | org.apache.activemq.store.kahadb.MessageDatabase
>>>>>> |
>>>>>> Slow KahaDB access: Journal append took: 1184 ms, Index update took
0
>>>>>> ms
>>>>>> ERROR 16:25:35,144 | james.pop3server | WriteRequest holds not a
an
>>>>>> valid Object but class java.lang.String
>>>>>> INFO  16:25:41,212 | org.apache.activemq.store.kahadb.MessageDatabase
>>>>>> |
>>>>>> Slow KahaDB access: cleanup took 7274
>>>>>> INFO  16:25:55,612 | org.apache.activemq.store.kahadb.MessageDatabase
>>>>>> |
>>>>>> Slow KahaDB access: cleanup took 1141
>>>>>> ERROR 16:26:42,059 | james.pop3server | WriteRequest holds not a
an
>>>>>> valid Object but class java.lang.String
>>>>>> INFO  16:27:00,003 | org.apache.activemq.store.kahadb.MessageDatabase
>>>>>> |
>>>>>> Slow KahaDB access: cleanup took 58100
>>>>>> INFO  16:27:00,003 | org.apache.activemq.store.kahadb.MessageDatabase
>>>>>> |
>>>>>> Slow KahaDB access: Journal append took: 10032 ms, Index update took
>>>>>> 52100 ms
>>>>>> INFO  16:27:02,183 | james.smtpserver | Successfully spooled mail
>>>>>> Mail1270218355606-9824 from inexperienceMO@toutgratuit.com on
>>>>>> 213.163.116.78 for [angelo.biddau@u-mangate.net]
>>>>>> WARN  16:36:27,919 |
>>>>>> org.springframework.jms.listener.DefaultMessageListenerContainer
|
>>>>>> Setup
>>>>>> of JMS message listener invoker failed for destination
>>>>>> 'processor.local-address-error' - trying to recover. Cause: Java
heap
>>>>>> space
>>>>>> ERROR 16:43:58,281 | james.pop3server | WriteRequest holds not a
an
>>>>>> valid Object but class java.lang.String
>>>>>> WARN  16:44:23,231 |
>>>>>> org.springframework.jms.listener.DefaultMessageListenerContainer
|
>>>>>> Setup
>>>>>> of JMS message listener invoker failed for destination
>>>>>> 'processor.spam'
>>>>>> - trying to recover. Cause: Java heap space
>>>>>> ERROR 16:47:23,508 | james.pop3server | WriteRequest holds not a
an
>>>>>> valid Object but class java.lang.String
>>>>>> ERROR 16:48:37,497 | james.pop3server | WriteRequest holds not a
an
>>>>>> valid Object but class java.lang.String
>>>>>> ERROR 16:49:53,684 | james.pop3server | WriteRequest holds not a
an
>>>>>> valid Object but class java.lang.String
>>>>>> ERROR 16:52:17,970 | org.apache.activemq.ActiveMQConnection | Caught
>>>>>> an
>>>>>> exception trying to create a JMSException for
>>>>>> java.lang.OutOfMemoryError: Java heap space
>>>>>> java.lang.OutOfMemoryError: Java heap space
>>>>>> ERROR 16:58:05,554 | james.pop3server | WriteRequest holds not a
an
>>>>>> valid Object but class java.lang.String
>>>>>> ERROR 17:04:31,427 | james.pop3server | WriteRequest holds not a
an
>>>>>> valid Object but class java.lang.String
>>>>>> WARN  17:09:11,055 |
>>>>>> org.springframework.jms.listener.DefaultMessageListenerContainer
|
>>>>>> Setup
>>>>>> of JMS message listener invoker failed for destination
>>>>>> 'processor.local-address-error' - trying to recover. Cause: Java
heap
>>>>>> space
>>>>>> ERROR 17:09:35,017 |
>>>>>> org.apache.james.transport.camel.JMSSelectorPollingConsumer | Consumer
>>>>>>
>>>>>>
>>>>>> Consumer[pollingjms://queue?consumer.endpointUri=activemq%3Aqueue%3Aoutgoing.retry&delay=30000]
>>>>>> could not poll endpoint:
>>>>>>
>>>>>>
>>>>>> pollingjms://queue?consumer.endpointUri=activemq%3Aqueue%3Aoutgoing.retry&delay=30000
>>>>>> caused by: Java heap space
>>>>>> java.lang.OutOfMemoryError: Java heap space
>>>>>> WARN  17:09:55,361 |
>>>>>> org.springframework.jms.listener.DefaultMessageListenerContainer
|
>>>>>> Setup
>>>>>> of JMS message listener invoker failed for destination
>>>>>> 'processor.spam'
>>>>>> - trying to recover. Cause: Java heap space
>>>>>> WARN  17:09:57,557 |
>>>>>> org.springframework.jms.listener.DefaultMessageListenerContainer
|
>>>>>> Setup
>>>>>> of JMS message listener invoker failed for destination
>>>>>> 'processor.root'
>>>>>> - trying to recover. Cause: Java heap space
>>>>>> WARN  17:11:00,695 |
>>>>>> org.springframework.jms.listener.DefaultMessageListenerContainer
|
>>>>>> Setup
>>>>>> of JMS message listener invoker failed for destination
>>>>>> 'processor.local-address-error' - trying to recover. Cause: Java
heap
>>>>>> space
>>>>>> WARN  17:22:19,706 |
>>>>>> org.springframework.jms.listener.DefaultMessageListenerContainer
|
>>>>>> Setup
>>>>>> of JMS message listener invoker failed for destination
>>>>>> 'processor.bounces' - trying to recover. Cause: Java heap space
>>>>>> WARN  17:22:42,422 |
>>>>>> org.springframework.jms.listener.DefaultMessageListenerContainer
|
>>>>>> Setup
>>>>>> of JMS message listener invoker failed for destination
>>>>>> 'processor.transport' - trying to recover. Cause: Java heap space
>>>>>>
>>>>>>
>>>>>> On 04/02/2010 09:00 AM, Norman Maurer wrote:
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> Hi Eric,
>>>>>>>
>>>>>>> this sounds promising.... Looking forward for your next report.
>>>>>>>
>>>>>>> Bye
>>>>>>> Norman
>>>>>>>
>>>>>>> 2010/4/2, Eric Charles<eric.charles@u-mangate.com>:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> James survived the night.
>>>>>>>> 300MB (exactly the same as when it started).
>>>>>>>> No m64 file in tmp.
>>>>>>>>
>>>>>>>> I will post another update this evening.
>>>>>>>>
>>>>>>>> Tks,
>>>>>>>>
>>>>>>>> Eric
>>>>>>>>
>>>>>>>>
>>>>>>>> On 04/01/2010 07:06 PM, Norman Maurer wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Yeah thats right.. just checout server/trunk.
>>>>>>>>>
>>>>>>>>> I'm looking forward to hear your feedback ;)
>>>>>>>>>
>>>>>>>>> Bye,
>>>>>>>>> Norman
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 2010/4/1 Eric Charles<eric.charles@u-mangate.com>:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Hello Norman,
>>>>>>>>>> Done :)
>>>>>>>>>> Tks, Eric
>>>>>>>>>>
>>>>>>>>>> PS1 : I didn't change the config, except for the
JDCBDomainList I
>>>>>>>>>> am
>>>>>>>>>> using.
>>>>>>>>>> PS2 : I only checkout
>>>>>>>>>> http://svn.apache.org/repos/asf/james/server/trunk
>>>>>>>>>> and
>>>>>>>>>> not the other projects. From what I read from the
poms, snapshot
>>>>>>>>>> version
>>>>>>>>>> of
>>>>>>>>>> imap-mailbox, jsieve,... are taken from the central
maven
>>>>>>>>>> repository
>>>>>>>>>> and
>>>>>>>>>> I
>>>>>>>>>> don't have to rebuild those projects.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>    Hi Eric,
>>>>>>>>>>>
>>>>>>>>>>>    its me again ;) Could you please svn up
to latest trunk and
>>>>>>>>>>> try
>>>>>>>>>>> again
>>>>>>>>>>> ?
>>>>>>>>>>>
>>>>>>>>>>>    Thx,
>>>>>>>>>>>    Norman
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 2010/4/1 Norman Maurer<norman.mau...@googlemail.com>:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>    Hi Eric,
>>>>>>>>>>>>
>>>>>>>>>>>>    I did a stress test in the last few
hours (running in the
>>>>>>>>>>>> background
>>>>>>>>>>>>    here) and I was not able to get a OOM
yet with
>>>>>>>>>>>>    "?broker.useJmx=false&amp;jms.prefetchPolicy.all=1"
for the
>>>>>>>>>>>>    jmsConnectionFactory. I sent 40000 emails
with 100k to it and
>>>>>>>>>>>> 3000
>>>>>>>>>>>>    with 5 mb.
>>>>>>>>>>>>
>>>>>>>>>>>>    I'm using -Xmx512m and memory usage
is not getting higher
>>>>>>>>>>>> then
>>>>>>>>>>>> 700m.
>>>>>>>>>>>>
>>>>>>>>>>>>    Bye.
>>>>>>>>>>>>    Norman
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>>>>>>>>>> For additional commands, e-mail: server-dev-help@james.apache.org
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>>>>>>>>> For additional commands, e-mail: server-dev-help@james.apache.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>>>>>>>> For additional commands, e-mail: server-dev-help@james.apache.org
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>>>>>>> For additional commands, e-mail: server-dev-help@james.apache.org
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>>>>>> For additional commands, e-mail: server-dev-help@james.apache.org
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>>>> For additional commands, e-mail: server-dev-help@james.apache.org
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>>> For additional commands, e-mail: server-dev-help@james.apache.org
>>>
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>> For additional commands, e-mail: server-dev-help@james.apache.org
>>
>>
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
> For additional commands, e-mail: server-dev-help@james.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org


Mime
View raw message