james-server-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Norman Maurer <norman.mau...@googlemail.com>
Subject Re: JAMES OOM, maybe the cause of MINA ?
Date Mon, 12 Apr 2010 07:45:12 GMT
Hi Ashish,

I bet its the loop I mention before.. This would make sense when
looking at the growing queued writes. I commited a change to trunk
which close the session after write the data to the client. We will
see if this helps..

Thx,
Norman



2010/4/12 Ashish <paliwalashish@gmail.com>:
> On Sun, Apr 11, 2010 at 8:57 PM, Norman Maurer
> <norman.maurer@googlemail.com> wrote:
>> Hi Ashish,
>>
>> after thinkin more about the whole stuff I wonder if the exception
>> handling in SMTP could sometimes be an endless loop. In the handler we
>> write a message back to the client on exception. But what happens if
>> this throw a new exception ? Could it so loop forever ?
>
> I am not sure, but it could be possible.
>
>>
>> http://svn.apache.org/viewvc/james/server/trunk/smtpserver/src/main/java/org/apache/james/smtpserver/mina/SMTPIoHandler.java?revision=930014&view=markup
>>
>> Maybe we should just close the connection..
>
> hmm.. its brute force. I am not convinced that we should do this, but
> not sure as I don't have a complete picture as such.
> If the exception is coz of packet decoding, ignore the packet and move
> on.. Its also driven by the protocol as such. You may need
> to respond back to the client with some error code etc.
>
> Eric: Please mark a copy to mina dev ML as well, else I don't get your mails :)
>
> The analysis is a session is in hanging state, whose queue is building
> up. Heap dump are not going to help any more.
> Can you trace down what the hack is going on with that session. I
> can't get much out of heap dump and unfortunately not able to run it
> on laptop.
> jconsole simply refuse to detect the James process :(
>
> So the key lies in analyzing
> 1. Why is this session hanging, its simply not writing the data back
> to the client. There is a while before the session is closed and
> session.isConnected() returns false.
>
> Eric: can you upload the logs as well, if possible :)
>
> Anyone else has some idea's here?
>
> thanks
> ashish
>
>>
>> Bye
>> Norman
>>
>> 2010/4/9 Eric Charles <eric.charles@u-mangate.com>:
>>> Hi Ashish,
>>>
>>> I am the user who has many OOM with the trunk and feed Norman with my issues
>>> :)
>>>
>>> 1.
>>> Indeed, you have the dump with CircularQueue from mina 2.0.0-RC1. This is
>>> the release we normally use in james and give the issue after a certain
>>> period of time. I mean, the server can stay working correctly 2 days or
>>> crash after 2 hours (see http://apache.u-mangate.com/james/oom/oom.png) due
>>> to a peak in memory usage. T
>>> To further investigate, we tried with mina trunk 2.0.0-RC2-SNAPSHOT : There,
>>> we see a direct crash.  I've uploaded such a dump on
>>> http://apache.u-mangate.com/james/oom/index.html (click on shift-f5 to force
>>> refresh, I setup the web server in a hurry).  You will see there recursive
>>> ConcurrentLinkedQueue.
>>>
>>> 2.
>>> As shown on http://apache.u-mangate.com/james/oom/oom.png, the system works
>>> while the line is flat (mail are received, spooled, delivered,...). On a few
>>> seconds, it peaks and of course, nothing works anymore.  So, the messages
>>> don't remain in the spool. I made tests with quite huge messages, and they
>>> are delivered very fast as soon as they arrive in James.
>>>
>>> Don't hesitate to ask more questions or propose additional tests.
>>>
>>> Many tks in advance,
>>>
>>> Eric
>>>
>>>
>>> On 04/09/2010 12:40 PM, Ashish wrote:
>>>>
>>>> Norman,
>>>>
>>>> Couple of more queries
>>>>
>>>> 1. The heap dump uses circularqueue class, so seem to be taken for an
>>>> earlier trunk snapshot. Is my take correct?
>>>>
>>>> 2. What's the state of the System? are the clients receiving the
>>>> messages. The queue seems to be holding a very large number of
>>>> objects.
>>>> Essentially what I want to know is, if the clients are receiving the
>>>> messages or the Server is holding them up.
>>>>
>>>> Will spend more time with the issue and see what I can figure out.
>>>>
>>>> thanks
>>>> ashish
>>>>
>>>> On Thu, Apr 8, 2010 at 2:03 PM, Norman Maurer
>>>> <norman.maurer@googlemail.com>  wrote:
>>>>
>>>>>
>>>>> Maybe Eric can do, cause he is the one who see it very freqently..
>>>>>
>>>>> So Eric...;) ?
>>>>>
>>>>> Thx,
>>>>> Norman
>>>>>
>>>>>
>>>>> 2010/4/8 Ashish<paliwalashish@gmail.com>:
>>>>>
>>>>>>
>>>>>> Can you provide the heapdump for this OOM?
>>>>>>
>>>>>> thanks
>>>>>> ashish
>>>>>>
>>>>>> On Thu, Apr 8, 2010 at 1:40 PM, Ashish<paliwalashish@gmail.com>
 wrote:
>>>>>>
>>>>>>>
>>>>>>> On Thu, Apr 8, 2010 at 1:28 PM, Norman Maurer
>>>>>>> <norman.maurer@googlemail.com>  wrote:
>>>>>>>
>>>>>>>>
>>>>>>>> Hi Ashish,
>>>>>>>>
>>>>>>>> I think we tracked down the source of the problem a bit more..
The OOM
>>>>>>>> seems to be related to IMAP. Our IMAP server component is
using the
>>>>>>>> StreamIoHandler (its the only one of our components who use
this
>>>>>>>> handler). So I suspect there is the problem.
>>>>>>>>
>>>>>>>>  So there are two possible problems:
>>>>>>>> 1) Bug in StreamIoHandler
>>>>>>>> 2) Wrong usage of StreamIoHandler. Our implementations is
here:
>>>>>>>>
>>>>>>>> http://svn.apache.org/viewvc/james/server/trunk/imapserver/src/main/java/org/apache/james/imapserver/mina/ImapIoHandler.java?view=markup
>>>>>>>>
>>>>>>>> Thx,
>>>>>>>> Norman
>>>>>>>>
>>>>>>>
>>>>>>> Sorry, haven't been able to look at this so far :(
>>>>>>> earliest I can give it a shot will be on Sunday.
>>>>>>>
>>>>>>> thanks
>>>>>>> ashish
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>>> For additional commands, e-mail: server-dev-help@james.apache.org
>>>
>>>
>>
>
>
>
> --
> thanks
> ashish
>
> Blog: http://www.ashishpaliwal.com/blog
> My Photo Galleries: http://www.pbase.com/ashishpaliwal
>

---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org


Mime
View raw message