james-server-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ashish <paliwalash...@gmail.com>
Subject Re: JAMES OOM, maybe the cause of MINA ?
Date Mon, 12 Apr 2010 07:42:09 GMT
On Sun, Apr 11, 2010 at 8:57 PM, Norman Maurer
<norman.maurer@googlemail.com> wrote:
> Hi Ashish,
>
> after thinkin more about the whole stuff I wonder if the exception
> handling in SMTP could sometimes be an endless loop. In the handler we
> write a message back to the client on exception. But what happens if
> this throw a new exception ? Could it so loop forever ?

I am not sure, but it could be possible.

>
> http://svn.apache.org/viewvc/james/server/trunk/smtpserver/src/main/java/org/apache/james/smtpserver/mina/SMTPIoHandler.java?revision=930014&view=markup
>
> Maybe we should just close the connection..

hmm.. its brute force. I am not convinced that we should do this, but
not sure as I don't have a complete picture as such.
If the exception is coz of packet decoding, ignore the packet and move
on.. Its also driven by the protocol as such. You may need
to respond back to the client with some error code etc.

Eric: Please mark a copy to mina dev ML as well, else I don't get your mails :)

The analysis is a session is in hanging state, whose queue is building
up. Heap dump are not going to help any more.
Can you trace down what the hack is going on with that session. I
can't get much out of heap dump and unfortunately not able to run it
on laptop.
jconsole simply refuse to detect the James process :(

So the key lies in analyzing
1. Why is this session hanging, its simply not writing the data back
to the client. There is a while before the session is closed and
session.isConnected() returns false.

Eric: can you upload the logs as well, if possible :)

Anyone else has some idea's here?

thanks
ashish

>
> Bye
> Norman
>
> 2010/4/9 Eric Charles <eric.charles@u-mangate.com>:
>> Hi Ashish,
>>
>> I am the user who has many OOM with the trunk and feed Norman with my issues
>> :)
>>
>> 1.
>> Indeed, you have the dump with CircularQueue from mina 2.0.0-RC1. This is
>> the release we normally use in james and give the issue after a certain
>> period of time. I mean, the server can stay working correctly 2 days or
>> crash after 2 hours (see http://apache.u-mangate.com/james/oom/oom.png) due
>> to a peak in memory usage. T
>> To further investigate, we tried with mina trunk 2.0.0-RC2-SNAPSHOT : There,
>> we see a direct crash.  I've uploaded such a dump on
>> http://apache.u-mangate.com/james/oom/index.html (click on shift-f5 to force
>> refresh, I setup the web server in a hurry).  You will see there recursive
>> ConcurrentLinkedQueue.
>>
>> 2.
>> As shown on http://apache.u-mangate.com/james/oom/oom.png, the system works
>> while the line is flat (mail are received, spooled, delivered,...). On a few
>> seconds, it peaks and of course, nothing works anymore.  So, the messages
>> don't remain in the spool. I made tests with quite huge messages, and they
>> are delivered very fast as soon as they arrive in James.
>>
>> Don't hesitate to ask more questions or propose additional tests.
>>
>> Many tks in advance,
>>
>> Eric
>>
>>
>> On 04/09/2010 12:40 PM, Ashish wrote:
>>>
>>> Norman,
>>>
>>> Couple of more queries
>>>
>>> 1. The heap dump uses circularqueue class, so seem to be taken for an
>>> earlier trunk snapshot. Is my take correct?
>>>
>>> 2. What's the state of the System? are the clients receiving the
>>> messages. The queue seems to be holding a very large number of
>>> objects.
>>> Essentially what I want to know is, if the clients are receiving the
>>> messages or the Server is holding them up.
>>>
>>> Will spend more time with the issue and see what I can figure out.
>>>
>>> thanks
>>> ashish
>>>
>>> On Thu, Apr 8, 2010 at 2:03 PM, Norman Maurer
>>> <norman.maurer@googlemail.com>  wrote:
>>>
>>>>
>>>> Maybe Eric can do, cause he is the one who see it very freqently..
>>>>
>>>> So Eric...;) ?
>>>>
>>>> Thx,
>>>> Norman
>>>>
>>>>
>>>> 2010/4/8 Ashish<paliwalashish@gmail.com>:
>>>>
>>>>>
>>>>> Can you provide the heapdump for this OOM?
>>>>>
>>>>> thanks
>>>>> ashish
>>>>>
>>>>> On Thu, Apr 8, 2010 at 1:40 PM, Ashish<paliwalashish@gmail.com>
 wrote:
>>>>>
>>>>>>
>>>>>> On Thu, Apr 8, 2010 at 1:28 PM, Norman Maurer
>>>>>> <norman.maurer@googlemail.com>  wrote:
>>>>>>
>>>>>>>
>>>>>>> Hi Ashish,
>>>>>>>
>>>>>>> I think we tracked down the source of the problem a bit more..
The OOM
>>>>>>> seems to be related to IMAP. Our IMAP server component is using
the
>>>>>>> StreamIoHandler (its the only one of our components who use this
>>>>>>> handler). So I suspect there is the problem.
>>>>>>>
>>>>>>>  So there are two possible problems:
>>>>>>> 1) Bug in StreamIoHandler
>>>>>>> 2) Wrong usage of StreamIoHandler. Our implementations is here:
>>>>>>>
>>>>>>> http://svn.apache.org/viewvc/james/server/trunk/imapserver/src/main/java/org/apache/james/imapserver/mina/ImapIoHandler.java?view=markup
>>>>>>>
>>>>>>> Thx,
>>>>>>> Norman
>>>>>>>
>>>>>>
>>>>>> Sorry, haven't been able to look at this so far :(
>>>>>> earliest I can give it a shot will be on Sunday.
>>>>>>
>>>>>> thanks
>>>>>> ashish
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>> For additional commands, e-mail: server-dev-help@james.apache.org
>>
>>
>



-- 
thanks
ashish

Blog: http://www.ashishpaliwal.com/blog
My Photo Galleries: http://www.pbase.com/ashishpaliwal

---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org


Mime
View raw message