mina-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Chalmers <chr...@ecis.co.za>
Subject Re: Blocking using Mina
Date Wed, 04 Jul 2007 08:38:46 GMT
Hi all

Checking this again, it seems that an OOM is being thrown after all. The 
IoFilter throttling doesn't seem to kick in until too late, unfortunately.

Looking at the SocketIoProcessor class, if I apply the throttle before 
the read call, this achieves the desired effect for me: (ie: limits 
memory usage significantly):

    private void process( Set<SelectionKey> selectedKeys )
              if (!throttle(session)) {
                read( session );

Unfortunately this solution is not clean as I am forced to modify the 
Mina source code and not use the API.
Any better ideas anyone?

Chris Chalmers wrote:
> Hi Peter
> I'm now getting something strange:
> - my read is throttling fine based on the number of outstanding writes 
> on a linked session
> - the session.resumeRead() method is being called when the linked 
> writes have dropped
> - however, the read is not resumed.
> I have limited memory usage to 64Mb, and at the point of session 
> resumption, 63.8Mb has been read in.
> When using 256Mb, the process completes; at the point of resume, it 
> seems like the allocated memory doubles - I am hazarding a guess that 
> this could be causing the problem when using 64Mb. However, I don't 
> get an out-of-memory exception.
> Any ideas?
> Thanks,
> Chris
> peter royal wrote:
>> On Jun 28, 2007, at 8:00 AM, Chris Chalmers wrote:
>>> I have a Mina server that simply pumps data through (based on the 
>>> Mina Proxy example) -
>>> however, when sending large files (500Mb+), the data is read in 
>>> quickly, but written slowly (the target uses a blocking Socket 
>>> read); this causes the server's memory usage to increase very quickly.
>>> I have tried to use the ReadThrottleFilterBuilder in combination 
>>> with the WriteBufferLimitFilterBuilder (from 
>>> http://issues.apache.org/jira/browse/DIRMINA-302), but the best I 
>>> can get is 226Mb memory usage when setting the read/write 
>>> setMaximumConnectionBufferSize to 50000.
>>> Is there any way that I can force Mina sessions to block incoming 
>>> messages instead? The source code explicitly sets the SocketChannels 
>>> to non-blocking and I can't see any way to change this using the API.
>>> Alternatively is there a better method that I am overlooking?
>> IoSession.suspendRead / resumeRead ... you can use that to keep MINA 
>> from reading the data, just turn it on/off in concert with your writing.
>> -pete

View raw message