mina-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Trustin Lee" <trus...@gmail.com>
Subject Re: OutOfMemory when using ReadThrottleFilter
Date Wed, 31 Jan 2007 01:59:17 GMT
Hi Swen,

On 1/31/07, Swen Moczarski <mail@swenmoc.de> wrote:
>
> Hi Trustin,
>
> thanks a lot for your response.
>
> When I'm using the remaining() calls, I still get an out-of-memory error.
> But when I'm using capacity() it looks ok.
> Maybe, this becomes only a problem when a lot of small messages are stored
> in bigger ByteBuffers. But I think it would
> be safer to prevent overloading when the ReadThrottleFilter tracks the
> memory allocated by the ByteBuffers (capacity())
> instead of the received bytes. WDYT?


The root cause of this problem is that the default receive buffer size is
too big for smaller sockets.  MINA allocates a buffer whose size is same
with the default receive buffer size because it doesn't know how large chunk
of data will come in.  I didn't thought the default receive buffer size is
big, but it is turned out to be bigger than 40k.  This means you allocate
40k buffer even if the size of the packet you exchange is much smaller than
that.  In this case, using remaining() might cause unexpected
OutOfMemoryError.  Configuring the size of the receive buffer with the
following code will do the trick:

      ((SocketSessionConfig) acceptor.getDefaultConfig
().getSessionConfig()).setReceiveBufferSize(512);

If we replace remaining() with capacity(), we won't need to do that, but we
will observe unexpected slowdown in low throughput communication.

Here is my simple stess-test:
>
> public class Test {
>
>     private final static int PORT = 8056;
>
>     public static void main(String[] args) throws Exception {
>        IoHandler handler = new IoHandlerAdapter() {
>
>           public void exceptionCaught(IoSession session, Throwable cause)
> throws Exception {
>              cause.printStackTrace();
>           }
>           public void sessionCreated(IoSession session) throws Exception {
>              ReadThrottleFilterBuilder throttleFilter = new
> ReadThrottleFilterBuilder();
>              throttleFilter.attach(session.getFilterChain());
>           }
>        };
>        SocketAcceptor acceptor = new SocketAcceptor();
>        acceptor.bind(new InetSocketAddress(PORT), handler);
>
>        Socket socket = new Socket("localhost", PORT);
>        Writer writer = new OutputStreamWriter(socket.getOutputStream());
>
>        while (true) {
>           writer.write("test");
>           writer.flush();
>        }
>     }
> }
>
> I have tried the test with version 1.0.1 and the current trunk (with the
> current trunk it takes longer until the memory
> error occures).
> Should I attach the test to
> https://issues.apache.org/jira/browse/DIRMINA-338 and reopen the issue?


It takes very long for me to reproduce the problem.  I guess you can get rid
of this problem trying these options:

1) Increase max total direct buffer size using -XX:MaxDirectMemorySizeoption (
e.g. -XX:MaxDirectMemorySize=128M)
2) Change the allocator to SimpleByteBufferAllocator (i.e.
ByteBuffer.setAllocator(new SimpleByteBufferAllocator());)
3) Allocate heap buffers instead of direct buffers (i.e.
ByteBuffer.setUseDirectBuffers(false);)

Please try each step not reverting back the previous step.  I believe
OutOfMemoryError will be gone once you reach to the step 3, but please let
me know the result.

HTH,
Trustin
-- 
what we call human nature is actually human habit
--
http://gleamynode.net/
--
PGP key fingerprints:
* E167 E6AF E73A CBCE EE41  4A29 544D DE48 FE95 4E7E
* B693 628E 6047 4F8F CFA4  455E 1C62 A7DC 0255 ECA6

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message