mina-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Maria Petridean (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DIRMINA-1039) Response messages queue up on the server side waiting to be written to socket, while the server continues to read more request messages, causing out of heap memory
Date Wed, 06 Jul 2016 14:05:10 GMT

    [ https://issues.apache.org/jira/browse/DIRMINA-1039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15364348#comment-15364348
] 

Maria Petridean commented on DIRMINA-1039:
------------------------------------------

This issue was not caused by a slow-reading client. The client is able to both read and write
messages fast. This issue addresses a problem on the server side, after the changes in mina
2. With mina 1, testing the server in the exact same conditions behaves correctly.

When writing responses, if we assume the response is small in size and it is followed by a
(empty message) marker in the queue (which is part of mina code). When writing, in AbstractPollingIoProcessor
flushNow method, there is a check that (localWrittenBytes == 0), which holds true for the
marker-WriteRequests. The comment in the code at this point, suggests that this is interpreted
as kernel buffer being full, which is false. This check is true because we just processed
a message marker. The return statement in this case will cause exiting the write-loop and
starting to read messages again. It seems to me that this check here should be (localWrittenBytes
== 0 && isNotAnEmptyMessage). And also add another check below: if ((localWrittenBytes
== 0 && isAnEmptyMessage), then simply set the session as interested in write and
continue the write cycle. The loop will then exit the write cycle when will be the case (when
the kernel buffer is full, or when the max written bytes is reached, etc).
Otherwise, by design, mina allows reading multiple messages during one read-phase (which depends
on the buffer size, message size) and then only allows writing one response during the write-phase
and return to read-phase. It seems to me this can be fixed (I'm sure my suggestion above is
not the only solution). I understand that this will not fix also the problem you [~elecharny]
mentioned - which has to be handled at application implementation level. 

I am not sure the title of this issue is the best to describe this issue, this is why i tried
to detail in this comment the problem and the possible fix. What do you think?

> Response messages queue up on the server side waiting to be written to socket, while
the server continues to read more request messages, causing out of heap memory
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: DIRMINA-1039
>                 URL: https://issues.apache.org/jira/browse/DIRMINA-1039
>             Project: MINA
>          Issue Type: Bug
>          Components: Core
>            Reporter: Maria Petridean
>   Original Estimate: 2h
>  Remaining Estimate: 2h
>
> One case in which this bug reproduces is by using a client which generates a heavy request-load.
The mina thread which processes both reads and writes - exits the write cycle after processing
every empty marker (the WriteRequest which wraps an empty buffer, acting as a message marker).
This will result in the thread resuming the read cycle, hence reading more client request
messages. After a few minutes, the number of read messages is much larger than the number
of written response messages, even though the responses are waiting in the queue, ready to
be written to socket.
> To solve this, the sever shouldn't exit the write cycle after processing every marker
WriteRequest. This way the ratio between the read and written messages will be more balanced;
this will avoid the heap memory getting full and causing server degradation.
> Also, an improvement can be considered here to avoid using the same single thread for
both reads and writes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message