mina-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Emmanuel L├ęcharny <elecha...@gmail.com>
Subject [MINA 3.0] Thoughts on TCP server
Date Mon, 01 Oct 2012 10:45:54 GMT
Hi guys, some thoughts about the TCP server. feel free to comment.

TCP server MINA 3

As we are reworking the server part of MINA, we can review the current 
architecture. There are a few problems we can address.

1) One vs Many selectors
In MINA 2, we do have at least 2 selectors used :
- one for the OP_ACCEPT event
- One or many for the OP_READ/OP_WRITE

I don't think that it makes a lot of sense to force such a choice. IMO, 
we could perfectly start with one single selector to handle all the 
events, assuming we are fast enough to process them. Otherwise, we can 
also configure the server to use more selectors, if needed.

Typically, the acceptor selector will just deal with incoming new 
connections, and the created channel will be registred on an IoProcessor 
selector on MINA 2. We could register this channel on the acceptor selector.

In any case, if we do create more than one selector, I suggest that the 
first one would always handle OP_ACCEPT events, and that's all.

The general algorithm will look likes :

signal started

while true
   nbSelect = select( delay )

   if nbSelect != 0
     for each selectionKey do
       case isAccept // Only if the selector is the first one, otherwise 
we don't need to heck this case
         create session

       case isRead
         process read

       case isWrite
         process write

   if dispose // In order to stop the server
     process dispose

The key is to start all the selector workers *before* accepting any 
connection, otherwise we may lose some messages. One more thing : each 
selector should signal that there have started before entering in the 
loop, so that the caller don't have to wait a random period of time for 
the selectors to be started.

2) Events processing
Now, in order to limit the number of selectors to use, we need to limit 
the time it takes to process the read/write/accept events. But even if 
we have many selectors, we should try to minimize the risk that one 
selector is blocked by a single session blocked somewhere while doing 
some heavy prcoessing, as it will block all the other sessions.

Having more than a selector is one way to mitigate this issue : as we 
have many threads (one per selector), we spread the loads on as many 
Another solution would be to use an executor in charge of processing the 
events, with a queue between the selector and the executor, queue that 
is used to process the events as fast as possible on the selector (this 
is really important for UDP, as we don't want to lose messages simply 
because the OS buffer is full).

The problem is that we are just not solving the problem of a rogue 
service that block a thread for a long time (if we use a limited size 
executor), or we may end with so many threads that it may kill the 
server. But anyway, it sounds like a better solution, as incoming events 
that won't require a long processing will be favored in the long term.

3) Write processing
This is a complex issue too : we may not be able to push all the data we 
want into the socket, if it becoms full (or was already full). In this 
case, we will have to store the data in a queue. The following algorithm 
describe this situation and a proposal to solve it

     if there are some data in the writeQueue
         // We need to enqueue the data, and write the head of the queue
         enqueue data

         // Now, we shoudl try to write as much of the queue as we can
         while ( queue not empty)
               poll the data from the queue
             nbWritten = channel.write( remaining data ) // We may have 
already written some part of the head data

             if nbWritten < data.remaining
                 // the socket is already full, set its OP_WRITE 
interest, and don't touch the queue
                 selectionKey.ops = OP_WRITE
                 break // We can stop, the socket is full anyway
                 pop the data from the queue // We just remove the head, 
and continue with the next data

           if queue is empry
               selectionKeys.ops = ! OP_WRITE // We remoe this flag, 
it's ueless now
         nbWritten = channel.write( remaining data ) // We may have 
already written some part of the head data

         if nbWritten < data.remaining
             // the socket is already full, set its OP_WRITE interest, 
and add the data in the queue
             selectionKey.ops = OP_WRITE
             writeQueue.add( remaining data )

4) Read/Write order
MINA 2 was processing all the reads first, then all the writes. This is 
not necessarilly a good idea. We may rather process read and write on a 
per session basis. It's perfectly possible that a session has to process 
some reads and some write at the same time. Waiting for all the reads to 
be processed create some memory load, as we will have to wait until all 
the reads are done, storing all the data to be written in the mean time, 
until we are done with the last session.

5) MINA events
Atm, we are processing the following events :
- messageReceived (when we have read some data from the channel)
- messageSent (when a user message has been sentcompletely)
- exceptionCaugth (when an exception has been caught)
- sessionIdle (if the session has not received or sent anything for a 
period of time)
- sessionCreated (when a new session is created)
- sessionOpened
- sessionClosed (when the session has been closed)
- filterWrite
- filterClose

The sessionOpened, filterWrite and filterClose are a bit mysterious. I 
don't see why we need a special event SessionOpened when we alreayd have 
the sessionCreated event. That may seems we *may* create a session, but 
not accept any incoming or outgoing messages for this session, until 
it's initialized. I'm not sure this is necessary.

The two other events are probably some YAGNY iplementation...

Emmanuel L├ęcharny

View raw message