mina-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eero Nevalainen <eero.nevalai...@indagon.com>
Subject Re: Session close problem
Date Mon, 28 May 2007 14:40:12 GMT
Response inline.

Rob Butler wrote:
> Wow Eero!
> Thanks for taking the time to determine what the problem was and
 > report it back to the group.
> So in summary (to make sure I understand this correctly)
> 1) If you send data too rapidly for the device to download from the
> GPRS (exceed it's bandwidth), then all data will queue up there
 > instead of it trickling down until caught up.  If the remote end
 > sends any message the GPRS send queue is unblocked and data is
 > sent to the remote device.  Is this correct?

Seems I should have stated my tests a bit more clearly.
I have NOT tested exceeding the bandwidth. I've only tested how the 
network behaves when the connection is idle for a long time. All the 
messages I sent were very small. For some reason our network "blocks" 
one-directionally, when there's no communication.

> 2) If you have a delay of more than 50 seconds between messages sent to the remote device
they will queue up at the GPRS until the remote end sends a message and the GPRS send queue
is unblocked.  Is this correct?


> Which cellular network were your tests done on?   Cingular/AT&T, Sprint/Nextel, Verizon,
T-Mobile, something else?
> Do you have the ability to repeat these tests with another cellular network?  It would
be interesting to see if these numbers and particular problem exist only with one network
or if it is a common one.
> In addition to the tests you've already run, could you do a few more?  Some thing's I'd
be interested in knowing is:

I don't want to name our network provider because we don't want to look 
like we are blaming them(We're actually quite happy with them). We also 
don't have SIMs from other providers so I can't repeat the tests on 
another network.

As for the suggested additional tests, I'm afraid I have to direct my 
time to more  urgent matters. Maybe someone else can pickup from here.

-Eero Nevalainen

> ----- Original Message ----
> From: Eero Nevalainen <eero.nevalainen@indagon.com>
> To: dev@mina.apache.org
> Cc: Rob Butler <crodster2k@yahoo.com>
> Sent: Monday, May 28, 2007 4:05:53 AM
> Subject: Re: Session close problem
> I've now finished my tests which revealed the problem. Took a while 
> because I had to redo the last test due to lack of materials..:)
> A description of the problem scenario is on my earlier message included 
> further below.
> Tests, their results and conclusions (immediately) below:
> 1. Test - Replicate the problem
> I simulated the gateway by listening with netcat. At first I sent 'PING' 
> messages rapidly and the connection worked fine. Then I waited for three 
> minutes and tried again. The connection wasn't working.
> - The problem can be replicated with other tools and is related to the 
> activity of communication.
> 2. Test - Measure delay threshold
> On the second setup I wrote a fake GW which sends PINGs with increasing 
> delays: 5s, 6s, 7s, ...
> At a certain threshold the remote device no longer replied PONG. Before 
> the threshold all PINGs received a response. On three measurements the 
> thresholds were:
> 48s
> 49s
> 48s
> 3. Test - Figure characteristic behaviour
> This time I used a remote device on our lab, which had an ethernet 
> connection in addition to the GPRS. I used netcat on both ends to send 
> chosen traffic activity over the GPRS. The following behavioural rules 
> were obtained:
> - messages from the remote device always arrive properly
> - if the gateway sends messages thicker than the ~50s threshold, the 
> messages are received and continue to be received normally.
> - if the gateway exceeds the threshold in it's communication, the 
> messages are queued. A message sent by the remote device flushes the queue.
> 4. Test - Test radio blackout scenario
> This test was like the 3rd test, except I yanked out the GPRS antenna. I 
>   had to redo this one because the tiny antenna connector was actually 
> providing enough of antenna for GPRS to work! Wrapping the device in 
> aluminum foil proved effective though :) The following was observed:
> - if the remote device has lost it's connection for more than 50s the 
> connection will become 'locked' even if the gateway sends messages 
> during that time. A message from the remote device flushes the queue as 
> usual.
> A coworker has heard this is a known problem with the Gateway GPRS 
> Support Node (GGSN). It will probably be fixed at some point, but 
> currently the only safe workaround is to send messages from the remote end.
> Hope this is useful.
> -Eero Nevalainen
> Rob Butler wrote:
>> Eero,
>> I came across this (http://www.cl.cam.ac.uk/~rc277/globe02.pdf).
>> It would appear that a cellular device could have issue with receiving data because
it temporarily goes out of range or has some other issue which causes packets to "back up"
at the GPRS device.  From the above document it looks like it could take up to a minute before
that backlog is cleared up.  Have you tried waiting around for a while to see if the ping
packet eventually makes it to the device?  If it does make it "eventually" you could decrease
your ping send interval while simultaneously increasing the time you wait for a ping response.
 Unfortunately this means your connection could go down for a little while before you notice
it, but at least you'd have something that lets you know the connection is bad eventually.
>> Another consideration may be the infamous naggle algorithm.  You can of course turn
it off on your server application and on your client as well.  Unfortunately you have no control
on whether the GPRS is using naggle for it's connection to the cellular device.  Have you
tried sending a lot of ping packets?  This is of course wasteful, but if you send approximately
1500 bytes worth of ping data and miraculously they all arrive at the device at once naggle,
or something equally nefarious, is probably the cause.
>> Be aware that 1500 bytes or so is the usual Maximum Transmission Unit on most LAN's
by default, but this can be changed easily and may very likely be different on the cellular
network.  I'd try sending multiple ping packets slowly (like 1 every 30 seconds to one minute)
and see how many you have sent before the first one arrives.  When the first one arrives you'll
probably get a bunch of them together at once.  Try to count how many arrive at once (or in
very rapid succession) and then try sending that many ping packets + 1 more all at once from
your server.  If your rapid fire of multiple pings works more reliably one solution may simply
be to send artifically large ping packets.  By fine tuning the packet size of your ping message
you may be able to more reliably circumvent the "grouping" of naggle and similar approaches.
 Whatever you find is the "sweet spot" for packet size you may want to increase it slightly
so on other networks with potentially larger MTU sizes your

>>  application still works.  Again, this is all wasteful from a bandwidth perspective
- especially when end-users are charged per MB of data transfer on some cell networks, but
at least it would be a working solution.
>> Please report your findings back to the list, or at least to me.  I'd be very interested
in the results.
>> Thanks!
>> Rob
>> ----- Original Message ----
>> From: Eero Nevalainen <eero.nevalainen@indagon.com>
>> To: dev@mina.apache.org
>> Sent: Friday, May 11, 2007 7:30:32 AM
>> Subject: Re: Session close problem
>> Hi all,
>> I've recently deployed a gateway server to mediate communication between 
>>   clients and remote devices. The gateway works fine in a 'regular' 
>> network, but now we're experiencing problems when the remote devices are 
>>   behind a GPRS network.
>> The gateway sends an application level (TCP) 'PING' message periodically 
>> to the remote devices. For some reason the PING messages seem to queue 
>> up in the GPRS connection, without the devices receiving them.
>> Meanwhile, sending an ICMP request to the same devices gives a much 
>> smaller roundtrip time (~700ms) and no message losses. Also the messages 
>> sent by the remote devices to the gateway always arrive properly.
>> I can't reproduce this problem with an ethernet connection so any help 
>> with the GPRS network would be most welcome!
>> -Eero Nevalainen
>> Dawie Malan wrote:
>>> You're quite right - there is something in the middle, and it does cause the
>>> symptoms you are reporting.
>>> A GPRS network has two essential elements:
>>> *Serving GPRS Support Node (SGSN)-Sends data to and receives data from mobile
>>> stations, and maintains information about the location of a mobile station (MS).
>>> The SGSN communicates between the MS and the GGSN.
>>> *Gateway GPRS Support Node (GGSN)-A wireless gateway that allows mobile cell
>>> phone users to access the public data network (PDN) or specified private IP
>>> networks.
>>> The GGSN acts like a proxy and is the device that sends the ACKs to the phone
>>> even though the session is already closed on the server side.
>>> I hope this answers your question.
>>> Regards,
>>> Dawie Malan
>>> -----Original Message-----
>>> From: Michael Bauroth [mailto:michael.bauroth@falcom.de] 
>>> Sent: 22 March 2007 04:40 PM
>>> To: dev@mina.apache.org
>>> Subject: Session close problem
>>> Hi,
>>> I'm using currently the trunk version of Mina. I use it to communicate 
>>> with GPRS devices on the other side. In most cases this works very well!
>>> But sometimes the things become strange. I've collected a bunch of 
>>> problems where I would need help / advices / ideas.
>>> 1. I call session.close() from inside my server. It seems, that the 
>>> session doesn't close as required.
>>> 2. I close the session. The GPRS device on the other side continues to 
>>> send data and seems to get ACKs too. It doesn't recognize, that the 
>>> other side of the connection (the server) has closed the session a while 
>>> before. It seems that there is something in the middle which responds 
>>> the ACKs even with not opened server session.
>>> Knows anybody one of the two (most painful) problems and has an answer 
>>> for me?
>>> Best Regards
>>> Michael
>> ____________________________________________________________________________________
>> Never miss an email again!
>> Yahoo! Toolbar alerts you the instant new Mail arrives.
>> http://tools.search.yahoo.com/toolbar/features/mail/
> ____________________________________________________________________________________Get
the free Yahoo! toolbar and rest assured with the added security of spyware protection.
> http://new.toolbar.yahoo.com/toolbar/features/norton/index.php

View raw message