mina-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jhcha <jh...@kds.co.kr>
Subject sessionClosed is not called sometimes in heavy stress test contions.
Date Mon, 04 Feb 2008 10:51:39 GMT

Dear Mina developers.
I'm trying to use mina 1.1.5 to develop the very big stress server.
I test my server with jmeter like some echo test now.

test environment

* jmeter :
echo message size : 1024 bytes
jmeter threads : 100
loop count : 1000
sampler : TCP Sampler
reuse connection : false

* server : HP-UX 2 cores
         java HotSpot(TM) Server VM (build 1.5.0.07 jinteg:03.20.07-11:05
IA64, mixed mode) 
         my server(mina)
         
* client : pc (dual core).
         jmeter 

when I tested the stress, I found sessionCreated called counts are a little
bigger than sessionClosed called counts.
so, I tried to find out why that mismatch occured during several days.
and I found the followings :


org.apache.mina.common.support.IoServiceListenerSupport
{

public void fireSessionCreated(IoSession session) {
		SocketAddress serviceAddress = session.getServiceAddress();

		// Get the session set.
		Set<IoSession> s = new IdentityHashSet<IoSession>();
		Set<IoSession> sessions = managedSessions.putIfAbsent(serviceAddress,
Collections
				.synchronizedSet(s));
		boolean firstSession;

		if (null == sessions) {
			sessions = s;
			firstSession = true;
		} else {
			firstSession = false;
		}

		// If already registered, ignore.
		if (!sessions.add(session)) {
			return;
		}

		// If the first connector session, fire a virtual service activation
		// event.
		if (session.getService() instanceof IoConnector && firstSession) {
			fireServiceActivated(session.getService(), session.getServiceAddress(),
session
					.getHandler(), session.getServiceConfig());
		}

		// Fire session events.
		session.getFilterChain().fireSessionCreated(session);  <-- called
correctly
		session.getFilterChain().fireSessionOpened(session);  <-- called correctly

		// Fire listener events.
		for (IoServiceListener listener : listeners) {
			listener.sessionCreated(session);  <-- called correctly
		}
	}


public void fireSessionDestroyed(IoSession session) {
		boolean lastSession = false;
		SocketAddress serviceAddress = session.getServiceAddress();

		// Get the session set.
		Set<IoSession> sessions = managedSessions.get(serviceAddress);
		// Ignore if unknown.
		if (sessions == null) {
			// logger.error("fireSessionDestroyed() : sessions == null < " +
session);
			return;  <-- occured sometimes !!!!!!!!!!!!!!!!!!
		} 

        sessions.remove(session);

		// Try to remove the remaining empty session set after removal.
		if (sessions.isEmpty()) {
			lastSession = managedSessions.remove(serviceAddress, sessions);
		}

		// Fire session events.
		session.getFilterChain().fireSessionClosed(session); <-- skipped sometimes

		// Fire listener events.
		try {
			for (IoServiceListener listener : listeners) {
				listener.sessionDestroyed(session); <-- skipped sometimes
			}
		} finally {
			// Fire a virtual service deactivation event for the last session of
			// the connector.
			// TODO double-check that this is *STILL* the last session. May not
			// be the case
			if (session.getService() instanceof IoConnector && lastSession) {
				fireServiceDeactivated(session.getService(),
session.getServiceAddress(), session
						.getHandler(), session.getServiceConfig()); <-- skipped sometimes
			}
		}
	}

the above fireSessionDestroyed method called as the same number as
fireSessionCreated method.

but the mysterious event is that the sessions is not gotten from
managedSessions in fireSessionDestroyed,
i.e. SocketAddress key is not found in managedSessions.
then the remaining logic was skipped.
so fireSessionClosed, sessionDestroyed ... are not called sometimes.

[IoServiceListenerSupport] - fireSessionDestroyed() : sessions == null <
(SOCKET, R: /172.26.73.107:15059, L: /10.88.50.205:30000, S:
0.0.0.0/0.0.0.0:30000)
[IoServiceListenerSupport] - fireSessionDestroyed() : sessions == null <
(SOCKET, R: /172.26.73.107:15961, L: /10.88.50.205:30000, S:
0.0.0.0/0.0.0.0:30000)
[IoServiceListenerSupport] - fireSessionDestroyed() : sessions == null <
(SOCKET, R: /172.26.73.107:23883, L: /10.88.50.205:30000, S:
0.0.0.0/0.0.0.0:30000)
[IoServiceListenerSupport] - fireSessionDestroyed() : sessions == null <
(SOCKET, R: /172.26.73.107:24036, L: /10.88.50.205:30000, S:
0.0.0.0/0.0.0.0:30000)
[IoServiceListenerSupport] - fireSessionDestroyed() : sessions == null <
(SOCKET, R: /172.26.73.107:31341, L: /10.88.50.205:30000, S:
0.0.0.0/0.0.0.0:30000)
[IoServiceListenerSupport] - fireSessionDestroyed() : sessions == null <
(SOCKET, R: /172.26.73.107:31343, L: /10.88.50.205:30000, S:
0.0.0.0/0.0.0.0:30000)
[IoServiceListenerSupport] - fireSessionDestroyed() : sessions == null <
(SOCKET, R: /172.26.73.107:31345, L: /10.88.50.205:30000, S:
0.0.0.0/0.0.0.0:30000)
[IoServiceListenerSupport] - fireSessionDestroyed() : sessions == null <
(SOCKET, R: /172.26.73.107:45839, L: /10.88.50.205:30000, S:
0.0.0.0/0.0.0.0:30000)
[IoServiceListenerSupport] - fireSessionDestroyed() : sessions == null <
(SOCKET, R: /172.26.73.107:5317, L: /10.88.50.205:30000, S:
0.0.0.0/0.0.0.0:30000)
[IoServiceListenerSupport] - fireSessionDestroyed() : sessions == null <
(SOCKET, R: /172.26.73.107:8471, L: /10.88.50.205:30000, S:
0.0.0.0/0.0.0.0:30000)

I think the reason that the 100 client threads tries to proceed "conect,
send, receive, close",
different sessions has the same client SocketAddress at times in server,
and the origin of that problem is client's stress environment.

It is possible the following senario : 

I/O processor thread     I/O processor thread      client thread  client
thread
one                      two                       one            two
--------------------------------------------------------------------------------
                                                   1) connect(the same
client SocketAddress)
2)fireSessionCreated
3)putIfAbsent
                                                   4)disconnect
                                                                 
5)connect(the same client SocketAddress)
                          6)fireSessionCreated
                          7)putIfAbsent
8)fireSessionDestroyed
9)remove
                                                                 
10)disconnect
                          11)fireSessionDestroyed
                          12)get fail



I changed the mina source IoServiceListenerSupport.java

org.apache.mina.common.support.IoServiceListenerSupport
...

public void fireSessionDestroyed(IoSession session) {
		boolean lastSession = false;
		SocketAddress serviceAddress = session.getServiceAddress();

		// Get the session set.
		Set<IoSession> sessions = managedSessions.get(serviceAddress);
		// Ignore if unknown.
		if (sessions == null) {
			// return;  <-- delete
		} else {       // <-- add

			sessions.remove(session);

			// Try to remove the remaining empty session set after removal.
			if (sessions.isEmpty()) {
				lastSession = managedSessions.remove(serviceAddress, sessions);
			}
		}              // <-- add
		....

then the Create, Destroy call mismatch was cleared.
but I don't know this is correct solution.

Would you tell me this is correct or not?

I think this situation is very rare but... 
I hope the mina is the best network framework 
and this situation shoud be also considered.


Thank you

J. H. Cha
-- 
View this message in context: http://www.nabble.com/sessionClosed-is-not-called-sometimes-in-heavy-stress-test-contions.-tp15265671s16868p15265671.html
Sent from the Apache MINA Support Forum mailing list archive at Nabble.com.


Mime
View raw message