mina-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gaston Dombiak" <gas...@jivesoftware.com>
Subject RE: Throughput monitoring issues.
Date Mon, 29 Jan 2007 18:33:45 GMT
Hey guys,

That is exactly what I needed for out load testing analysis. We needed to be able to get the
total numbers instead of per connection number.

The nice thing of the current StatCollector implementation is that if you are not interested
in stats then there is no overhead since there is no polling. However, having a poll mechanism
makes the overall solution more complex compared to the option of just changing the sessions
to add up their stats to the service stats.

"It'll improve performance a for people who just want to see the service throughtput.". I'm
not sure about performance but for sure it will help to get a more accurate stat of a snapshot
in a given point in time. Anyway, I think that the margin of error is acceptable when using
the polling method. We were testing with 30K sessions and I agree that iterating on them is
not as iterating on 10 elements. :) Having said that, I like the idea of just adding up stats
in the service object based on session notifications. However, we need to be very careful
to monitor possible overhead of this option. My guess is that the overhead (that will exist)
will be practically minimal thus acceptable.

BTW, I tried to find Julien's post about his feedback regarding the suggested fix/improvement
but it seems that I lost it. :( However, I do remember that he mentioned something about that
rates are no longer supported but numbers. The reason for such modification were 2: 1) Rates
can be calculated based on the total number and the frequency of getting those numbers and
2) in our case seeing the total number (instead of rate) was more meaningful. :)


  -- Gato

-----Original Message-----
From: Trustin Lee [mailto:trustin@gmail.com] 
Sent: Saturday, January 27, 2007 9:32 PM
To: dev@mina.apache.org; jvermillard@archean.fr
Subject: Re: Throughput monitoring issues.

On 1/24/07, Julien Vermillard <jvermillard@archean.fr> wrote:
> Le mercredi 24 janvier 2007 à 20:59 +0900, Trustin Lee a écrit :
> > Hi folks,
> >
> > 1) We need 'built-in per-service performance counter in IoService'.
> >
> > In the recent talk with Julien about throughput monitoring, we realized
> that
> > we need to reapproach to the throughput monitoring issue from a
> different
> > point of view.
> >
> > Until now, we focused on providing per-session throughput.  IoSession
> > provided appropriate performance counters to calculate per-session
> > throughput, but IoService didn't.  But what would an administrator want
> to
> > see from his management console?  Julien and I think it's not throughput
> for
> > an individual session but throughput for the service he or she is
> managing.
> Actualy you can see the two, the per service stats  are in the
> StatColector, and IoSession stats attached to them. Basicly per service
> stats are sum of active session stats, plus addition of the closed
> sessions in the last polling period.

True.  What I wanted to say is that we need to *focus* on per-service
statistics rather than per-session statistics.  Per-service statistics can
be calculated more easily and it's not for now.  We need to fix it in 2.0.

> For now, I/O threads doesn't provide any performance counter from the
> > service standpoint, so we have a lot of difficulties in calculating
> > meaningful performance factors without slowing down a network
> application.
> > Of course, we still need to monitor an individual session's performance
> > counters, but not usually for most cases, especially for stateless
> protocols
> > like HTTP.
> Computing the per service, from collected read/write totals in IoService
> that will be much faster and won't need to compute all the per session
> stats. I think the per session throughput stats needs to be enabled
> manualy by the user, if needed.

I can't agree more, either.

> 2) Gaston's patch
> >
> > Gaston's patch (https://issues.apache.org/jira/browse/DIRMINA-332)
> contains
> > several interesting ideas.  We could probably borrow his idea that uses
> a
> > queue to reduce synchronization in 1.0.x.  Other stuff would be
> incorporate
> > with the 'built-in per-service performance counter in IoService' because
> > there's no backward compatibility in the patch.
> >
> > I must admit that we have design flaw in per-service performance counter
> > calculation, and it is hard to fix it without breaking backward
> > compatibility or API addition.  Resolving this issue in trunk (
> > 2.0-M1-SNAPSHOT) would be the better option, although it breaks my
> heart.
> > :(
> For 1.X we need to fix synchro issues. for 2.X we need to change the API
> for being able to compute per service stats without needing to poll all
> the IoSessions. It'll improve performance a for people who just want to
> see the service throughtput.

Good.  What do you think about it, Gaston?

what we call human nature is actually human habit
PGP key fingerprints:
* E167 E6AF E73A CBCE EE41  4A29 544D DE48 FE95 4E7E
* B693 628E 6047 4F8F CFA4  455E 1C62 A7DC 0255 ECA6

View raw message