incubator-s4-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthieu Morel <mmo...@apache.org>
Subject Re: throttling vs blocking
Date Wed, 17 Jul 2013 16:12:29 GMT

On Jul 17, 2013, at 17:16 , Jiaan Zeng <l.allen09@gmail.com> wrote:

> Hi,
> 
> 1)
> I wonder what is the main difference between
> org.apache.s4.comm.staging.ThrottlingThreadPoolExecutorService and
> org.apache.s4.comm.staging.BlockingThreadPoolExecutorService
> 
> I understand ThrottlingThreadPoolExecutorService uses RateLimiter to
> control the rate while BlockingThreadPoolExecutorService uses
> Semaphore. I guess ThrottlingThreadPoolExecutorService is supposed to
> drop events while BlockingThreadPoolExecutorService is supposed to
> block until thread pool can handle events.
> 
> However, in ThrottlingThreadPoolExecutorService -> submit(Runnable
> task), rateLimitedPermits.acquire() is called which means the
> invocation is blocked until permit is acquired. It seems to me that
> ThrottlingThreadPoolExecutorService never drops events and works in
> the same way as BlockingThreadPoolExecutorService does. And there is
> no unit test for event drop on this.

For throttling, trying to acquire faster than the max rate will fill up the work queue of
the thread pool and new tasks will be dropped (rejected by the executor). 
For blocking, we control the work queue directly through semaphores, and that's blocking.

> 
> 2)
> Too what is the the main difference between
> org.apache.s4.core.staging.LoadSheddingSenderExecutorServiceFactory
> and org.apache.s4.core.staging.ThrottlingSenderExecutorServiceFactory
> 
> They both claim they will drop event.

Both drop events if downstream consumers are too slow.
In addition, throttling drops events when tasks arrive faster than a specified rate.


Regards,

Matthieu



> 
> 
> Thanks.
> 
> --
> Regards,
> Jiaan


Mime
View raw message