stratos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Akila Ravihansa Perera <raviha...@wso2.com>
Subject Re: Stratos creating thread pools per application?
Date Thu, 03 Dec 2015 05:41:22 GMT
Hi Isuru,

Big +1 for this effort. Shall we create a JIRA to track this and get it
fixed in next release?

I've noticed few other issues related to Stratos threading behavior.

 - New threads are created in certain places [1] which should be avoided as
much as possible.

@Reka: Any idea on why we create new Threads in
ClusterLevelPartitionContext at [1]? This would cause Stratos to go out of
resources at some point right?

 - New threads are created in component activators [2]. Although level of
impact is minimum since this is a one-time creating, we should be using the
thread pool allocated at all times.

 - Here is a code piece found in CC service component;

private void shutdownExecutorService(String executorServiceId) {
    ExecutorService executorService =
StratosThreadPool.getExecutorService(executorServiceId, 1);
    if (executorService != null) {
        shutdownExecutorService(executorService);
    }
}

private void shutdownExecutorService(ExecutorService executorService) {
try {
executorService.shutdownNow();
} catch (Exception e) {
log.warn("An error occurred while shutting down executor service", e);
}
}


It really doesn't make sense to get the thread pool object already created
by passing a thread pool size. We should have a fully flexible API in
StratosThreadPool class in Stratos common component to perform those
operations.

[1]
https://github.com/apache/stratos/blob/stratos-4.1.x/components/org.apache.stratos.autoscaler/src/main/java/org/apache/stratos/autoscaler/context/partition/ClusterLevelPartitionContext.java#L116-L121

[2]
https://github.com/apache/stratos/blob/stratos-4.1.x/components/org.apache.stratos.autoscaler/src/main/java/org/apache/stratos/autoscaler/internal/AutoscalerServiceComponent.java#L120


Thanks.

On Thu, Dec 3, 2015 at 10:20 AM, Isuru Haththotuwa <isuruh@apache.org>
wrote:

> Hi Devs,
>
> Currently what we have is a static set of thread pools:
>
>    1. monitor thread pool
>    2. autoscaler thread pool
>    3. cartridge agent event listener thread pool
>    4. cloud controller context thread pool
>    5. cloud controller thread pool
>    6. cloud controller instance manager thread pool
>    7. cloud controller stats publisher thread pool
>    8. autoscaler stats publisher thread pool
>    9. load balancer thread pool
>    10. mock iaas event listener thread pool
>    11. stratos manager thread pool
>
> This is in addition to the scheduled executor services we are using. Shall
> we optimize the StratosThreadPool implementation to dynamically shrink and
> grow when required? We are currently using fixed thread pools.
>
>
>
> On Sun, Nov 22, 2015 at 12:15 PM, Imesh Gunaratne <imesh@apache.org>
> wrote:
>
>> Hi Akila,
>>
>> On Sat, Nov 21, 2015 at 1:20 PM, Akila Ravihansa Perera <
>> ravihansa@wso2.com> wrote:
>>
>>>
>>> Any idea why thread pools are created per monitor object? Is it not
>>> possible to share a common thread pool as per the current design?
>>>
>>
>> No, we do not create a thread pool per monitor object, have a look at the
>> code carefully:
>>
>> executorService = StratosThreadPool.getExecutorService(
>>                 AutoscalerConstants.MONITOR_THREAD_POOL_ID,
>> threadPoolSize);
>>
>> The above statement ask Stratos Thread Pool class to return an executor
>> service with the name AutoscalerConstants.MONITOR_THREAD_POOL_ID:
>>
>> public static ExecutorService getExecutorService(String identifier, int
>> threadPoolSize) {
>>         ExecutorService executorService =
>> executorServiceMap.get(identifier);
>>         if (executorService == null) {
>>             synchronized (executorServiceMapLock) {
>>                 if (executorService == null) {
>>                     executorService =
>> Executors.newFixedThreadPool(threadPoolSize);
>>                     executorServiceMap.put(identifier, executorService);
>>                     log.info(String.format("Thread pool created: [type]
>> Executor Service [id] %s [size] %d", identifier, threadPoolSize));
>>                 }
>>             }
>>         }
>>         return executorService;
>>     }
>>
>> This will only create an executor service once for a given identifier.
>>
>> Thanks
>>
>>
>>>
>>> Stratos server thread count goes beyond 1500 when deploying 2 or 3 apps.
>>> And it keeps growing proportional to the deployed app count. This is a very
>>> high resource usage.
>>>
>>> [1]
>>> https://github.com/apache/stratos/blob/stratos-4.1.x/components/org.apache.stratos.autoscaler/src/main/java/org/apache/stratos/autoscaler/monitor/cluster/ClusterMonitor.java#L108
>>>
>>> [2]
>>> https://github.com/apache/stratos/blob/stratos-4.1.x/components/org.apache.stratos.autoscaler/src/main/java/org/apache/stratos/autoscaler/monitor/component/ApplicationMonitor.java#L82
>>>
>>> [3]
>>> https://github.com/apache/stratos/blob/stratos-4.1.x/components/org.apache.stratos.autoscaler/src/main/java/org/apache/stratos/autoscaler/monitor/component/GroupMonitor.java#L88
>>>
>>> [4]
>>> https://github.com/apache/stratos/blob/stratos-4.1.x/components/org.apache.stratos.autoscaler/src/main/java/org/apache/stratos/autoscaler/monitor/component/ParentComponentMonitor.java#L112
>>>
>>> Thanks.
>>>
>>> --
>>> Akila Ravihansa Perera
>>> WSO2 Inc.;  http://wso2.com/
>>>
>>> Blog: http://ravihansa3000.blogspot.com
>>>
>>
>>
>>
>> --
>> Imesh Gunaratne
>>
>> Senior Technical Lead, WSO2
>> Committer & PMC Member, Apache Stratos
>>
>> --
>> Thanks and Regards,
>>
>> Isuru H.
>> +94 716 358 048* <http://wso2.com/>*
>>
>>
>> * <http://wso2.com/>*
>>
>>
>>


-- 
Akila Ravihansa Perera
WSO2 Inc.;  http://wso2.com/

Blog: http://ravihansa3000.blogspot.com

Mime
View raw message