stratos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Isuru Haththotuwa <isu...@apache.org>
Subject Re: Stratos creating thread pools per application?
Date Thu, 03 Dec 2015 07:24:47 GMT
Created [1] to track the improvements, will do the changes for the new
release.

[1]. https://issues.apache.org/jira/browse/STRATOS-1639

On Thu, Dec 3, 2015 at 12:38 PM, Reka Thirunavukkarasu <reka@wso2.com>
wrote:

> Hi Akila,
>
> On Thu, Dec 3, 2015 at 11:11 AM, Akila Ravihansa Perera <
> ravihansa@wso2.com> wrote:
>
>> Hi Isuru,
>>
>> Big +1 for this effort. Shall we create a JIRA to track this and get it
>> fixed in next release?
>>
>> I've noticed few other issues related to Stratos threading behavior.
>>
>>  - New threads are created in certain places [1] which should be avoided
>> as much as possible.
>>
>> @Reka: Any idea on why we create new Threads in
>> ClusterLevelPartitionContext at [1]? This would cause Stratos to go out of
>> resources at some point right?
>>
>
> I think this had been added to maintain few tasks for pending member
> watcher, obsolete member watcher and etc in the
> ClusterLevelPartitionContext. Those can be converted to scheduled tasks
> with executor service.
>
>>
>>  - New threads are created in component activators [2]. Although level of
>> impact is minimum since this is a one-time creating, we should be using the
>> thread pool allocated at all times.
>>
>>  - Here is a code piece found in CC service component;
>>
>> private void shutdownExecutorService(String executorServiceId) {
>>     ExecutorService executorService = StratosThreadPool.getExecutorService(executorServiceId,
1);
>>     if (executorService != null) {
>>         shutdownExecutorService(executorService);
>>     }
>> }
>>
>> private void shutdownExecutorService(ExecutorService executorService) {
>> try {
>> executorService.shutdownNow();
>> } catch (Exception e) {
>> log.warn("An error occurred while shutting down executor service", e);
>> }
>> }
>>
>>
>> It really doesn't make sense to get the thread pool object already
>> created by passing a thread pool size. We should have a fully flexible API
>> in StratosThreadPool class in Stratos common component to perform those
>> operations.
>>
>> [1]
>> https://github.com/apache/stratos/blob/stratos-4.1.x/components/org.apache.stratos.autoscaler/src/main/java/org/apache/stratos/autoscaler/context/partition/ClusterLevelPartitionContext.java#L116-L121
>>
>> [2]
>> https://github.com/apache/stratos/blob/stratos-4.1.x/components/org.apache.stratos.autoscaler/src/main/java/org/apache/stratos/autoscaler/internal/AutoscalerServiceComponent.java#L120
>>
>>
>> Thanks.
>>
>> On Thu, Dec 3, 2015 at 10:20 AM, Isuru Haththotuwa <isuruh@apache.org>
>> wrote:
>>
>>> Hi Devs,
>>>
>>> Currently what we have is a static set of thread pools:
>>>
>>>    1. monitor thread pool
>>>    2. autoscaler thread pool
>>>    3. cartridge agent event listener thread pool
>>>    4. cloud controller context thread pool
>>>    5. cloud controller thread pool
>>>    6. cloud controller instance manager thread pool
>>>    7. cloud controller stats publisher thread pool
>>>    8. autoscaler stats publisher thread pool
>>>    9. load balancer thread pool
>>>    10. mock iaas event listener thread pool
>>>    11. stratos manager thread pool
>>>
>>> This is in addition to the scheduled executor services we are using.
>>> Shall we optimize the StratosThreadPool implementation to dynamically
>>> shrink and grow when required? We are currently using fixed thread pools.
>>>
>>>
>>>
>>> On Sun, Nov 22, 2015 at 12:15 PM, Imesh Gunaratne <imesh@apache.org>
>>> wrote:
>>>
>>>> Hi Akila,
>>>>
>>>> On Sat, Nov 21, 2015 at 1:20 PM, Akila Ravihansa Perera <
>>>> ravihansa@wso2.com> wrote:
>>>>
>>>>>
>>>>> Any idea why thread pools are created per monitor object? Is it not
>>>>> possible to share a common thread pool as per the current design?
>>>>>
>>>>
>>>> No, we do not create a thread pool per monitor object, have a look at
>>>> the code carefully:
>>>>
>>>> executorService = StratosThreadPool.getExecutorService(
>>>>                 AutoscalerConstants.MONITOR_THREAD_POOL_ID,
>>>> threadPoolSize);
>>>>
>>>> The above statement ask Stratos Thread Pool class to return an executor
>>>> service with the name AutoscalerConstants.MONITOR_THREAD_POOL_ID:
>>>>
>>>> public static ExecutorService getExecutorService(String identifier, int
>>>> threadPoolSize) {
>>>>         ExecutorService executorService =
>>>> executorServiceMap.get(identifier);
>>>>         if (executorService == null) {
>>>>             synchronized (executorServiceMapLock) {
>>>>                 if (executorService == null) {
>>>>                     executorService =
>>>> Executors.newFixedThreadPool(threadPoolSize);
>>>>                     executorServiceMap.put(identifier, executorService);
>>>>                     log.info(String.format("Thread pool created:
>>>> [type] Executor Service [id] %s [size] %d", identifier, threadPoolSize));
>>>>                 }
>>>>             }
>>>>         }
>>>>         return executorService;
>>>>     }
>>>>
>>>> This will only create an executor service once for a given identifier.
>>>>
>>>> Thanks
>>>>
>>>>
>>>>>
>>>>> Stratos server thread count goes beyond 1500 when deploying 2 or 3
>>>>> apps. And it keeps growing proportional to the deployed app count. This
is
>>>>> a very high resource usage.
>>>>>
>>>>> [1]
>>>>> https://github.com/apache/stratos/blob/stratos-4.1.x/components/org.apache.stratos.autoscaler/src/main/java/org/apache/stratos/autoscaler/monitor/cluster/ClusterMonitor.java#L108
>>>>>
>>>>> [2]
>>>>> https://github.com/apache/stratos/blob/stratos-4.1.x/components/org.apache.stratos.autoscaler/src/main/java/org/apache/stratos/autoscaler/monitor/component/ApplicationMonitor.java#L82
>>>>>
>>>>> [3]
>>>>> https://github.com/apache/stratos/blob/stratos-4.1.x/components/org.apache.stratos.autoscaler/src/main/java/org/apache/stratos/autoscaler/monitor/component/GroupMonitor.java#L88
>>>>>
>>>>> [4]
>>>>> https://github.com/apache/stratos/blob/stratos-4.1.x/components/org.apache.stratos.autoscaler/src/main/java/org/apache/stratos/autoscaler/monitor/component/ParentComponentMonitor.java#L112
>>>>>
>>>>> Thanks.
>>>>>
>>>>> --
>>>>> Akila Ravihansa Perera
>>>>> WSO2 Inc.;  http://wso2.com/
>>>>>
>>>>> Blog: http://ravihansa3000.blogspot.com
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Imesh Gunaratne
>>>>
>>>> Senior Technical Lead, WSO2
>>>> Committer & PMC Member, Apache Stratos
>>>>
>>>> --
>>>> Thanks and Regards,
>>>>
>>>> Isuru H.
>>>> +94 716 358 048* <http://wso2.com/>*
>>>>
>>>>
>>>> * <http://wso2.com/>*
>>>>
>>>>
>>>>
>>
>>
>> --
>> Akila Ravihansa Perera
>> WSO2 Inc.;  http://wso2.com/
>>
>> Blog: http://ravihansa3000.blogspot.com
>>
>
>
>
> --
> Reka Thirunavukkarasu
> Senior Software Engineer,
> WSO2, Inc.:http://wso2.com,
> Mobile: +94776442007
>
> --
> <%2B94776442007>
> <%2B94776442007>
> Thanks and Regards,
>
> Isuru H.
> <%2B94776442007>
> +94 716 358 048 <%2B94776442007>* <http://wso2.com/>*
>
>
> * <http://wso2.com/>*
>
>
>

Mime
View raw message