stratos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Imesh Gunaratne <im...@apache.org>
Subject Re: Stratos creating thread pools per application?
Date Fri, 08 Jan 2016 15:25:00 GMT
It's good to see the summary of the thread pools Isuru. Will add this to
the Wiki.

Thanks

On Fri, Jan 8, 2016 at 11:14 AM, Thanuja Uruththirakodeeswaran <
thanujau@wso2.com> wrote:

>
>
> On Fri, Jan 8, 2016 at 11:12 AM, Isuru Haththotuwa <isuruh@apache.org>
> wrote:
>
>> This is a short summary on Thread Pools used by Stratos atm:
>>
>> IdClassUsageConfigurable?Default Sizemonitor.thread.pool
>> ApplicationMonitor
>> ClusterMonitor
>> GroupMonitormonitoring tasks (Cluster, Application and Group)
>> Yes100autoscaler.thread.poolAutoscalerServiceComponentany general
>> autoscaler controller related work
>> (currently used for coordinator election and
>> Application monitor adder)Yes10cloud.controller.thread.pool
>> CloudControllerServiceComponentany general cloud controller related work
>> (currently used for coordinator election)Yes10
>> cloud.controller.instance.manager.threadCloudControllerServiceImplinstance
>> creation
>> instance terminationYes50cloud.controller.stats.publisher.thread.poolDASMemberInformationPublisher,
>>
>> DASMemberStatusPublisherinformation publishing to Monitoring Server
>> No10autoscaler.stats.publisher.thread.poolDASScalingDecisionPublisherinformation
>> publishing to Monitoring ServerNo10load.balancer.thread.pool
>> LoadBalancerStatisticsExecutorLB statistics publishingNo20
>> stratos-event-receiver-poolStratosEventReceiverevent receiving in
>> messaging component
>> Yes25stratos.manager.thread.poolStratosManagerServiceComponentany
>> general SM related work
>> (currently used for coordinator election)Yes10
>> The default thread counts are fine for basic scenarios. However, need to
>> configure the thread counts in cases where a large number of applications
>> are deployed and used.
>>
>> Here, cloud.controller.stats.publisher.thread.pool,
>> autoscaler.stats.publisher.thread.pool and load.balancer.thread.pool is
>> currently not configurable. Since the usage of threads in these pools will
>> change with the number of Members in the Topology, IMHO we need to make
>> them configurable. @Thanjua: WDYT?
>>
>
> +1 for make them configurable.
>
> Thanks.
>
>>
>> On Wed, Jan 6, 2016 at 7:08 AM, Imesh Gunaratne <imesh@apache.org> wrote:
>>
>>> Great work Isuru! It might be better to document the list of thread
>>> pools, their usage and how to tune them.
>>>
>>> Thanks
>>>
>>>
>>> On Tuesday, January 5, 2016, Isuru Haththotuwa <isuruh@apache.org>
>>> wrote:
>>>
>>>> The following are the list of improvements done:
>>>>
>>>>    - make the threads pools dynamic by allowing to grow/shrink when
>>>>    required
>>>>    - determining the min threads that a thread pool should have based
>>>>    on a ratio (configurable) between max and min number of threads
>>>>    - using a custom thread factory (extending from
>>>>    java.util.concurrent.ThreadFactory)
>>>>    - give a proper name to created threads based on the context
>>>>    (useful for identifying the thread and its origin for debugging purposes)
>>>>    - adding graceful thread pool termination support
>>>>    - using scheduled thread pools per Monitor than using a static
>>>>    scheduled thread pool for all monitors
>>>>
>>>> Will commit the changes.
>>>>
>>>> On Thu, Dec 3, 2015 at 5:41 PM, Imesh Gunaratne <imesh@apache.org>
>>>> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Thu, Dec 3, 2015 at 10:20 AM, Isuru Haththotuwa <isuruh@apache.org>
>>>>> wrote:
>>>>>>
>>>>>> This is in addition to the scheduled executor services we are using.
>>>>>> Shall we optimize the StratosThreadPool implementation to dynamically
>>>>>> shrink and grow when required? We are currently using fixed thread
pools.
>>>>>>
>>>>> +1
>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Sun, Nov 22, 2015 at 12:15 PM, Imesh Gunaratne <imesh@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Akila,
>>>>>>>
>>>>>>> On Sat, Nov 21, 2015 at 1:20 PM, Akila Ravihansa Perera <
>>>>>>> ravihansa@wso2.com> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>> Any idea why thread pools are created per monitor object?
Is it not
>>>>>>>> possible to share a common thread pool as per the current
design?
>>>>>>>>
>>>>>>>
>>>>>>> No, we do not create a thread pool per monitor object, have a
look
>>>>>>> at the code carefully:
>>>>>>>
>>>>>>> executorService = StratosThreadPool.getExecutorService(
>>>>>>>                 AutoscalerConstants.MONITOR_THREAD_POOL_ID,
>>>>>>> threadPoolSize);
>>>>>>>
>>>>>>> The above statement ask Stratos Thread Pool class to return an
>>>>>>> executor service with the name AutoscalerConstants.MONITOR_THREAD_POOL_ID:
>>>>>>>
>>>>>>> public static ExecutorService getExecutorService(String identifier,
>>>>>>> int threadPoolSize) {
>>>>>>>         ExecutorService executorService =
>>>>>>> executorServiceMap.get(identifier);
>>>>>>>         if (executorService == null) {
>>>>>>>             synchronized (executorServiceMapLock) {
>>>>>>>                 if (executorService == null) {
>>>>>>>                     executorService =
>>>>>>> Executors.newFixedThreadPool(threadPoolSize);
>>>>>>>                     executorServiceMap.put(identifier,
>>>>>>> executorService);
>>>>>>>                     log.info(String.format("Thread pool created:
>>>>>>> [type] Executor Service [id] %s [size] %d", identifier, threadPoolSize));
>>>>>>>                 }
>>>>>>>             }
>>>>>>>         }
>>>>>>>         return executorService;
>>>>>>>     }
>>>>>>>
>>>>>>> This will only create an executor service once for a given
>>>>>>> identifier.
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> Stratos server thread count goes beyond 1500 when deploying
2 or 3
>>>>>>>> apps. And it keeps growing proportional to the deployed app
count. This is
>>>>>>>> a very high resource usage.
>>>>>>>>
>>>>>>>> [1]
>>>>>>>> https://github.com/apache/stratos/blob/stratos-4.1.x/components/org.apache.stratos.autoscaler/src/main/java/org/apache/stratos/autoscaler/monitor/cluster/ClusterMonitor.java#L108
>>>>>>>>
>>>>>>>> [2]
>>>>>>>> https://github.com/apache/stratos/blob/stratos-4.1.x/components/org.apache.stratos.autoscaler/src/main/java/org/apache/stratos/autoscaler/monitor/component/ApplicationMonitor.java#L82
>>>>>>>>
>>>>>>>> [3]
>>>>>>>> https://github.com/apache/stratos/blob/stratos-4.1.x/components/org.apache.stratos.autoscaler/src/main/java/org/apache/stratos/autoscaler/monitor/component/GroupMonitor.java#L88
>>>>>>>>
>>>>>>>> [4]
>>>>>>>> https://github.com/apache/stratos/blob/stratos-4.1.x/components/org.apache.stratos.autoscaler/src/main/java/org/apache/stratos/autoscaler/monitor/component/ParentComponentMonitor.java#L112
>>>>>>>>
>>>>>>>> Thanks.
>>>>>>>>
>>>>>>>> --
>>>>>>>> Akila Ravihansa Perera
>>>>>>>> WSO2 Inc.;  http://wso2.com/
>>>>>>>>
>>>>>>>> Blog: http://ravihansa3000.blogspot.com
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Imesh Gunaratne
>>>>>>>
>>>>>>> Senior Technical Lead, WSO2
>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>
>>>>>>> --
>>>>>>> Thanks and Regards,
>>>>>>>
>>>>>>> Isuru H.
>>>>>>> +94 716 358 048* <http://wso2.com/>*
>>>>>>>
>>>>>>>
>>>>>>> * <http://wso2.com/>*
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Imesh Gunaratne
>>>>>
>>>>> Senior Technical Lead, WSO2
>>>>> Committer & PMC Member, Apache Stratos
>>>>>
>>>>> --
>>>>> Thanks and Regards,
>>>>>
>>>>> Isuru H.
>>>>> +94 716 358 048* <http://wso2.com/>*
>>>>>
>>>>>
>>>>> * <http://wso2.com/>*
>>>>>
>>>>>
>>>>>
>>>
>>> --
>>> Imesh Gunaratne
>>>
>>> Senior Technical Lead, WSO2
>>> Committer & PMC Member, Apache Stratos
>>>
>>> --
>>> Thanks and Regards,
>>>
>>> Isuru H.
>>> +94 716 358 048* <http://wso2.com/>*
>>>
>>>
>>> * <http://wso2.com/>*
>>>
>>>
>>>
>
>
> --
> Thanuja Uruththirakodeeswaran
> Software Engineer
> WSO2 Inc.;http://wso2.com
> lean.enterprise.middleware
>
> mobile: +94 774363167
>



-- 
Imesh Gunaratne

Senior Technical Lead, WSO2
Committer & PMC Member, Apache Stratos

Mime
View raw message