helix-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lei Xia (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HELIX-613) TaskStateModel generates significant amount of threads and causing thread leaking problem
Date Sun, 08 Nov 2015 06:43:12 GMT
Lei Xia created HELIX-613:
-----------------------------

             Summary: TaskStateModel generates significant amount of threads and causing thread
leaking problem
                 Key: HELIX-613
                 URL: https://issues.apache.org/jira/browse/HELIX-613
             Project: Apache Helix
          Issue Type: Bug
    Affects Versions: 0.6.x
            Reporter: Lei Xia
            Assignee: Lei Xia


Current TaskStateModel creates a thread-pool containing 40 threads for each instance of TaskStateModel,
thus it creates 40 threads for each task (partition). Since Job are dynamic resources, the
thread pool is not properly shutdown when task has completed (or timeouted, failed, etc).
 We saw ~10000 threads were created in our production machines.

Also, the timeout timer in each TaskStateModel is not properly cancelled even though the task
has completed or failed. The timer consume a thread even though it is not used anymore. 

The proposed solution is to use a shared thread pool for all TaskStateModel in a single TaskStateModelFactory
for all regular tasks and timeout task.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message