spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shixiong Zhu (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (SPARK-25181) Block Manager master and slave thread pools are unbounded
Date Wed, 22 Aug 2018 17:37:00 GMT

     [ https://issues.apache.org/jira/browse/SPARK-25181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Shixiong Zhu resolved SPARK-25181.
----------------------------------
       Resolution: Fixed
         Assignee: Mukul Murthy
    Fix Version/s: 2.4.0

> Block Manager master and slave thread pools are unbounded
> ---------------------------------------------------------
>
>                 Key: SPARK-25181
>                 URL: https://issues.apache.org/jira/browse/SPARK-25181
>             Project: Spark
>          Issue Type: Bug
>          Components: Spark Core
>    Affects Versions: 2.3.0
>            Reporter: Mukul Murthy
>            Assignee: Mukul Murthy
>            Priority: Major
>             Fix For: 2.4.0
>
>
> Currently, BlockManagerMasterEndpoint and BlockManagerSlaveEndpoint both have thread
pools with unbounded numbers of threads. In certain cases, this can lead to driver OOM errors.
We should add an upper bound on the number of threads in these thread pools; this should not
break any existing behavior because they still have queues of size Integer.MAX_VALUE.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message