spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Spark (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (SPARK-9968) BlockGenerator lock structure can cause lock starvation of the block updating thread
Date Fri, 14 Aug 2015 13:27:46 GMT

     [ https://issues.apache.org/jira/browse/SPARK-9968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Apache Spark reassigned SPARK-9968:
-----------------------------------

    Assignee: Apache Spark  (was: Tathagata Das)

> BlockGenerator lock structure can cause lock starvation of the block updating thread
> ------------------------------------------------------------------------------------
>
>                 Key: SPARK-9968
>                 URL: https://issues.apache.org/jira/browse/SPARK-9968
>             Project: Spark
>          Issue Type: Sub-task
>          Components: Streaming
>            Reporter: Tathagata Das
>            Assignee: Apache Spark
>
> When the rate limiter is actually limiting the rate at which data is inserted into the
buffer, the synchronized block of BlockGenerator.addData stays blocked for long time. This
causes the thread switching the buffer and generating blocks (synchronized with addData) to
starve and not generate blocks for seconds. The correct solution is to not block on the rate
limiter within the synchronized block for adding data to the buffer. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message