spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Or (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-1172) Improve naming of the BlockManager classes
Date Sun, 30 Mar 2014 18:37:15 GMT

    [ https://issues.apache.org/jira/browse/SPARK-1172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13954787#comment-13954787
] 

Andrew Or commented on SPARK-1172:
----------------------------------

Here's a quick clarification of the current naming scheme:

BlockMaster runs on both executors the driver. On the driver, it's only used in the local
mode.

BlockManagerMaster runs on both executors and the driver. It handles the communication from
each BlockManager to the BlockManagerMasterActor. On the driver, this is a local inter-thread
communication.

BlockManagerMasterActor exists only on the driver. It maintains an overview of which blocks
are where and other useful information. It does not necessarily know about all blocks, however,
because there is an option in the BlockManager to update a block without telling master (i.e.
BlockManagerMasterActor).

BlockManagerSlaveActor exists on both executors and the driver. It handles the communication
from the BlockManagerMasterActor to each BlockManager. As before, this is a local inter-thread
communication on the driver.

> Improve naming of the BlockManager classes
> ------------------------------------------
>
>                 Key: SPARK-1172
>                 URL: https://issues.apache.org/jira/browse/SPARK-1172
>             Project: Apache Spark
>          Issue Type: Improvement
>          Components: Block Manager
>            Reporter: Patrick Wendell
>            Assignee: Andrew Or
>            Priority: Minor
>             Fix For: 1.0.0
>
>
> There is something called BlockManagerMaster that doesn't run on the driver (only on
the Executors). There is a thing called BlockManagerWorker that is not really a "worker" component
but a bridge between the connection manager and the block manager. I found these names confusing
on my first read of this code and in the last week two other people have asked me to explain
it to them. So perhaps we can find more intuitive names here... :)



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message