storm-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From gov...@apache.org
Subject [storm] branch master updated: STORM-3615: Adds documentation for NUMA support
Date Fri, 22 May 2020 19:36:43 GMT
This is an automated email from the ASF dual-hosted git repository.

govind pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/storm.git


The following commit(s) were added to refs/heads/master by this push:
     new 515faaf  STORM-3615: Adds documentation for NUMA support
     new 59891a5  Merge pull request #3272 from govind-menon/STORM-3615
515faaf is described below

commit 515faafa231f07e806d5837036cf2f31deca2e79
Author: Govind Menon <govindappumenon@gmail.com>
AuthorDate: Wed May 20 18:26:00 2020 -0500

    STORM-3615: Adds documentation for NUMA support
---
 docs/NUMA.md | 97 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 97 insertions(+)

diff --git a/docs/NUMA.md b/docs/NUMA.md
new file mode 100644
index 0000000..4d4836f
--- /dev/null
+++ b/docs/NUMA.md
@@ -0,0 +1,97 @@
+---
+title: NUMA Support
+layout: documentation
+documentation: true
+---
+
+# Table of Contents
+1. [Introduction](#Introduction)
+2. [Architecture](#Architecture)
+3. [Resource Isolation Interface interaction](#RII-interaction)
+    1. [CgroupManager](#Setting-Memory-Requirement)
+    2. [DockerManager](#Setting-Shared-Memory)
+    3. [RuncLibManager](#Setting-CPU-Requirement)
+4. [Configuring NUMA](#Configuring-NUMA)
+    
+<div id='Introduction'/>
+
+## Introduction
+
+Non Uniform Memory Access ([NUMA](https://www.cc.gatech.edu/~echow/ipcc/hpc-course/HPC-numa.pdf))
is a new system architecture
+where the hosts resources are grouped by cores and memory in NUMA zones. Storm supports isolating/pinning
worker prcesses to specific
+NUMA zones via the supervisor to take advantage of this resource isolation and avoid the
penalty of using cross zone bus transfers
+<div id='Architecture'/>
+
+## Architecture
+
+Once Storm supervisors are configured for NUMA (see section below) they now heartbeat multiple
heartbeats - one for each NUMA zone.
+Each of these NUMA supervisors have a supervisor id with the same prefixed supervisor id
with the NUMA id differentiating them.
+Nimbus, and by extension the scheduler, see these heartbeats and views the supervisor as
multiple supervisors - one per configured NUMA zone. 
+Nimbus schedules topologies and assignments accordingly. The supervisor reads all assignments
with the prefixed assignments and then
+pins each worker to the NUMA zone according to the numa id in the assignment. The pinning
depends on the Resource Isolation Interface used
+and is elaborated on in the following section.
+
+<div id='RII-Interaction'/>
+
+### Resource Isolation Interface interaction
+
+Each implementation of the Resource Isolation Interface (RII) should now implement NUMA pinning.
The following are the current/soon to be available
+implementations                                                                         
                                                                        
+
+#### CgroupManager
+
+The CgroupManager prefixes the worker launch command with the numactl command for Linux hosts-
+
+```
+numactl --cpunodebind=<numaId>> --membind=<numaId> <worker launch command>
+```
+
+The worker is then bound to the NUMA zone's CPU cores and memory zone.
+#### DockerManager
+
+Will be updated upon adding Docker support
+
+#### RuncLibManager
+
+Will be updated upon adding Runc support
+
+<div id='Configuring NUMA'/>
+
+### Configuring NUMA
+
+In the Supervisor Config the following settings need to be set
+
+```
+supervisor.numa.meta:
+    "0": # Numa zone id
+        numa.cores: # Cores in NUMA zone (can be determined by using the numastat command)
+            - 0
+            - 1
+            - 2
+            - 3
+            - 4
+            - 5
+            - 12
+            - 13
+            - 14
+            - 15
+            - 16
+
+        numa.generic.resources.map: # Generic Resources in the zone to be used for generic
resource scheduling (optional)
+            network.resource.units: 50.0
+
+        numa.memory.mb: 42461 # Size of memory zone
+        numa.ports: # Ports to be assigned to workers pinned to the NUMA zone (this may include
ports not specified in SUPERVISOR_SLOTS_PORTS
+            - 6700
+            - 6701
+            - 6702
+            - 6703
+            - 6704
+            - 6705
+            - 6706
+            - 6707
+            - 6708
+            - 6709
+            - 6710
+            - 6711
+```
\ No newline at end of file


Mime
View raw message