drill-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bridg...@apache.org
Subject drill git commit: throttling edits
Date Tue, 14 Nov 2017 21:37:55 GMT
Repository: drill
Updated Branches:
  refs/heads/gh-pages 3a7c71910 -> d79e23427


throttling edits


Project: http://git-wip-us.apache.org/repos/asf/drill/repo
Commit: http://git-wip-us.apache.org/repos/asf/drill/commit/d79e2342
Tree: http://git-wip-us.apache.org/repos/asf/drill/tree/d79e2342
Diff: http://git-wip-us.apache.org/repos/asf/drill/diff/d79e2342

Branch: refs/heads/gh-pages
Commit: d79e234272eea78e9a101cf044440d88811b4544
Parents: 3a7c719
Author: Bridget Bevens <bbevens@maprtech.com>
Authored: Tue Nov 14 13:36:17 2017 -0800
Committer: Bridget Bevens <bbevens@maprtech.com>
Committed: Tue Nov 14 13:36:17 2017 -0800

----------------------------------------------------------------------
 _data/docs.json                                 | 116 ++++++++++++++++---
 .../query-plans-and-tuning/065-throttling.md    |  12 +-
 2 files changed, 106 insertions(+), 22 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/drill/blob/d79e2342/_data/docs.json
----------------------------------------------------------------------
diff --git a/_data/docs.json b/_data/docs.json
index 4e555ef..a37ac18 100644
--- a/_data/docs.json
+++ b/_data/docs.json
@@ -2540,8 +2540,8 @@
             "next_title": "Identifying Performance Issues", 
             "next_url": "/docs/identifying-performance-issues/", 
             "parent": "Query Plans and Tuning", 
-            "previous_title": "Enabling Query Queuing", 
-            "previous_url": "/docs/enabling-query-queuing/", 
+            "previous_title": "Throttling", 
+            "previous_url": "/docs/throttling/", 
             "relative_path": "_docs/performance-tuning/query-plans-and-tuning/070-controlling-parallelization-to-balance-performance-with-multi-tenancy.md",

             "title": "Controlling Parallelization to Balance Performance with Multi-Tenancy",

             "url": "/docs/controlling-parallelization-to-balance-performance-with-multi-tenancy/"
@@ -3887,8 +3887,8 @@
                 }
             ], 
             "children": [], 
-            "next_title": "Controlling Parallelization to Balance Performance with Multi-Tenancy",

-            "next_url": "/docs/controlling-parallelization-to-balance-performance-with-multi-tenancy/",

+            "next_title": "Throttling", 
+            "next_url": "/docs/throttling/", 
             "parent": "Query Plans and Tuning", 
             "previous_title": "Sort-Based and Hash-Based Memory-Constrained Operators", 
             "previous_url": "/docs/sort-based-and-hash-based-memory-constrained-operators/",

@@ -7025,8 +7025,8 @@
                                 }
                             ], 
                             "children": [], 
-                            "next_title": "Controlling Parallelization to Balance Performance
with Multi-Tenancy", 
-                            "next_url": "/docs/controlling-parallelization-to-balance-performance-with-multi-tenancy/",

+                            "next_title": "Throttling", 
+                            "next_url": "/docs/throttling/", 
                             "parent": "Query Plans and Tuning", 
                             "previous_title": "Sort-Based and Hash-Based Memory-Constrained
Operators", 
                             "previous_url": "/docs/sort-based-and-hash-based-memory-constrained-operators/",

@@ -7046,11 +7046,32 @@
                                 }
                             ], 
                             "children": [], 
-                            "next_title": "Identifying Performance Issues", 
-                            "next_url": "/docs/identifying-performance-issues/", 
+                            "next_title": "Controlling Parallelization to Balance Performance
with Multi-Tenancy", 
+                            "next_url": "/docs/controlling-parallelization-to-balance-performance-with-multi-tenancy/",

                             "parent": "Query Plans and Tuning", 
                             "previous_title": "Enabling Query Queuing", 
                             "previous_url": "/docs/enabling-query-queuing/", 
+                            "relative_path": "_docs/performance-tuning/query-plans-and-tuning/065-throttling.md",

+                            "title": "Throttling", 
+                            "url": "/docs/throttling/"
+                        }, 
+                        {
+                            "breadcrumbs": [
+                                {
+                                    "title": "Query Plans and Tuning", 
+                                    "url": "/docs/query-plans-and-tuning/"
+                                }, 
+                                {
+                                    "title": "Performance Tuning", 
+                                    "url": "/docs/performance-tuning/"
+                                }
+                            ], 
+                            "children": [], 
+                            "next_title": "Identifying Performance Issues", 
+                            "next_url": "/docs/identifying-performance-issues/", 
+                            "parent": "Query Plans and Tuning", 
+                            "previous_title": "Throttling", 
+                            "previous_url": "/docs/throttling/", 
                             "relative_path": "_docs/performance-tuning/query-plans-and-tuning/070-controlling-parallelization-to-balance-performance-with-multi-tenancy.md",

                             "title": "Controlling Parallelization to Balance Performance
with Multi-Tenancy", 
                             "url": "/docs/controlling-parallelization-to-balance-performance-with-multi-tenancy/"
@@ -8023,8 +8044,8 @@
                         }
                     ], 
                     "children": [], 
-                    "next_title": "Controlling Parallelization to Balance Performance with
Multi-Tenancy", 
-                    "next_url": "/docs/controlling-parallelization-to-balance-performance-with-multi-tenancy/",

+                    "next_title": "Throttling", 
+                    "next_url": "/docs/throttling/", 
                     "parent": "Query Plans and Tuning", 
                     "previous_title": "Sort-Based and Hash-Based Memory-Constrained Operators",

                     "previous_url": "/docs/sort-based-and-hash-based-memory-constrained-operators/",

@@ -8044,11 +8065,32 @@
                         }
                     ], 
                     "children": [], 
-                    "next_title": "Identifying Performance Issues", 
-                    "next_url": "/docs/identifying-performance-issues/", 
+                    "next_title": "Controlling Parallelization to Balance Performance with
Multi-Tenancy", 
+                    "next_url": "/docs/controlling-parallelization-to-balance-performance-with-multi-tenancy/",

                     "parent": "Query Plans and Tuning", 
                     "previous_title": "Enabling Query Queuing", 
                     "previous_url": "/docs/enabling-query-queuing/", 
+                    "relative_path": "_docs/performance-tuning/query-plans-and-tuning/065-throttling.md",

+                    "title": "Throttling", 
+                    "url": "/docs/throttling/"
+                }, 
+                {
+                    "breadcrumbs": [
+                        {
+                            "title": "Query Plans and Tuning", 
+                            "url": "/docs/query-plans-and-tuning/"
+                        }, 
+                        {
+                            "title": "Performance Tuning", 
+                            "url": "/docs/performance-tuning/"
+                        }
+                    ], 
+                    "children": [], 
+                    "next_title": "Identifying Performance Issues", 
+                    "next_url": "/docs/identifying-performance-issues/", 
+                    "parent": "Query Plans and Tuning", 
+                    "previous_title": "Throttling", 
+                    "previous_url": "/docs/throttling/", 
                     "relative_path": "_docs/performance-tuning/query-plans-and-tuning/070-controlling-parallelization-to-balance-performance-with-multi-tenancy.md",

                     "title": "Controlling Parallelization to Balance Performance with Multi-Tenancy",

                     "url": "/docs/controlling-parallelization-to-balance-performance-with-multi-tenancy/"
@@ -12483,6 +12525,27 @@
             "title": "Text Files: CSV, TSV, PSV", 
             "url": "/docs/text-files-csv-tsv-psv/"
         }, 
+        "Throttling": {
+            "breadcrumbs": [
+                {
+                    "title": "Query Plans and Tuning", 
+                    "url": "/docs/query-plans-and-tuning/"
+                }, 
+                {
+                    "title": "Performance Tuning", 
+                    "url": "/docs/performance-tuning/"
+                }
+            ], 
+            "children": [], 
+            "next_title": "Controlling Parallelization to Balance Performance with Multi-Tenancy",

+            "next_url": "/docs/controlling-parallelization-to-balance-performance-with-multi-tenancy/",

+            "parent": "Query Plans and Tuning", 
+            "previous_title": "Enabling Query Queuing", 
+            "previous_url": "/docs/enabling-query-queuing/", 
+            "relative_path": "_docs/performance-tuning/query-plans-and-tuning/065-throttling.md",

+            "title": "Throttling", 
+            "url": "/docs/throttling/"
+        }, 
         "Troubleshooting": {
             "breadcrumbs": [], 
             "children": [], 
@@ -16365,8 +16428,8 @@
                                 }
                             ], 
                             "children": [], 
-                            "next_title": "Controlling Parallelization to Balance Performance
with Multi-Tenancy", 
-                            "next_url": "/docs/controlling-parallelization-to-balance-performance-with-multi-tenancy/",

+                            "next_title": "Throttling", 
+                            "next_url": "/docs/throttling/", 
                             "parent": "Query Plans and Tuning", 
                             "previous_title": "Sort-Based and Hash-Based Memory-Constrained
Operators", 
                             "previous_url": "/docs/sort-based-and-hash-based-memory-constrained-operators/",

@@ -16386,11 +16449,32 @@
                                 }
                             ], 
                             "children": [], 
-                            "next_title": "Identifying Performance Issues", 
-                            "next_url": "/docs/identifying-performance-issues/", 
+                            "next_title": "Controlling Parallelization to Balance Performance
with Multi-Tenancy", 
+                            "next_url": "/docs/controlling-parallelization-to-balance-performance-with-multi-tenancy/",

                             "parent": "Query Plans and Tuning", 
                             "previous_title": "Enabling Query Queuing", 
                             "previous_url": "/docs/enabling-query-queuing/", 
+                            "relative_path": "_docs/performance-tuning/query-plans-and-tuning/065-throttling.md",

+                            "title": "Throttling", 
+                            "url": "/docs/throttling/"
+                        }, 
+                        {
+                            "breadcrumbs": [
+                                {
+                                    "title": "Query Plans and Tuning", 
+                                    "url": "/docs/query-plans-and-tuning/"
+                                }, 
+                                {
+                                    "title": "Performance Tuning", 
+                                    "url": "/docs/performance-tuning/"
+                                }
+                            ], 
+                            "children": [], 
+                            "next_title": "Identifying Performance Issues", 
+                            "next_url": "/docs/identifying-performance-issues/", 
+                            "parent": "Query Plans and Tuning", 
+                            "previous_title": "Throttling", 
+                            "previous_url": "/docs/throttling/", 
                             "relative_path": "_docs/performance-tuning/query-plans-and-tuning/070-controlling-parallelization-to-balance-performance-with-multi-tenancy.md",

                             "title": "Controlling Parallelization to Balance Performance
with Multi-Tenancy", 
                             "url": "/docs/controlling-parallelization-to-balance-performance-with-multi-tenancy/"

http://git-wip-us.apache.org/repos/asf/drill/blob/d79e2342/_docs/performance-tuning/query-plans-and-tuning/065-throttling.md
----------------------------------------------------------------------
diff --git a/_docs/performance-tuning/query-plans-and-tuning/065-throttling.md b/_docs/performance-tuning/query-plans-and-tuning/065-throttling.md
index f01b127..25048c5 100644
--- a/_docs/performance-tuning/query-plans-and-tuning/065-throttling.md
+++ b/_docs/performance-tuning/query-plans-and-tuning/065-throttling.md
@@ -1,12 +1,12 @@
 ---
 title: "Throttling"
-date: 2016-11-21 22:28:44 UTC
+date: 2017-11-14 21:36:24 UTC
 parent: "Query Plans and Tuning"
 --- 
 
 Drill 1.12 introduces throttling. Throttling limits the number of concurrent queries that
run to prevent queries from failing with out-of-memory errors. When you enable throttling,
you configure the number of concurrent queries that can run and the resource requirements
for each query. Drill calculates the amount of memory to assign per query per node.
 
-If throttling is disabled, you most likely need to increase the amount of memory assigned
to the planner.memory.max_query_memory_per_node option based on the amount of direct memory
that Drill will allocate to the Sort and Hash Aggregate operators during each query on a node.
Drill must decide how much memory to assign to each of these operators, without knowing how
many concurrent queries might run. If Drill does not give enough memory to the Sort and Hash
Aggregate operators, queries fail. Using the throttling feature prevents this from happening.
+If throttling is disabled, you most likely need to increase the amount of memory assigned
to the planner.memory.max_query_memory_per_node option based on the amount of direct memory
that Drill will allocate to the [Sort and Hash Aggregate operators]({{site.baseurl}}/docs/sort-based-and-hash-based-memory-constrained-operators/)
during each query on a node. Drill must decide how much memory to assign to each of these
operators, without knowing how many concurrent queries might run. If Drill does not give enough
memory to the Sort and Hash Aggregate operators, queries fail. Using the throttling feature
prevents this from happening.
    
 
 ## Configuring Throttling  
@@ -81,10 +81,10 @@ You can change the memory reserve ration using the exec.queue.memory_reserve_rat
 
 ## Tuning  
 
-To use query profiles to determine the proper parameters: 
-
-- Set the query sizes conservatively to ensure queries succeed.  
-- Experiment to adjust the size threshold by observing actual costs for typical queries.
 
+To use query profiles to determine the proper parameters: 
+
+- Set the query sizes conservatively to ensure queries succeed.  
+- Experiment to adjust the size threshold by observing actual costs for typical queries.
 
 - Tweak the memory settings if problems occur such as out-of-memory due to joins and so on.
 
 


Mime
View raw message