drill-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bridg...@apache.org
Subject [drill] branch gh-pages updated: monitoring metrics doc updates
Date Sat, 10 Nov 2018 04:19:09 GMT
This is an automated email from the ASF dual-hosted git repository.

bridgetb pushed a commit to branch gh-pages
in repository https://gitbox.apache.org/repos/asf/drill.git


The following commit(s) were added to refs/heads/gh-pages by this push:
     new a79b24f  monitoring metrics doc updates
a79b24f is described below

commit a79b24f83b80a21fbf113423147e223857025ca9
Author: Bridget Bevens <bbevens@maprtech.com>
AuthorDate: Fri Nov 9 20:18:21 2018 -0800

    monitoring metrics doc updates
---
 _docs/performance-tuning/070-monitoring-metrics.md | 93 +++++++++++++++++-----
 1 file changed, 74 insertions(+), 19 deletions(-)

diff --git a/_docs/performance-tuning/070-monitoring-metrics.md b/_docs/performance-tuning/070-monitoring-metrics.md
index d53113b..5e2b374 100644
--- a/_docs/performance-tuning/070-monitoring-metrics.md
+++ b/_docs/performance-tuning/070-monitoring-metrics.md
@@ -1,10 +1,25 @@
 ---
 title: "Monitoring Metrics"
-date: 2016-07-14 18:41:09 UTC
+date: 2018-11-10
 parent: "Performance Tuning"
 --- 
 
-As of Drill 1.7, Drill uses JMX ([Java Management Extensions](https://docs.oracle.com/javase/tutorial/jmx/))
to monitor queries at runtime. JMX provides the architecture to dynamically manage and monitor
applications. JMX collects Drill system-level metrics that you can access through the Metrics
tab in the Drill Web Console or a remote JMX monitoring tool, such as JConsole or the VisualVM
+ MBeans plugin. The Web Console Metrics tab contains the collected metrics as tables, counters,
hi [...]
+The Metrics page in the Drill Web UI (http(s)://<drillbit-ip-address>:8047/metrics)
lists JVM and operating system metrics. You can use these metrics to debug the state of the
cluster. The Drill-specific metrics are prepended with `drill`, for example `drill.fragments.running`.
The other metrics relate specifically to the JVM.
+
+Drill uses JMX ([Java Management Extensions](https://docs.oracle.com/javase/tutorial/jmx/))
to monitor queries at runtime. JMX provides the architecture to dynamically manage and monitor
applications. JMX collects Drill system-level metrics that you can see in the Metrics tab
in the Drill Web UI or a remote JMX monitoring tool, such as JConsole or the VisualVM + MBeans
plugin.   
+
+Metrics collected by JMX are divided into the following categories on the Metrics page in
the Drill Web UI:  
+
+* **Gauges**  
+A gauge is an instantaneous measure of a value. See [Gauges]({{site.baseurl}}/docs/monitoring-metrics/#gauges).
  
+* **Counters**  
+A counter is a snapshot of the count of metrics at a particular point in time. (A gauge for
an [AtomicLong](https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/atomic/AtomicLong.html)
instance.) See [Counters]({{site.baseurl}}/docs/monitoring-metrics/#counters).  
+* **Histograms**    
+A histogram measures the statistical distribution of values in a stream of data.  
+* **Meters**  
+A meter measures the rate of events over time, for example requests per second.  
+* **Timers**  
+A timer measures the rate that a particular piece of code is called and the distribution
of its duration.  
 
 ## Remote Monitoring  
 You can enable the remote JMX Java feature to monitor a specific JVM from a remote location.
You can enable remote JMX with or without authentication. See the [Java documentation](http://docs.oracle.com/javase/7/docs/technotes/guides/management/agent.html).

@@ -18,25 +33,65 @@ JMX metric collection is enabled, by default. You can disable the metrics
option
 
 In `$DRILL_HOME/conf/drill-env.sh`, set the `drill.metrics.jmx.enabled` option to false through
the `DRILLBIT_JAVA_OPTS` variable. Add the variable in `drill-env.sh` if it does not exist:
 
-       export DRILLBIT_JAVA_OPTS="$DRILLBIT_JAVA_OPTS -Ddrill.metrics.jmx.enabled=false"
+       export DRILLBIT_JAVA_OPTS="$DRILLBIT_JAVA_OPTS -Ddrill.metrics.jmx.enabled=false"
  
+
+
+## Gauges 
+
+The following table lists the Drill-specific metrics in the Gauges section of the Metrics
page: 
+
+| Metric                               | Description                                    
                                                                                         
                                                                                         
                                                                                         
                                                                                         
                                            [...]
+|--------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
[...]
+| blocked.count                        | The number of threads that are blocked because they
are waiting on a monitor   lock. This metric is useful for debugging Drill issues.       
                                                                                         
                                                                                         
                                                                                         
                                        [...]
+| drill.fragments.running              | The number of query fragments currently running
in the Drillbit.                                                                         
                                                                                         
                                                                                         
                                                                                         
                                            [...]
+| drill.allocator.root.used            | The amount of memory (in bytes) used by the internal
memory allocator.                                                                        
                                                                                         
                                                                                         
                                                                                         
                                       [...]
+| drill.allocator.root.peak            | The peak amount of memory (in bytes) used by the
internal memory allocator.                                                               
                                                                                         
                                                                                         
                                                                                         
                                           [...]
+| drill.allocator.rpc.bit.control.peak | The maximum amount of bytes used across all   outgoing
and incoming control connections for this Drillbit at this moment.                       
                                                                                         
                                                                                         
                                                                                         
                                     [...]
+| drill.allocator.rpc.bit.control.used | The total number of bytes   currently used across
all outgoing and incoming control connections for this   Drillbit.                       
                                                                                         
                                                                                         
                                                                                         
                                          [...]
+| drill.allocator.rpc.bit.data.peak    | The maximum amount of memory   used between all
outgoing and incoming data connections for this Drillbit up   to this moment.            
                                                                                         
                                                                                         
                                                                                         
                                            [...]
+| drill.allocator.rpc.bit.data.used    | The total amount of memory used between all   outgoing
and incoming data connections tor this Drillbit.                                         
                                                                                         
                                                                                         
                                                                                         
                                     [...]
+| drill.allocator.rpc.bit.user.peak    | The maximum amount of memory used across all incoming
Drill client connections to this Drillbit up to this moment.                             
                                                                                         
                                                                                         
                                                                                         
                                      [...]
+| drill.allocator.rpc.bit.user.used    | The total amount of memory used across all incoming
Drill client connections to this Drillbit.                                               
                                                                                         
                                                                                         
                                                                                         
                                        [...]
+| drill.allocator.huge.size            | Total size in bytes of huge (greater than 16MB)
direct buffers allocated until now                                                       
                                                                                         
                                                                                         
                                                                                         
                                            [...]
+| drill.allocator.huge.count           | Number of allocations done for   direct buffer of
size greater than 16MB. Each of these allocation happens   from OS which comes with an overhead
rather from Netty's buffer pool.                                                         
                                                                                         
                                                                                         
                                    [...]
+| drill.allocator.normal.count         | Number of allocations done for   direct buffer of
size less than equal to 16MB. Each of these allocation   happens from Netty's buffer pool.
This counter is only updated in debug   environment when asserts are enabled to avoid overhead
for each allocation   during normal execution.                                           
                                                                                         
                                    [...]
+| drill.allocator.normal.size          | Total size in bytes of normal   (less than and equal
to 16MB) direct buffers allocated until now. This   counter is only updated in debug environment
when asserts are enabled to   avoid overhead for each allocation during normal execution.
                                                                                         
                                                                                         
                                [...]
+| drill.allocator.normal.size          | Total size in bytes of normal   (less than and equal
to 16MB) direct buffers allocated until now. This   counter is only updated in debug environment
when asserts are enabled to   avoid overhead for each allocation during normal execution.
                                                                                         
                                                                                         
                                [...]
+| heap.used                            | The   amount of heap memory (in bytes) used by the
JVM. 
+|                                                                                       
                                                                                         
                                                                                         
                                                                                         
                                                                                         
                                            [...]
+|
+| non-heap.used                        | The   amount of non-heap memory (in bytes) used
by the JVM.                                                                              
                                                                                         
                                                                                         
                                                                                         
                                            [...]
+| fd.usage                             | The   ratio of used file descriptors to total file
descriptors on *nix systems.                                                             
                                                                                         
                                                                                         
                                                                                         
                                         [...]
+| direct.used                          | The   amount of direct memory (in bytes) used by
the JVM. This metric is useful for   debugging Drill issues.                             
                                                                                         
                                                                                         
                                                                                         
                                           [...]
+| runnable.count                       | The   number of threads executing an action in the
JVM. This metric is useful for   debugging Drill issues.                                 
                                                                                         
                                                                                         
                                                                                         
                                         [...]
+| waiting.count                        | The number of threads waiting to execute. Typically,
threads waiting on other   threads to perform an action. This metric is useful for debugging
Drill   issues.                                                                          
                                                                                         
                                                                                         
                                    [...]
+| load.avg                             | Returns the "recent cpu   usage" for the Drillbit
process. This value is a double in the [0.0,1.0]   interval. A value of 0.0 means that none
of the CPUs were running threads   from the Drillbit process during the recent period of time
observed, while a   value of 1.0 means that all CPUs were actively running threads from the
  Drillbit process 100% of the time during the recent period being observed.   Threads from
the Drillbit process includes t [...]
+| uptime                               | Total uptime of Drillbit JVM in miliseconds. See
[getUptime()](https://docs.oracle.com/javase/7/docs/api/java/lang/management/RuntimeMXBean.html#getUptime()).
                                                                                         
                                                                                         
                                                                                         
                       [...]
+
+
+## Counters  
+
+The following table lists the Drill-specific metrics in the Counters section of the Metrics
page:   
+
+| Metric                  | Description                                                 
                                 |
+|-------------------------|-----------------------------------------------------------------------------------------------|
+| drill.queries.running   | The number of queries running for which this drillbit is the
Foreman.                         |
+| drill.queries.completed | The number of queries completed, canceled, or failed for which
this drillbit was the Foreman. |   
+
+
+## Histograms
+
+The following table lists the Drill-specific metrics in the Histograms section of the Metrics
page:  
 
-The following table lists the predefined Drill system-level metrics that you can view through
a JMX monitoring tool or the Drill Web Console:  
+| Metric                      | Description                                             
                                                                                         
                                                                                         
                                                                                         
                           |
+|-----------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| drill.allocator.huge.hist   | Displays the distribution of allocation of huge buffers up
to the current   time. Like count, it specifies number of huge buffer allocations completed
so   far. Max/Min specifies maximum/minimum size in bytes of the huge buffer   allocated.
Mean and other percentiles show the distribution of the huge   buffer allocation size in bytes.
                 |
+| drill.allocator.normal.hist | Displays the distribution of allocation of the normal size
buffers up to   the current time. Like count, it specifies the number of normal buffer   allocations
completed so far. Max/Min specifies maximum/minimum size in bytes   of the normal buffer allocated.
Mean and other percentiles show the   distribution of normal buffer allocation size in bytes.
|
 
-|    Metric                 | Description                                               
                                                                                         
               |
-|---------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| drill.queries.running     | The   number of queries running for which this drillbit is
the Foreman.                                                                             
                |
-| drill.queries.completed   | The   number of queries completed, canceled, or failed for
which this drillbit was   the Foreman.                                                   
                |
-| drill.fragments.running   | The   number of query fragments currently running in the drillbit.
                                                                                         
       |
-| drill.allocator.root.used | The   amount of memory (in bytes) used by the internal memory
allocator.                                                                               
            |
-| drill.allocator.root.peak | The   peak amount of memory (in bytes) used by the internal
memory allocator.                                                                        
              |
-| heap.used                 | The   amount of heap memory (in bytes) used by the JVM.   
                                                                                         
               |
-| non-heap.used             | The   amount of non-heap memory (in bytes) used by the JVM.
                                                                                         
              |
-| count                     | The   number of live threads, including daemon and non-daemon
threads.                                                                                 
            |
-| fd.usage                  | The   ratio of used file descriptors to total file descriptors
on *nix systems.                                                                         
           |
-|       direct.used         | The   amount of direct memory (in bytes) used by the JVM. This
metric is useful for   debugging Drill issues.                                           
           |
-| runnable.count            | The   number of threads executing an action in the JVM. This
metric is useful for   debugging Drill issues.                                           
             |
-| waiting.count             | The   number of threads waiting to execute. Typically, threads
waiting on other   threads to perform an action. This metric is useful for debugging Drill
  issues. |
-| blocked.count             | The   number of threads that are blocked because they are waiting
on a monitor   lock. This metric is useful for debugging Drill issues.                   
        |
+## Meters  
+No Drill-specific meters.
 
 
 
+## Timers  
+No Drill-specific timers.
\ No newline at end of file


Mime
View raw message