phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-3655) Metrics for PQS
Date Mon, 15 May 2017 23:59:04 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-3655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16011520#comment-16011520
] 

ASF GitHub Bot commented on PHOENIX-3655:
-----------------------------------------

Github user rahulsIOT commented on the issue:

    https://github.com/apache/phoenix/pull/242
  
    Metrics Snippet
    **Statement Level -** 
    {"Statement-OverAllReadRequestMetrics":{"RESULT_SET_TIME_MS":22,"QUERY_FAILED_COUNTER":0,"QUERY_TIMEOUT_COUNTER":0,"WALL_CLOCK_TIME_MS":52,"CACHE_REFRESH_SPLITS_COUNTER":0,"NUM_PARALLEL_SCANS":1}}
    
    {"Statement-RequestReadMetrics":{"QUERYSERVERBASICSIT1494892379503":{"TASK_QUEUE_WAIT_TIME":0,"SCAN_BYTES":208,"TASK_END_TO_END_TIME":10,"TASK_EXECUTED_COUNTER":1,"TASK_EXECUTION_TIME":10,"TASK_REJECTED_COUNTER":0}}}
    
    **Global Metrics -**
    
    {"global":[{"numberOfSamples":0,"description":"Batch sizes of mutations","currentMetricState":"MUTATION_BATCH_SIZE:
0, Number of samples: 0","totalSum":0,"name":"MUTATION_BATCH_SIZE","value":0},{"numberOfSamples":0,"description":"Size
of mutations in bytes","currentMetricState":"MUTATION_BYTES: 0, Number of samples: 0","totalSum":0,"name":"MUTATION_BYTES","value":0},{"numberOfSamples":0,"description":"Time
it took to commit mutations","currentMetricState":"MUTATION_COMMIT_TIME: 0, Number of samples:
0","totalSum":0,"name":"MUTATION_COMMIT_TIME","value":0},{"numberOfSamples":7,"description":"Query
times","currentMetricState":"QUERY_TIME: 52, Number of samples: 7","totalSum":52,"name":"QUERY_TIME","value":52},{"numberOfSamples":11,"description":"Number
of scans that were executed in parallel","currentMetricState":"NUM_PARALLEL_SCANS: 11, Number
of samples: 11","totalSum":11,"name":"NUM_PARALLEL_SCANS","value":11},{"numberOfSamples":2,"description":"Number
of bytes read by scans","currentMetricState":"SCAN_BYTES: 192, Number of samples: 2","totalSum":192,"name":"SCAN_BYTES","value":192},{"numberOfSamples":0,"description":"Size
of spool files created in bytes","currentMetricState":"SPOOL_FILE_SIZE: 0, Number of samples:
0","totalSum":0,"name":"SPOOL_FILE_SIZE","value":0},{"numberOfSamples":0,"description":"Number
of bytes allocated by the memory manager","currentMetricState":"MEMORY_CHUNK_BYTES: 0, Number
of samples: 0","totalSum":0,"name":"MEMORY_CHUNK_BYTES","value":0},{"numberOfSamples":0,"description":"Number
of milliseconds threads needed to wait for memory to be allocated through memory manager","currentMetricState":"MEMORY_WAIT_TIME:
0, Number of samples: 0","totalSum":0,"name":"MEMORY_WAIT_TIME","value":0},{"numberOfSamples":11,"description":"Time
in milliseconds tasks had to wait in the queue of the thread pool executor","currentMetricState":"TASK_QUEUE_WAIT_TIME:
5, Number of samples: 11","totalSum":5,"name":"TASK_QUEUE_WAIT_TIME","value":5},{"numberOfSamples":11,"description":"Time
in milliseconds spent by tasks from creation to completion","currentMetricState":"TASK_END_TO_END_TIME:
139, Number of samples: 11","totalSum":139,"name":"TASK_END_TO_END_TIME","value":139},{"numberOfSamples":11,"description":"Time
in milliseconds tasks took to execute","currentMetricState":"TASK_EXECUTION_TIME: 134, Number
of samples: 11","totalSum":134,"name":"TASK_EXECUTION_TIME","value":134},{"numberOfSamples":99,"description":"Counter
for number of mutation sql statements","currentMetricState":"MUTATION_SQL_COUNTER: 99, Number
of samples: 99","totalSum":99,"name":"MUTATION_SQL_COUNTER","value":99},{"numberOfSamples":7,"description":"Counter
for number of sql queries","currentMetricState":"SELECT_SQL_COUNTER: 7, Number of samples:
7","totalSum":7,"name":"SELECT_SQL_COUNTER","value":7},{"numberOfSamples":11,"description":"Counter
for number of tasks submitted to the thread pool executor","currentMetricState":"TASK_EXECUTED_COUNTER:
11, Number of samples: 11","totalSum":11,"name":"TASK_EXECUTED_COUNTER","value":11},{"numberOfSamples":0,"description":"Counter
for number of tasks that were rejected by the thread pool executor","currentMetricState":"TASK_REJECTED_COUNTER:
0, Number of samples: 0","totalSum":0,"name":"TASK_REJECTED_COUNTER","value":0},{"numberOfSamples":0,"description":"Number
of times query timed out","currentMetricState":"QUERY_TIMEOUT_COUNTER: 0, Number of samples:
0","totalSum":0,"name":"QUERY_TIMEOUT_COUNTER","value":0},{"numberOfSamples":0,"description":"Number
of times query failed","currentMetricState":"QUERY_FAILED_COUNTER: 0, Number of samples: 0","totalSum":0,"name":"QUERY_FAILED_COUNTER","value":0},{"numberOfSamples":0,"description":"Number
of spool files created","currentMetricState":"SPOOL_FILE_COUNTER: 0, Number of samples: 0","totalSum":0,"name":"SPOOL_FILE_COUNTER","value":0},{"numberOfSamples":14,"description":"Number
of open phoenix connections","currentMetricState":"OPEN_PHOENIX_CONNECTIONS_COUNTER: 0, Number
of samples: 14","totalSum":0,"name":"OPEN_PHOENIX_CONNECTIONS_COUNTER","value":0},{"numberOfSamples":1,"description":"Number
of ConnectionQueryServicesImpl instantiated","currentMetricState":"QUERY_SERVICES_COUNTER:
1, Number of samples: 1","totalSum":1,"name":"QUERY_SERVICES_COUNTER","value":1},{"numberOfSamples":1,"description":"Number
of HConnections created by phoenix driver","currentMetricState":"HCONNECTIONS_COUNTER: 1,
Number of samples: 1","totalSum":1,"name":"HCONNECTIONS_COUNTER","value":1}]}
    
    
    Global Metrics is also exposed via JMX. ( screen shot attached)
    ![global_metrics](https://cloud.githubusercontent.com/assets/22037123/26084258/bc60c16a-398f-11e7-9607-a9d8d50a6aa1.png)
    
    



> Metrics for PQS
> ---------------
>
>                 Key: PHOENIX-3655
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-3655
>             Project: Phoenix
>          Issue Type: New Feature
>    Affects Versions: 4.8.0
>         Environment: Linux 3.13.0-107-generic kernel, v4.9.0-HBase-0.98
>            Reporter: Rahul Shrivastava
>            Assignee: Rahul Shrivastava
>             Fix For: 4.9.0
>
>         Attachments: MetricsforPhoenixQueryServerPQS.pdf
>
>   Original Estimate: 240h
>  Remaining Estimate: 240h
>
> Phoenix Query Server runs a separate process compared to its thin client. Metrics collection
is currently done by PhoenixRuntime.java i.e. at Phoenix driver level. We need the following
> 1. For every jdbc statement/prepared statement/ run by PQS , we need capability to collect
metrics at PQS level and push the data to external sink i.e. file, JMX , other external custom
sources. 
> 2. Besides this global metrics could be periodically collected and pushed to the sink.

> 2. PQS can be configured to turn on metrics collection and type of collect ( runtime
or global) via hbase-site.xml
> 3. Sink could be configured via an interface in hbase-site.xml. 
> All metrics definition https://phoenix.apache.org/metrics.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message