[KYUUBI #1607 ] [DOC] Add operation and backend service metrics in metrics doc

<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
Add operation and backend service metrics doc.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [x] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
![FireShot Capture 002 - 2  Monitoring Kyuubi - Server Metrics — Kyuubi 1 5 0-SNAPSHOT documen_ - localhost](https://user-images.githubusercontent.com/29809822/147097486-c2fc3867-de9a-4545-b927-85af3aeaa50a.png)

Closes #1607 from zhenjiaguo/add_be_metric_doc.

Closes #1607

e10bf146 [zhenjiaguo] Comment
0062f3d8 [zhenjiaguo] Metric format adjustment
af84534c [zhenjiaguo] Add metrics version compatibility description
c6f9eb26 [zhenjiaguo] Add operation and backend service method metric info doc

Authored-by: zhenjiaguo <zhenjia_guo@163.com>
Signed-off-by: Kent Yao <yao@apache.org>

2021-12-23 11:31:57 +08:00

11 KiB

Raw Blame History

Monitoring Kyuubi - Server Metrics

Kyuubi has a configurable metrics system based on the Dropwizard Metrics Library. This allows users to report Kyuubi metrics to a variety of kyuubi.metrics.reporters. The metrics provide instrumentation for specific activities and Kyuubi server.

Configurations

The metrics system is configured via $KYUUBI_HOME/conf/kyuubi-defaults.conf.

Key	Default	Meaning	Type	Since
kyuubi.metrics .enabled	true	Set to true to enable kyuubi metrics system	boolean	1.2.0
kyuubi.metrics .reporters	JSON	A comma separated list for all metrics reporters CONSOLE - ConsoleReporter which outputs measurements to CONSOLE periodically. JMX - JmxReporter which listens for new metrics and exposes them as MBeans. JSON - JsonReporter which outputs measurements to json file periodically. PROMETHEUS - PrometheusReporter which exposes metrics in prometheus format. SLF4J - Slf4jReporter which outputs measurements to system log periodically.	seq	1.2.0
kyuubi.metrics .console.interval	PT5S	How often should report metrics to console	duration	1.2.0
kyuubi.metrics.json .interval	PT5S	How often should report metrics to json file	duration	1.2.0
kyuubi.metrics.json .location	metrics	Where the json metrics file located	string	1.2.0
kyuubi.metrics .prometheus.path	/metrics	URI context path of prometheus metrics HTTP server	string	1.2.0
kyuubi.metrics .prometheus.port	10019	Prometheus metrics HTTP server port	int	1.2.0
kyuubi.metrics.slf4j .interval	PT5S	How often should report metrics to SLF4J logger	duration	1.2.0

Metrics

These metrics include:

Metrics Prefix	Metrics Suffix	Type	Since	Description
kyuubi .exec.pool .threads.alive		gauge	1.2.0	threads keepAlive in the backend executive thread pool
kyuubi .exec.pool .threads.active		gauge	1.2.0	threads active in the backend executive thread pool
kyuubi .connection.total		counter	1.2.0	cumulative connection count
kyuubi .connection.opened		gauge	1.2.0	current active connection count
kyuubi .connection.opened	`${user}`	counter	1.2.0	current active connections count requested by a `${user}`
kyuubi .connection.failed		counter	1.2.0	cumulative failed connection count
kyuubi .connection.failed	`${user}`	counter	1.2.0	cumulative failed connections for a `${user}`
kyuubi .operation.total		counter	1.5.0	cumulative opened operation count
kyuubi .operation.total	`${operationType}`	counter	1.5.0	cumulative opened count for the operation `${operationType}`
kyuubi .operation.opened		gauge	1.5.0	current opened operation count
kyuubi .operation.opened	`${operationType}`	counter	1.5.0	current opened count for the operation `${operationType}`
kyuubi .operation.failed	`${operationType}` `.${errorType}`	counter	1.5.0	cumulative failed count for the operation `${operationType}` with a particular `${errorType}`, e.g. `execute_statement.AnalysisException`
kyuubi .engine.total		counter	1.2.0	cumulative created engines
kyuubi .engine.timeout		counter	1.2.0	cumulative timeout engines
kyuubi .engine.failed	`${user}`	counter	1.2.0	cumulative explicitly failed engine count for a `${user}`
kyuubi .engine.failed	`${errorType}`	counter	1.2.0	cumulative explicitly failed engine count for a particular `${errorType}`, e.g. `ClassNotFoundException`
kyuubi .backend_service .open_session_ms		histogram	1.5.0	kyuubi backend service `openSession` method time cost histogram statistics
kyuubi .backend_service .close_session_ms		histogram	1.5.0	kyuubi backend service `closeSession` method time cost histogram statistics
kyuubi .backend_service .get_info_ms		histogram	1.5.0	kyuubi backend service `getInfo` method time cost histogram statistics
kyuubi .backend_service .execute_statement_ms		histogram	1.5.0	kyuubi backend service `executeStatement` method time cost histogram statistics
kyuubi .backend_service .get_type_info_ms		histogram	1.5.0	kyuubi backend service `getTypeInfo` method time cost histogram statistics
kyuubi .backend_service .get_catalogs_ms		histogram	1.5.0	kyuubi backend service `getCatalogs` method time cost histogram statistics
kyuubi .backend_service .get_schemas_ms		histogram	1.5.0	kyuubi backend service `getSchemas` method time cost histogram statistics
kyuubi .backend_service .get_tables_ms		histogram	1.5.0	kyuubi backend service `getTables` method time cost histogram statistics
kyuubi .backend_service .get_table_types_ms		histogram	1.5.0	kyuubi backend service `getTableTypes` method time cost histogram statistics
kyuubi .backend_service .get_columns_ms		histogram	1.5.0	kyuubi backend service `getColumns` method time cost histogram statistics
kyuubi .backend_service .get_functions_ms		histogram	1.5.0	kyuubi backend service `getFunctions` method time cost histogram statistics
kyuubi .backend_service .get_operation_status_ms		histogram	1.5.0	kyuubi backend service `getOperationStatus` method time cost histogram statistics
kyuubi .backend_service .cancel_operation_ms		histogram	1.5.0	kyuubi backend service `cancelOperation` method time cost histogram statistics
kyuubi .backend_service .close_operation_ms		histogram	1.5.0	kyuubi backend service `closeOperation` method time cost histogram statistics
kyuubi .backend_service .get_result_set_metadata_ms		histogram	1.5.0	kyuubi backend service `getResultSetMetadata` method time cost histogram statistics
kyuubi .backend_service .fetch_results_ms		histogram	1.5.0	kyuubi backend service `fetchResults` method time cost histogram statistics

Before v1.5.0, if you use these metrics:

kyuubi.statement.total
kyuubi.statement.opened
kyuubi.statement.failed.${errorType}

Since v1.5.0, you can use the following metrics to replace:

kyuubi.operation.total.ExecuteStatement
kyuubi.operation.opened.ExecuteStatement
kyuubi.operation.failed.ExecuteStatement.${errorType}

11 KiB Raw Blame History

Monitoring Kyuubi - Server Metrics

Configurations

Metrics

11 KiB

Raw Blame History