[KYUUBI #630 ] [METRICS] Support expose Prometheus metrics

<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/NetEase/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
Support expose Prometheus metrics

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request

Closes #630 from pan3793/prometheus.

Closes #630

fff68b0 [Cheng Pan] nit
47c19eb [Cheng Pan] simplify
203e776 [Cheng Pan] address comments
10a877c [Cheng Pan] synchronized
26eb26e [Cheng Pan] simplify random port
827427c [Cheng Pan] fix import
476ee3a [Cheng Pan] address comments
9787c77 [Cheng Pan] update ut
c8e1b0e [Cheng Pan] update license
2387a97 [Cheng Pan] [METRICS] Support expose Prometheus metrics

Authored-by: Cheng Pan <379377944@qq.com>
Signed-off-by: Kent Yao <yao@apache.org>

2021-05-13 12:32:27 +08:00

5.9 KiB

Raw Blame History

Kyuubi Server Metrics

Kyuubi has a configurable metrics system based on the Dropwizard Metrics Library. This allows users to report Kyuubi metrics to a variety of kyuubi.metrics.reporters. The metrics provide instrumentation for specific activities and Kyuubi server.

Configurations

The metrics system is configured via $KYUUBI_HOME/conf/kyuubi-defaults.conf.

Key	Default	Meaning	Type	Since
kyuubi.metrics .enabled	true	Set to true to enable kyuubi metrics system	boolean	1.2.0
kyuubi.metrics .reporters	JSON	A comma separated list for all metrics reporters CONSOLE - ConsoleReporter which outputs measurements to CONSOLE periodically. JMX - JmxReporter which listens for new metrics and exposes them as MBeans. JSON - JsonReporter which outputs measurements to json file periodically. PROMETHEUS - PrometheusReporter which exposes metrics in prometheus format. SLF4J - Slf4jReporter which outputs measurements to system log periodically.	seq	1.2.0
kyuubi.metrics .console.interval	PT5S	How often should report metrics to console	duration	1.2.0
kyuubi.metrics.json .interval	PT5S	How often should report metrics to json file	duration	1.2.0
kyuubi.metrics.json .location	metrics	Where the json metrics file located	string	1.2.0
kyuubi.metrics .prometheus.path	/metrics	URI context path of prometheus metrics HTTP server	string	1.2.0
kyuubi.metrics .prometheus.port	10019	Prometheus metrics HTTP server port	int	1.2.0
kyuubi.metrics.slf4j .interval	PT5S	How often should report metrics to SLF4J logger	duration	1.2.0

Metrics

These metrics include:

Metrics Prefix	Metrics Suffix	Type	Since	Description
kyuubi .exec.pool .threads.alive		gauge	1.2.0	threads keepAlive in the backend executive thread pool
kyuubi .exec.pool .threads.active		gauge	1.2.0	threads active in the backend executive thread pool
kyuubi .connection.total		counter	1.2.0	cumulative connection count
kyuubi .connection.opened		gauge	1.2.0	current active connection count
kyuubi .connection.opened	`${user}`	counter	1.2.0	cumulative connections requested by a `${user}`
kyuubi .connection.failed		counter	1.2.0	cumulative failed connection count
kyuubi .connection.failed	`${user}`	counter	1.2.0	cumulative failed connections for a `${user}`
kyuubi .statement.total		counter	1.2.0	cumulative opened statement count
kyuubi .statement.opened		counter	1.2.0	current opened statement count
kyuubi .statement.failed	`${errorType}`	counter	1.2.0	cumulative failed statement for a particular `${errorType}`, e.g. `AnalysisException`
kyuubi .engine.total		counter	1.2.0	cumulative created engines
kyuubi .engine.timeout		counter	1.2.0	cumulative timeout engines
kyuubi .engine.failed	`${user}`	counter	1.2.0	cumulative explicitly failed engine count for a `${user}`
kyuubi .engine.failed	`${errorType}`	counter	1.2.0	cumulative explicitly failed engine count for a particular `${errorType}`, e.g. `ClassNotFoundException`

5.9 KiB Raw Blame History

Kyuubi Server Metrics

Configurations

Metrics

5.9 KiB

Raw Blame History