Commit Graph

6 Commits

Author SHA1 Message Date
Wang, Fei
7fbeea66fd [KYUUBI #7072][FOLLOWUP] Fix engine startup permit grafana pannel unit
### Why are the changes needed?

Followup for https://github.com/apache/kyuubi/pull/7072
The metrics unit should not be `ms`.

### How was this patch tested?

<img width="569" alt="image" src="https://github.com/user-attachments/assets/df83b003-762d-4ee2-bbe1-c1af55ae9795" />

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #7091 from turboFei/7072_followup.

Closes #7072

d7c4fe4f9 [Wang, Fei] fix unit

Authored-by: Wang, Fei <fwang12@ebay.com>
Signed-off-by: Wang, Fei <fwang12@ebay.com>
2025-06-12 23:34:59 -07:00
Lennon Chin
cad5a392f3
[KYUUBI #7072] Expose metrics of engine startup permit state
### Why are the changes needed?

The metrics `kyuubi_operation_state_LaunchEngine_*` cannot reflect the state of Semaphore after configuring the maximum engine startup limit through `kyuubi.server.limit.engine.startup`, add some metrics to show the relevant permit state.

### How was this patch tested?

### Was this patch authored or co-authored using generative AI tooling?

Closes #7072 from LennonChin/engine_startup_metrics.

Closes #7072

d6bf3696a [Lennon Chin] Expose metrics of engine startup permit status

Authored-by: Lennon Chin <i@coderap.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2025-05-29 13:27:42 +08:00
Cheng Pan
6d99b20e04
[KYUUBI #6870][FOLLOWUP] Correct file name of grafana/README.md
### Why are the changes needed?

Fix a typo of file name.

### How was this patch tested?

Review.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #7074 from pan3793/6870-f.

Closes #6870

45915d978 [Cheng Pan] [KYUUBI #6870][FOLLOWUP] Correct file name of grafana/README.md

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2025-05-26 12:48:30 +08:00
Wang, Fei
2b37c037dc
[KYUUBI #6870] Add more metrics for grafana dashboard
### Why are the changes needed?

Add more metrics for monitoring.

### How was this patch tested?

1. Overall
<img width="1723" alt="image" src="https://github.com/user-attachments/assets/65cb7e5a-4ea0-45c8-885e-392fc643f305" />2. Connection(repeat for connectionType - `thrift_binary_connection,rest_connection,thrift_http_connection,metadata_request`)
For example:
<img width="1909" alt="image" src="https://github.com/user-attachments/assets/51c6f729-55da-43a9-bd85-fd22b1c3a447" />
<img width="1916" alt="image" src="https://github.com/user-attachments/assets/72b3d5dc-a7dc-4a3b-8092-0f68aaaa7326" />

<img width="1721" alt="image" src="https://github.com/user-attachments/assets/156dc67a-c58a-4879-9d04-de4e60a444f2" />

3. Operation(repeat for operationType - `ExecuteStatement,BatchJobSubmission,LaunchEngine`)

<img width="1901" alt="image" src="https://github.com/user-attachments/assets/2d7e500e-18b3-4fc9-bddc-a11d8deff203" />
<img width="1913" alt="image" src="https://github.com/user-attachments/assets/e5557297-7bb8-49d6-b16d-860e14fd1b69" />

4. Extra
<img width="1721" alt="image" src="https://github.com/user-attachments/assets/141ded05-4e9e-4d54-99f6-455f51e7afb6" />

5. Move the JVM statistics down.
### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #6870 from turboFei/grafana_metrics.

Closes #6870

25fb32960 [Wang, Fei] Overall => Overview
12919411f [Fei Wang] Grafana metrics ssl (#32)
879b8a49f [Wang, Fei] Gc increase during trendInterval
3a7b6be62 [Wang, Fei] Engine Submitting => Launching
1f4be56c1 [Wang, Fei] add total opened interactive /batch connection
dde6aa785 [Wang, Fei] format
909e8e2e6 [Wang, Fei] remove plugin version
86c0a752d [Wang, Fei] metadata request new()
174bc1dd6 [Wang, Fei] threshold off
529ebfff6 [Wang, Fei] remove overrides
fa5d9b554 [Wang, Fei] Revert "remove 1m trend interval"
106718280 [Wang, Fei] remove 1m trend interval
e33e31036 [Wang, Fei] repeat - includeAll, multi
d6172ef2d [Wang, Fei] id1
f9c5d7e96 [Fei Wang] Grafana metrics update (#31)
369243d18 [Wang, Fei] metadata
bb492f8c1 [Wang, Fei] add trend interval
bf1188619 [Wang, Fei] add operation type
7e8a4067e [Wang, Fei] conn type

Lead-authored-by: Wang, Fei <fwang12@ebay.com>
Co-authored-by: Fei Wang <cn.feiwang@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2024-12-27 10:24:42 +08:00
Wang, Fei
031caf3d7d
[KYUUBI #6867] Refactor the grafana dashboard template
### Why are the changes needed?

The original dashboard dashboard template does not work for my use case:

In this PR:
1. I add the `baseFilter` variable, default value is `job=~"kyuubi"`.
2. apply the `baseFilter` for all expressions
3. add `baseLegend` for the legendFormat
4. Fix hardcode typos
5. remove the `sum` from the expressions for JVM

### How was this patch tested?

<img width="1723" alt="image" src="https://github.com/user-attachments/assets/2da3366d-4cd7-41fe-9a6b-1159d0b61734" />

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #6867 from turboFei/dash_board.

Closes #6867

d0d1a3612 [Wang, Fei] comments
9fc3b5c20 [Wang, Fei] use kyuubi_jvm_uptime to get instance
280f97c59 [Wang, Fei] green
f8fc35193 [Wang, Fei] all filter
8d1ac2fdf [Wang, Fei] base legend
9c18bfcb0 [Wang, Fei] save
1a4799013 [Wang, Fei] job=kyuubi
ea7634580 [Wang, Fei] instant datasource and remove cluster
19db2b3e1 [Wang, Fei] add filter

Authored-by: Wang, Fei <fwang12@ebay.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2024-12-26 14:03:05 +08:00
hezhao2
7e8275b7b4
[KYUUBI #5834] Add Grafana dashboard template
### _Why are the changes needed?_

This PR adds a basic Grafana Dashboard template, also updates the metrics docs to guide users to use Prometheus and Grafana to monitor the Kyuubi server.

The Grafana Dashboard template is exported from the Grafana OSS v11.4.0

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [x] Add screenshots for manual tests if appropriate

<img width="1484" alt="image" src="https://github.com/user-attachments/assets/417b35fa-cd12-4e51-b73f-2955282aa187" />

- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request

Closes #5147 from zhaohehuhu/Improvement-0809.

Closes #5834

f6fc2d71e [Cheng Pan] fix style
465f0546a [Cheng Pan] update dashboard
3fa2d237e [hezhao2] add status chart
4b2bd3dbc [hezhao2] add status chart
185f2cccf [hezhao2] make it compatible with kyuubi 1.8
457085be5 [hezhao2] add REAMDE.md to guide users
45e3ba3e5 [hezhao2] add docker file build a grafana image and load dashboards available
dbc22108b [hezhao2] Add Grafana dashboard template

Lead-authored-by: hezhao2 <hezhao2@cisco.com>
Co-authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2024-12-24 10:30:50 +08:00