[KYUUBI #5381] Change the default metrics reporter to Prometheus
### _Why are the changes needed?_ Close #5381 change default metrics reporter to prometheus since Kyuubi 1.8 ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request ### _Was this patch authored or co-authored using generative AI tooling?_ Closes #5344 from zhaohehuhu/Improvement-0928. Closes #5381 84f4c8208 [hezhao2] reset METRICS_REPORTERS for test case b9ee5f711 [Cheng Pan] Update kyuubi-server/src/test/scala/org/apache/kyuubi/events/handler/ServerJsonLoggingEventHandlerSuite.scala 86165a6fe [Cheng Pan] Update kyuubi-server/src/test/scala/org/apache/kyuubi/events/handler/ServerJsonLoggingEventHandlerSuite.scala a3605b626 [hezhao2] set METRICS_PROMETHEUS_PORT to 0 for test cases f1a4d2861 [hezhao2] restore version number for kyuubi.metrics.reporters in doc dae40e1a2 [hezhao2] change default metrics reporter to prometheus Lead-authored-by: hezhao2 <hezhao2@cisco.com> Co-authored-by: Cheng Pan <pan3793@gmail.com> Signed-off-by: Cheng Pan <chengpan@apache.org>
This commit is contained in:
hezhao2
Cheng Pan
parent
1b229b63a0
commit
4bb67bdf3b
@ -352,16 +352,16 @@ You can configure the Kyuubi properties in `$KYUUBI_HOME/conf/kyuubi-defaults.co
|
||||
|
||||
### Metrics
|
||||
|
||||
| Key | Default | Meaning | Type | Since |
|
||||
|---------------------------------|----------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|-------|
|
||||
| kyuubi.metrics.console.interval | PT5S | How often should report metrics to console | duration | 1.2.0 |
|
||||
| kyuubi.metrics.enabled | true | Set to true to enable kyuubi metrics system | boolean | 1.2.0 |
|
||||
| kyuubi.metrics.json.interval | PT5S | How often should report metrics to JSON file | duration | 1.2.0 |
|
||||
| kyuubi.metrics.json.location | metrics | Where the JSON metrics file located | string | 1.2.0 |
|
||||
| kyuubi.metrics.prometheus.path | /metrics | URI context path of prometheus metrics HTTP server | string | 1.2.0 |
|
||||
| kyuubi.metrics.prometheus.port | 10019 | Prometheus metrics HTTP server port | int | 1.2.0 |
|
||||
| kyuubi.metrics.reporters | JSON | A comma-separated list for all metrics reporters<ul> <li>CONSOLE - ConsoleReporter which outputs measurements to CONSOLE periodically.</li> <li>JMX - JmxReporter which listens for new metrics and exposes them as MBeans.</li> <li>JSON - JsonReporter which outputs measurements to json file periodically.</li> <li>PROMETHEUS - PrometheusReporter which exposes metrics in Prometheus format.</li> <li>SLF4J - Slf4jReporter which outputs measurements to system log periodically.</li></ul> | set | 1.2.0 |
|
||||
| kyuubi.metrics.slf4j.interval | PT5S | How often should report metrics to SLF4J logger | duration | 1.2.0 |
|
||||
| Key | Default | Meaning | Type | Since |
|
||||
|---------------------------------|------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|-------|
|
||||
| kyuubi.metrics.console.interval | PT5S | How often should report metrics to console | duration | 1.2.0 |
|
||||
| kyuubi.metrics.enabled | true | Set to true to enable kyuubi metrics system | boolean | 1.2.0 |
|
||||
| kyuubi.metrics.json.interval | PT5S | How often should report metrics to JSON file | duration | 1.2.0 |
|
||||
| kyuubi.metrics.json.location | metrics | Where the JSON metrics file located | string | 1.2.0 |
|
||||
| kyuubi.metrics.prometheus.path | /metrics | URI context path of prometheus metrics HTTP server | string | 1.2.0 |
|
||||
| kyuubi.metrics.prometheus.port | 10019 | Prometheus metrics HTTP server port | int | 1.2.0 |
|
||||
| kyuubi.metrics.reporters | PROMETHEUS | A comma-separated list for all metrics reporters<ul> <li>CONSOLE - ConsoleReporter which outputs measurements to CONSOLE periodically.</li> <li>JMX - JmxReporter which listens for new metrics and exposes them as MBeans.</li> <li>JSON - JsonReporter which outputs measurements to json file periodically.</li> <li>PROMETHEUS - PrometheusReporter which exposes metrics in Prometheus format.</li> <li>SLF4J - Slf4jReporter which outputs measurements to system log periodically.</li></ul> | set | 1.2.0 |
|
||||
| kyuubi.metrics.slf4j.interval | PT5S | How often should report metrics to SLF4J logger | duration | 1.2.0 |
|
||||
|
||||
### Operation
|
||||
|
||||
|
||||
@ -24,6 +24,9 @@
|
||||
To restore previous behavior, set `kyuubi.metadata.store.jdbc.database.type=DERBY` and
|
||||
`kyuubi.metadata.store.jdbc.url=jdbc:derby:memory:kyuubi_state_store_db;create=true`.
|
||||
|
||||
* Since Kyuubi 1.8, PROMETHEUS is changed as the default metrics reporter. To restore previous behavior,
|
||||
set `kyuubi.metrics.reporters=JSON`.
|
||||
|
||||
## Upgrading from Kyuubi 1.7.1 to 1.7.2
|
||||
|
||||
* Since Kyuubi 1.7.2, for Kyuubi BeeLine, please use `--python-mode` option to run python code or script.
|
||||
|
||||
@ -46,7 +46,7 @@ object MetricsConf {
|
||||
.transformToUpperCase
|
||||
.toSet()
|
||||
.checkValues(ReporterType)
|
||||
.createWithDefault(Set(JSON.toString))
|
||||
.createWithDefault(Set(PROMETHEUS.toString))
|
||||
|
||||
val METRICS_CONSOLE_INTERVAL: ConfigEntry[Long] = buildConf("kyuubi.metrics.console.interval")
|
||||
.doc("How often should report metrics to console")
|
||||
|
||||
@ -34,6 +34,7 @@ import org.apache.kyuubi._
|
||||
import org.apache.kyuubi.client.util.BatchUtils._
|
||||
import org.apache.kyuubi.config.KyuubiConf
|
||||
import org.apache.kyuubi.events.ServerEventHandlerRegister
|
||||
import org.apache.kyuubi.metrics.MetricsConf
|
||||
import org.apache.kyuubi.operation.HiveJDBCTestHelper
|
||||
import org.apache.kyuubi.operation.OperationState._
|
||||
import org.apache.kyuubi.server.KyuubiServer
|
||||
@ -56,6 +57,7 @@ class ServerJsonLoggingEventHandlerSuite extends WithKyuubiServer with HiveJDBCT
|
||||
.set(KyuubiConf.SERVER_EVENT_JSON_LOG_PATH, serverLogRoot)
|
||||
.set(KyuubiConf.ENGINE_SPARK_EVENT_LOGGERS, Seq("JSON"))
|
||||
.set(KyuubiConf.ENGINE_EVENT_JSON_LOG_PATH, engineLogRoot)
|
||||
.set(MetricsConf.METRICS_REPORTERS, Set.empty[String])
|
||||
}
|
||||
|
||||
override protected def jdbcUrl: String = getJdbcUrl
|
||||
|
||||
Loading…
Reference in New Issue
Block a user