d54c884124
# 🔍 Description ## Issue References 🔗 This pull request fixes #5944 ## Describe Your Solution 🔧 Add prometheus and grafana docker image to docker compose yaml, and by defaults, prometheus will get kyuubi server's metrics and grafana set prometheus as default datasource. Help developers quickly build a dashboard development environment. ## Types of changes 🔖 - [ ] Bugfix (non-breaking change which fixes an issue) - [x] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) ## Test Plan 🧪 In local test - [x] all container start success - [x] By defaults, prometheus get kyuubi's metrics - [x] Grafana take the prometheus as default datasource #### Behavior Without This Pull Request ⚰️ #### Behavior With This Pull Request 🎉 #### Related Unit Tests --- # Checklist 📝 - [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html) **Be nice. Be informative.** Closes #5945 from zwangsheng/KYUUBI#5944. Closes #5944 031deedeb [Binjie Yang] Update docker/playground/README.md dbf41e37d [zwangsheng] declare metrics related config e6efe3071 [zwangsheng] make grafana as anonymous 404dd42ff [zwangsheng] fix comments f7ca4ae1c [zwangsheng] fix prometheus 089f8d08c [zwangsheng] eof 68720788e [zwangsheng] add license header cfa6edfef [zwangsheng] [KYUUBI #5944][Improvement] Introduce Prometheus and Grafana for Kyuubi Playground Lead-authored-by: zwangsheng <binjieyang@apache.org> Co-authored-by: Binjie Yang <binjieyang@apache.org> Signed-off-by: Cheng Pan <chengpan@apache.org>
1.7 KiB
1.7 KiB
Playground
For Users
Setup
- Install Docker and Docker Compose;
- Go to
docker/playground, and usedocker compose up -dto run compose services as daemon;
Play
- Connect using
beeline
docker exec -it kyuubi /opt/kyuubi/bin/beeline -u 'jdbc:hive2://0.0.0.0:10009/tpcds/tiny';
- Connect using DBeaver
Add a Kyuubi datasource with
- connection url
jdbc:hive2://0.0.0.0:10009/tpcds/tiny - username:
anonymous - password:
<empty>
- Use built-in dataset
Kyuubi supply some built-in dataset, after Kyuubi started, you can run the following command to load the different datasets:
- For loading TPC-DS tiny dataset to
spark_catalog.tpcds_tiny, rundocker exec -it kyuubi /opt/kyuubi/bin/beeline -u 'jdbc:hive2://0.0.0.0:10009/' -f /opt/load_data/load-dataset-tpcds-tiny.sql - For loading TPC-H tiny dataset to
spark_catalog.tpch_tiny, rundocker exec -it kyuubi /opt/kyuubi/bin/beeline -u 'jdbc:hive2://0.0.0.0:10009/' -f /opt/load_data/load-dataset-tpch-tiny.sql
Access Service
- MinIO: http://localhost:9001
- PostgreSQL localhost:5432 (username: postgres, password: postgres)
- Spark UI: http://localhost:4040 (available after Spark application launching by Kyuubi, port may be 4041, 4042... if you launch more than one Spark applications)
- Prometheus: http://localhost:9090
- Grafana: http://localhost:3000 (username: admin, password: admin)
Shutdown
- Stop compose services by
docker compose down;
For Maintainers
Build
- Build images
docker/playground/build-image.sh; - Optional to use
buildxto build and publish cross-platform imagesBUILDX=1 docker/playground/build-image.sh;