apache/kyuubi
1
0
Fork 0
Code Issues Pull Requests Actions 5 Packages Projects Releases Wiki Activity
8ab4763ce4
kyuubi/docker/playground
History
Cheng Pan 0fe00d9c5b [KYUUBI #5640] Upgrade playground to Kyuubi 1.8.0 and Spark 3.4.1 
### _Why are the changes needed?_

Promote [1.8.0](https://kyuubi.apache.org/release/1.8.0.html), the docker images already uploaded to DockerHub

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [x] Add screenshots for manual tests if appropriate

Go to `docker/playground`, and use `docker compose up -d` to run compose services as daemon

```
docker exec -it kyuubi /opt/kyuubi/bin/beeline -u 'jdbc:hive2://0.0.0.0:10009/tpcds/tiny'
```

```
0: jdbc:hive2://0.0.0.0:10009/tpcds/tiny> select kyuubi_version();
2023-11-07 06:12:57.332 INFO org.apache.kyuubi.operation.ExecuteStatement: Processing anonymous's query[41fe38d5-dc3f-4fc3-8a25-3e3c4200ce31]: PENDING_STATE -> RUNNING_STATE, statement:
select kyuubi_version()
23/11/07 06:12:57 INFO ExecuteStatement: Processing anonymous's query[41fe38d5-dc3f-4fc3-8a25-3e3c4200ce31]: PENDING_STATE -> RUNNING_STATE, statement:
select kyuubi_version()
23/11/07 06:12:57 INFO ExecuteStatement:
           Spark application name: kyuubi_USER_SPARK_SQL_anonymous_default_1d81daf7-9136-4b0d-97f6-2cd71bcb74fb
                 application ID: local-1699337554479
                 application web UI: http://0.0.0.0:4040
                 master: local[*]
                 deploy mode: client
                 version: 3.4.1
           Start time: 2023-11-07T06:12:33.491
           User: anonymous
23/11/07 06:12:57 INFO ExecuteStatement: Execute in incremental collect mode
23/11/07 06:12:57 INFO CodeGenerator: Code generated in 14.080013 ms
23/11/07 06:12:57 INFO ExecuteStatement: Processing anonymous's query[41fe38d5-dc3f-4fc3-8a25-3e3c4200ce31]: RUNNING_STATE -> FINISHED_STATE, time taken: 0.126 seconds
2023-11-07 06:12:57.461 INFO org.apache.kyuubi.operation.ExecuteStatement: Query[41fe38d5-dc3f-4fc3-8a25-3e3c4200ce31] in FINISHED_STATE
2023-11-07 06:12:57.461 INFO org.apache.kyuubi.operation.ExecuteStatement: Processing anonymous's query[41fe38d5-dc3f-4fc3-8a25-3e3c4200ce31]: RUNNING_STATE -> FINISHED_STATE, time taken: 0.129 seconds
+-------------------+
| kyuubi_version()  |
+-------------------+
| 1.8.0             |
+-------------------+
1 row selected (0.329 seconds)
0: jdbc:hive2://0.0.0.0:10009/tpcds/tiny>
```

- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request

### _Was this patch authored or co-authored using generative AI tooling?_

No

Closes #5640 from pan3793/play-180.

Closes #5640

f8c6cf19d [Cheng Pan] Upgrade playground to Kyuubi 1.8.0 and Spark 3.4.1

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2023-11-08 10:44:20 +08:00
..
conf [KYUUBI #5365] Don't use Log4j2's extended throwable conversion pattern in default logging configurations 2023-10-11 21:41:22 +08:00
image [KYUUBI #5266] Upgrade playground to Kyuubi 1.7.1 2023-09-11 11:13:05 +00:00
script
.env [KYUUBI #5640] Upgrade playground to Kyuubi 1.8.0 and Spark 3.4.1 2023-11-08 10:44:20 +08:00
build-image.sh
compose.yml [KYUUBI #5266] Upgrade playground to Kyuubi 1.7.1 2023-09-11 11:13:05 +00:00
README.md [KYUUBI #4312] [DOCS] Include **/README.md in markdown style check 2023-02-14 02:23:32 +08:00

README.md

Playground

For Users

Setup

  1. Install Docker and Docker Compose;
  2. Go to docker/playground, and use docker compose up -d to run compose services as daemon;

Play

  1. Connect using beeline

docker exec -it kyuubi /opt/kyuubi/bin/beeline -u 'jdbc:hive2://0.0.0.0:10009/tpcds/tiny';

  1. Connect using DBeaver

Add a Kyuubi datasource with

  • connection url jdbc:hive2://0.0.0.0:10009/tpcds/tiny
  • username: anonymous
  • password: <empty>
  1. Use built-in dataset

Kyuubi supply some built-in dataset, after Kyuubi started, you can run the following command to load the different datasets:

  • For loading TPC-DS tiny dataset to spark_catalog.tpcds_tiny, run docker exec -it kyuubi /opt/kyuubi/bin/beeline -u 'jdbc:hive2://0.0.0.0:10009/' -f /opt/load_data/load-dataset-tpcds-tiny.sql
  • For loading TPC-H tiny dataset to spark_catalog.tpch_tiny, run docker exec -it kyuubi /opt/kyuubi/bin/beeline -u 'jdbc:hive2://0.0.0.0:10009/' -f /opt/load_data/load-dataset-tpch-tiny.sql

Access Service

  • MinIO: http://localhost:9001
  • PostgreSQL localhost:5432 (username: postgres, password: postgres)
  • Spark UI: http://localhost:4040 (available after Spark application launching by Kyuubi, port may be 4041, 4042... if you launch more than one Spark applications)

Shutdown

  1. Stop compose services by docker compose down;

For Maintainers

Build

  1. Build images docker/playground/build-image.sh;
  2. Optional to use buildx to build and publish cross-platform images BUILDX=1 docker/playground/build-image.sh;