kyuubi/docker/playground
Cheng Pan 86645ad54a
[KYUUBI #3881] Upgrade playground components to latest versions
### _Why are the changes needed?_

This PR upgrades Kyuubi playground components to latest versions, including Spark 3.3.1, Kyuubi 1.6.1-incubating, Iceberg 1.1.0, and reorganizes the files in catalog folders.

The images has been uploaded to Docker Hub.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [x] Add screenshots for manual tests if appropriate

```
cd docker/playground
docker compose up -d
```

```
docker exec -it kyuubi /opt/kyuubi/bin/beeline -u 'jdbc:hive2://0.0.0.0:10009/tpcds/tiny'
```

<img width="1416" alt="image" src="https://user-images.githubusercontent.com/26535726/204845474-b0dff173-d335-43bf-b61f-d8c34ee26ff8.png">

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3881 from pan3793/playground.

Closes #3881

a3d42377 [Cheng Pan] Upgrade playground components to latest versions

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-12-02 10:51:41 +08:00
..
conf [KYUUBI #3881] Upgrade playground components to latest versions 2022-12-02 10:51:41 +08:00
image [KYUUBI #3881] Upgrade playground components to latest versions 2022-12-02 10:51:41 +08:00
script [KYUUBI #3881] Upgrade playground components to latest versions 2022-12-02 10:51:41 +08:00
.env [KYUUBI #3881] Upgrade playground components to latest versions 2022-12-02 10:51:41 +08:00
build-image.sh [KYUUBI #3881] Upgrade playground components to latest versions 2022-12-02 10:51:41 +08:00
compose.yml [KYUUBI #3881] Upgrade playground components to latest versions 2022-12-02 10:51:41 +08:00
README.md [KYUUBI #3881] Upgrade playground components to latest versions 2022-12-02 10:51:41 +08:00

Playground

For Users

Setup

  1. Install Docker and Docker Compose;
  2. Go to docker/playground, and use docker compose up -d to run compose services as daemon;

Play

  1. Connect using beeline

docker exec -it kyuubi /opt/kyuubi/bin/beeline -u 'jdbc:hive2://0.0.0.0:10009/tpcds/tiny';

  1. Connect using DBeaver

Add a Kyuubi datasource with

  • connection url jdbc:hive2://0.0.0.0:10009/tpcds/tiny
  • username: anonymous
  • password: <empty>
  1. Use built-in dataset

Kyuubi supply some built-in dataset, after Kyuubi started, you can run the following command to load the different datasets:

  • For loading TPC-DS tiny dataset to spark_catalog.tpcds_tiny, run docker exec -it kyuubi /opt/kyuubi/bin/beeline -u 'jdbc:hive2://0.0.0.0:10009/' -f /opt/load_data/load-dataset-tpcds-tiny.sql
  • For loading TPC-H tiny dataset to spark_catalog.tpch_tiny, run docker exec -it kyuubi /opt/kyuubi/bin/beeline -u 'jdbc:hive2://0.0.0.0:10009/' -f /opt/load_data/load-dataset-tpch-tiny.sql

Access Service

  • MinIO: http://localhost:9001
  • PostgreSQL localhost:5432 (username: postgres, password: postgres)
  • Spark UI: http://localhost:4040 (available after Spark application launching by Kyuubi, port may be 4041, 4042... if you launch more than one Spark applications)

Shutdown

  1. Stop compose services by docker compose down;

For Maintainers

Build

  1. Build images docker/playground/build-image.sh;
  2. Optional to use buildx to build and publish cross-platform images BUILDX=1 docker/playground/build-image.sh;