9075fbb623
# 🔍 Description ## Issue References 🔗 This pull request fixes #6281 ## Describe Your Solution 🔧 The change initialize a CI job to run unit testing on python client, including: - Set up Github Action based on docker-compose - Update test cases and test succeeded for dialect `presto` and `trino` - Temporary disabled hive related test due to test cases are not valid, not about connection - Update dev dependencies to support python 3.10 - Speed up testing with `pytest-xdist` plugin ## Types of changes 🔖 - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) ## Test Plan 🧪 #### Behavior Without This Pull Request ⚰️ Not able to ran unit test in local and on CI #### Behavior With This Pull Request 🎉 Able to run and partially cover a couple of test cases #### Related Unit Tests No ## Additional notes Next action is about fixing failing test cases or considering skipping some of them if necessary --- # Checklist 📝 - [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html) **Be nice. Be informative.** Closes #6343 from sudohainguyen/ci/init. Closes #6281 682e575c4 [Harry] Remove xdist out of scope dc42ca1ff [Harry] Pin pytest packages version 469f1d955 [Harry] Pin ubuntu version 00cef476a [Harry] Use v4 checkout action 96ef83148 [Harry] Remove unnecessary steps 732344a2c [Harry] Add step to tear down containers 1e2c2481a [Harry] Resolved trino and presto test 5b33e3924 [Harry] Make tests runnable 1be033ba3 [Harry] Remove randome flag which causes failed test run 2bc6dc036 [Harry] Switch action setup provider to docker ea2a76319 [Harry] Initialize github action for python unit testing Authored-by: Harry <quanghai.ng1512@gmail.com> Signed-off-by: Cheng Pan <chengpan@apache.org>
62 lines
1.5 KiB
YAML
62 lines
1.5 KiB
YAML
version: "3"
|
|
|
|
services:
|
|
namenode:
|
|
image: bde2020/hadoop-namenode:2.0.0-hadoop2.7.4-java8
|
|
volumes:
|
|
- namenode:/hadoop/dfs/name
|
|
environment:
|
|
- CLUSTER_NAME=test
|
|
env_file:
|
|
- hadoop-hive.env
|
|
ports:
|
|
- "50070:50070"
|
|
datanode:
|
|
image: bde2020/hadoop-datanode:2.0.0-hadoop2.7.4-java8
|
|
volumes:
|
|
- datanode:/hadoop/dfs/data
|
|
env_file:
|
|
- hadoop-hive.env
|
|
environment:
|
|
SERVICE_PRECONDITION: "namenode:50070"
|
|
ports:
|
|
- "50075:50075"
|
|
hive-server:
|
|
image: bde2020/hive:2.3.2-postgresql-metastore
|
|
env_file:
|
|
- hadoop-hive.env
|
|
volumes:
|
|
- ../scripts:/opt/hive/scripts
|
|
environment:
|
|
HIVE_CORE_CONF_javax_jdo_option_ConnectionURL: "jdbc:postgresql://hive-metastore/metastore"
|
|
SERVICE_PRECONDITION: "hive-metastore:9083"
|
|
ports:
|
|
- "10000:10000"
|
|
hive-metastore:
|
|
image: bde2020/hive:2.3.2-postgresql-metastore
|
|
env_file:
|
|
- hadoop-hive.env
|
|
command: /opt/hive/bin/hive --service metastore
|
|
environment:
|
|
SERVICE_PRECONDITION: "namenode:50070 datanode:50075 hive-metastore-postgresql:5432"
|
|
ports:
|
|
- "9083:9083"
|
|
hive-metastore-postgresql:
|
|
image: bde2020/hive-metastore-postgresql:2.3.0
|
|
presto-coordinator:
|
|
image: shawnzhu/prestodb:0.181
|
|
ports:
|
|
- "8080:8080"
|
|
volumes:
|
|
- ./conf/presto/:/etc/presto
|
|
trino:
|
|
image: trinodb/trino:351
|
|
ports:
|
|
- "18080:18080"
|
|
volumes:
|
|
- ./conf/trino:/etc/trino
|
|
|
|
volumes:
|
|
namenode:
|
|
datanode:
|