Commit Graph

6 Commits

Author SHA1 Message Date
wenjie.wang01
a0b9873f81
[KYUUBI #6489] [PYTHON] PyKyuubi get_table_names also supports Spark SQL dialect
# 🔍 Description
## Issue References 🔗

This pull request fixes #6489

## Describe Your Solution 🔧

After my investigation, I found the bug and solution.
The function get_table_names returns an incorrect value when I used Superset to connect to Kyuubi for Spark SQL.
[get_table_names](https://github.com/apache/kyuubi/blob/master/python/pyhive/sqlalchemy_hive.py#L380)

The following code is used to connect to hive directly.
`return [row[0] for row in connection.execute(text(query))]`

Because The following value is returned when the Hive is connected.

show tables in default :
[('student',), ('student_scores',)]

The following code is used to connect to Kyuubi.
`return [row[1] for row in connection.execute(text(query))]`

Because The following value is returned when the Kyuubi is connected.

show tables in default :
[('default', 'employees', False), ('default', 'student', False), ('default', 'student_scores', False)]

So, for the difference in return value, I modified the code.

And I test them in Superset. The code works.

Hive
<img width="1214" alt="image" src="https://github.com/apache/kyuubi/assets/29974394/9048b21d-053e-4b5d-be35-ba29d3bd6848">

Kyuubi
<img width="1085" alt="image" src="https://github.com/apache/kyuubi/assets/29974394/d600dfed-1127-41ea-a0bf-ca662a5487df">

Spark SQL also works properly.
<img width="1199" alt="image" src="https://github.com/apache/kyuubi/assets/29974394/7026e39e-6d63-473d-9e43-eeab580719ea">

## Types of changes 🔖

- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6490 from BruceWong96/branch-kyuubi-6489.

Closes #6489

94a52c0e5 [wenjie.wang01] add else branch.
8ab20becf [wenjie.wang01] fix bug for function get_table_names.
136c7b795 [wenjie.wang01] fix bug for function get_table_names.

Authored-by: wenjie.wang01 <wenjie.wang01@liulishuo.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2024-06-21 19:03:43 +08:00
camper42
96d8b8e2de
[KYUUBI #6445] Normalize extra name for optional Python distribution dependencies
…pendencies

# 🔍 Description
## Issue References 🔗

This pull request fixes #6445

## Describe Your Solution 🔧

use `hive-pure-sasl` instead of `hive_pure_sasl` for extra name for optional distribution dependencies.

this avoid potential `WARNING: pyhive x.y.z does not provide the extra 'hive-pure-sasl'` and missing dependencies when a package depends on `pyhive[hive_pure_sasl]` and python build system choose to normalize it to `pyhive[hive-pure-saal]`.

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

a package depends on `pyhive[hive_pure_sasl]` may complain `WARNING: pyhive x.y.z does not provide the extra 'hive-pure-sasl'` and missing dependencies to support hive feature.

#### Behavior With This Pull Request 🎉

based on my test on our internal pypi, users use `pyhive[hive_pure_sasl]` are not affected by this change. But we should update README when we release a new version of pyhive.

#### Related Unit Tests

---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6454 from camper42/python.

Closes #6445

d82e5cae9 [camper42] [KYUUBI #6445] use normalized extra name for optional distribution dependencies

Authored-by: camper42 <camper.xlii@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2024-06-06 15:06:37 +08:00
Harry
06af125b9f
[KYUUBI #6281][PY] Enable hive test in python client
# 🔍 Description
## Issue References 🔗

This pull request enables running hive test cases in python client, however there's one trivial case not covered yet and two others require a proper container setup

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️
Hive test disabled in #6343

#### Behavior With This Pull Request 🎉
Can cover hive test cases

#### Related Unit Tests
No

---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6381 from sudohainguyen/ci/hive.

Closes #6281

a861382b1 [Harry] [KYUUBI #6281][PY] Enable hive test in python client

Authored-by: Harry <quanghai.ng1512@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2024-05-15 14:55:44 +08:00
Harry
78e104d18b
[KYUUBI #6281][PY] Speed up testing with xdist plugin
# 🔍 Description
## Issue References 🔗

This pull request fixes #6281

Following up to #6343 , this PR adds `pytest-xdist` plugin to speed up python client unit testing

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️
Running tests usually takes ~10 mins

#### Behavior With This Pull Request 🎉
Running tests can take up to 5 mins

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6369 from sudohainguyen/ci/speed-up-test.

Closes #6281

f0ce55719 [Harry] [KYUUBI #6281][PY] Speed up testing with xdist plugin

Authored-by: Harry <quanghai.ng1512@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2024-05-08 11:43:39 +08:00
Harry
9075fbb623
[KYUUBI #6281][PY] Initialize github action for python unit testing
# 🔍 Description
## Issue References 🔗

This pull request fixes #6281

## Describe Your Solution 🔧

The change initialize a CI job to run unit testing on python client, including:
- Set up Github Action based on docker-compose
- Update test cases and test succeeded for dialect `presto` and `trino`
- Temporary disabled hive related test due to test cases are not valid, not about connection
- Update dev dependencies to support python 3.10
- Speed up testing with `pytest-xdist` plugin

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️
Not able to ran unit test in local and on CI

#### Behavior With This Pull Request 🎉
Able to run and partially cover a couple of test cases

#### Related Unit Tests
No

## Additional notes
Next action is about fixing failing test cases or considering skipping some of them if necessary

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6343 from sudohainguyen/ci/init.

Closes #6281

682e575c4 [Harry] Remove xdist out of scope
dc42ca1ff [Harry] Pin pytest packages version
469f1d955 [Harry] Pin ubuntu version
00cef476a [Harry] Use v4 checkout action
96ef83148 [Harry] Remove unnecessary steps
732344a2c [Harry] Add step to tear down containers
1e2c2481a [Harry] Resolved trino and presto test
5b33e3924 [Harry] Make tests runnable
1be033ba3 [Harry] Remove randome flag which causes failed test run
2bc6dc036 [Harry] Switch action setup provider to docker
ea2a76319 [Harry] Initialize github action for python unit testing

Authored-by: Harry <quanghai.ng1512@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2024-05-07 18:05:03 +08:00
Cheng Pan
f8c7b93f55
[KYUUBI #5686][FOLLOWUP] Rename pyhive to python
# 🔍 Description

This is the follow-up of #5686, renaming `./pyhive` to `./python`, and also adding `**/python/*` to RAT exclusion list temporarily.

"PyHive" may not be a suitable name after being part of Apache Kyuubi, let's use a generic dir name `python`, and discuss the official name later(we probably keep the code at `./python` eventually).

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

Recover RAT checked.

---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6279 from pan3793/pyhive-1.

Closes #5686

42d338e71 [Cheng Pan] [KYUUBI #5686][FOLLOWUP] Rename pyhive to python

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2024-04-09 20:30:02 +08:00