### Why are the changes needed?
HTTP dialect ignores the database specified in the URL and uses the "default" instead.
### How was this patch tested?
Tested manually.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes#6906 from developster/pyhive-update1.
Closes#6905
6e21d7259 [Cheng Pan] Update python/pyhive/sqlalchemy_hive.py
ec7d4629e [Octavian Ciubotaru] [KYUUBI #6905] PyHive HTTP/HTTPS dialect to use the database name from url
Lead-authored-by: Octavian Ciubotaru <ociubotaru@developmentgateway.org>
Co-authored-by: Cheng Pan <pan3793@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
This pull request fixes#6485
## Describe Your Solution 🔧
Ignore uppercase and lowercase letters in table names when using regular expressions to match.
## Types of changes 🔖
- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Added unit tests when table names have capital letters.
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6605 from BruceWong96/fix-presto-regex.
Closes#6485
06f737f24 [Bruce Wong] Fix typos
93071754a [Bruce Wong] Added unit tests for table names with both upper and lower case letters
9837030a1 [Bruce Wong] fix table not found
Authored-by: Bruce Wong <603334301@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
This pull request fixes#6567
## Describe Your Solution 🔧
With `setuptools==72.0.0`, the `test` command was removed after being deprecated for a long time.
This completely breaks the installation of any packages using the `test` command, including PyHive.
This PR fixes PyHive to be compatible with `setuptools` 72+ again.
It seems that `setup.py test` wasn't used anywhere at all, which is why I just removed it.
## Types of changes 🔖
- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
```
$ pip install -e .
Obtaining file:///home/localstack/Repos/kyuubi/python
Preparing metadata (setup.py) ... error
error: subprocess-exited-with-error
× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [6 lines of output]
Traceback (most recent call last):
File "<string>", line 2, in <module>
File "<pip-setuptools-caller>", line 34, in <module>
File "/home/localstack/Repos/kyuubi/python/setup.py", line 4, in <module>
from setuptools.command.test import test as TestCommand
ModuleNotFoundError: No module named 'setuptools.command.test'
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details
```
#### Behavior With This Pull Request 🎉
```
$ pip install --upgrade setuptools
Requirement already satisfied: setuptools in .../.pyenv/versions/3.11.5/lib/python3.11/site-packages (72.0.0)
$ pip install -e .
Obtaining file:///home/.../repos/kyuubi/python
Preparing metadata (setup.py) ... done
Collecting future (from PyHive==0.7.0)
Obtaining dependency information for future from ae30dadffc/future-1.0.0-py3-none-any.whl.metadata
Using cached future-1.0.0-py3-none-any.whl.metadata (4.0 kB)
Requirement already satisfied: python-dateutil in /home/.../.pyenv/versions/3.11.5/lib/python3.11/site-packages (from PyHive==0.7.0) (2.8.2)
Requirement already satisfied: six>=1.5 in /home/.../.pyenv/versions/3.11.5/lib/python3.11/site-packages (from python-dateutil->PyHive==0.7.0) (1.16.0)
Using cached future-1.0.0-py3-none-any.whl (491 kB)
Installing collected packages: future, PyHive
Running setup.py develop for PyHive
Successfully installed PyHive-0.7.0 future-1.0.0
[notice] A new release of pip is available: 23.2.1 -> 24.1.2
[notice] To update, run: pip install --upgrade pip
```
#### Related Unit Tests
None
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6568 from alexrashed/fix-pyhive-setuptools-72.
Closes#6567
acfb80903 [Alexander Rashed] remove usage of setuptools.command.test
Authored-by: Alexander Rashed <alexander.rashed@localstack.cloud>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
This pull request fixes#6489
## Describe Your Solution 🔧
After my investigation, I found the bug and solution.
The function get_table_names returns an incorrect value when I used Superset to connect to Kyuubi for Spark SQL.
[get_table_names](https://github.com/apache/kyuubi/blob/master/python/pyhive/sqlalchemy_hive.py#L380)
The following code is used to connect to hive directly.
`return [row[0] for row in connection.execute(text(query))]`
Because The following value is returned when the Hive is connected.
show tables in default :
[('student',), ('student_scores',)]
The following code is used to connect to Kyuubi.
`return [row[1] for row in connection.execute(text(query))]`
Because The following value is returned when the Kyuubi is connected.
show tables in default :
[('default', 'employees', False), ('default', 'student', False), ('default', 'student_scores', False)]
So, for the difference in return value, I modified the code.
And I test them in Superset. The code works.
Hive
<img width="1214" alt="image" src="https://github.com/apache/kyuubi/assets/29974394/9048b21d-053e-4b5d-be35-ba29d3bd6848">
Kyuubi
<img width="1085" alt="image" src="https://github.com/apache/kyuubi/assets/29974394/d600dfed-1127-41ea-a0bf-ca662a5487df">
Spark SQL also works properly.
<img width="1199" alt="image" src="https://github.com/apache/kyuubi/assets/29974394/7026e39e-6d63-473d-9e43-eeab580719ea">
## Types of changes 🔖
- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6490 from BruceWong96/branch-kyuubi-6489.
Closes#6489
94a52c0e5 [wenjie.wang01] add else branch.
8ab20becf [wenjie.wang01] fix bug for function get_table_names.
136c7b795 [wenjie.wang01] fix bug for function get_table_names.
Authored-by: wenjie.wang01 <wenjie.wang01@liulishuo.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
…pendencies
# 🔍 Description
## Issue References 🔗
This pull request fixes#6445
## Describe Your Solution 🔧
use `hive-pure-sasl` instead of `hive_pure_sasl` for extra name for optional distribution dependencies.
this avoid potential `WARNING: pyhive x.y.z does not provide the extra 'hive-pure-sasl'` and missing dependencies when a package depends on `pyhive[hive_pure_sasl]` and python build system choose to normalize it to `pyhive[hive-pure-saal]`.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
a package depends on `pyhive[hive_pure_sasl]` may complain `WARNING: pyhive x.y.z does not provide the extra 'hive-pure-sasl'` and missing dependencies to support hive feature.
#### Behavior With This Pull Request 🎉
based on my test on our internal pypi, users use `pyhive[hive_pure_sasl]` are not affected by this change. But we should update README when we release a new version of pyhive.
#### Related Unit Tests
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6454 from camper42/python.
Closes#6445
d82e5cae9 [camper42] [KYUUBI #6445] use normalized extra name for optional distribution dependencies
Authored-by: camper42 <camper.xlii@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
This pull request enables running hive test cases in python client, however there's one trivial case not covered yet and two others require a proper container setup
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
Hive test disabled in #6343
#### Behavior With This Pull Request 🎉
Can cover hive test cases
#### Related Unit Tests
No
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6381 from sudohainguyen/ci/hive.
Closes#6281
a861382b1 [Harry] [KYUUBI #6281][PY] Enable hive test in python client
Authored-by: Harry <quanghai.ng1512@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
This pull request fixes#6281
Following up to #6343 , this PR adds `pytest-xdist` plugin to speed up python client unit testing
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
Running tests usually takes ~10 mins
#### Behavior With This Pull Request 🎉
Running tests can take up to 5 mins
#### Related Unit Tests
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6369 from sudohainguyen/ci/speed-up-test.
Closes#6281
f0ce55719 [Harry] [KYUUBI #6281][PY] Speed up testing with xdist plugin
Authored-by: Harry <quanghai.ng1512@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
This pull request fixes#6281
## Describe Your Solution 🔧
The change initialize a CI job to run unit testing on python client, including:
- Set up Github Action based on docker-compose
- Update test cases and test succeeded for dialect `presto` and `trino`
- Temporary disabled hive related test due to test cases are not valid, not about connection
- Update dev dependencies to support python 3.10
- Speed up testing with `pytest-xdist` plugin
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
Not able to ran unit test in local and on CI
#### Behavior With This Pull Request 🎉
Able to run and partially cover a couple of test cases
#### Related Unit Tests
No
## Additional notes
Next action is about fixing failing test cases or considering skipping some of them if necessary
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6343 from sudohainguyen/ci/init.
Closes#6281
682e575c4 [Harry] Remove xdist out of scope
dc42ca1ff [Harry] Pin pytest packages version
469f1d955 [Harry] Pin ubuntu version
00cef476a [Harry] Use v4 checkout action
96ef83148 [Harry] Remove unnecessary steps
732344a2c [Harry] Add step to tear down containers
1e2c2481a [Harry] Resolved trino and presto test
5b33e3924 [Harry] Make tests runnable
1be033ba3 [Harry] Remove randome flag which causes failed test run
2bc6dc036 [Harry] Switch action setup provider to docker
ea2a76319 [Harry] Initialize github action for python unit testing
Authored-by: Harry <quanghai.ng1512@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
This is the follow-up of #5686, renaming `./pyhive` to `./python`, and also adding `**/python/*` to RAT exclusion list temporarily.
"PyHive" may not be a suitable name after being part of Apache Kyuubi, let's use a generic dir name `python`, and discuss the official name later(we probably keep the code at `./python` eventually).
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Recover RAT checked.
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6279 from pan3793/pyhive-1.
Closes#5686
42d338e71 [Cheng Pan] [KYUUBI #5686][FOLLOWUP] Rename pyhive to python
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
