# 🔍 Description
## Issue References 🔗
## Describe Your Solution 🔧
Add docs for spark-3.4 and spark-3.5 as supported Spark profiles.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
<img width="774" alt="image" src="https://github.com/apache/kyuubi/assets/1935105/29b0690d-574f-4b3c-ac70-50d5fb67a75e">
#### Behavior With This Pull Request 🎉
<img width="766" alt="image" src="https://github.com/apache/kyuubi/assets/1935105/58af356b-0f9c-4fe9-8f2b-8769ccebcc3a">
#### Related Unit Tests
---
# Checklists
## 📝 Author Self Checklist
- [x] My code follows the [style guidelines](https://kyuubi.readthedocs.io/en/master/contributing/code/style.html) of this project
- [x] I have performed a self-review
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
## 📝 Committer Pre-Merge Checklist
- [ ] Pull request title is okay.
- [ ] No license issues.
- [ ] Milestone correctly set?
- [ ] Test coverage is ok
- [ ] Assignees are selected.
- [ ] Minimum number of approvals
- [ ] No changes are requested
**Be nice. Be informative.**
Closes#5801 from bowenliang123/doc-spark-profiles.
Closes#5801
35254df09 [liangbowen] style
2b55824fd [liangbowen] add spark-3.4 and spark-3.5 to the docs of building from code
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
# 🔍 Description
## Issue References 🔗
This pull request fixes#5572
## Describe Your Solution 🔧
Introduce Kyuubi Web UI Engine Page as the screenshot shown.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
<img width="835" alt="Screenshot 2023-11-27 at 10 39 55" src="https://github.com/apache/kyuubi/assets/32693629/363426e8-9808-4fd7-a711-2f0f80164147">
#### Related Unit Tests
---
# Checklists
## 📝 Author Self Checklist
- [x] My code follows the [style guidelines](https://kyuubi.readthedocs.io/en/master/contributing/code/style.html) of this project
- [x] I have performed a self-review
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
## 📝 Committer Pre-Merge Checklist
- [x] Pull request title is okay.
- [x] No license issues.
- [x] Milestone correctly set?
- [x] Test coverage is ok
- [x] Assignees are selected.
- [x] Minimum number of approvals
- [x] No changes are requested
**Be nice. Be informative.**
Closes#5762 from zhaohehuhu/dev-1123.
Closes#5762
4a85af03e [hezhao2] compress picture
5aed43e63 [hezhao2] refactor
16d2d2f83 [hezhao2] reformat
ad9c30bf1 [hezhao2] Introduce Kyuubi Web UI Engine Page
Authored-by: hezhao2 <hezhao2@cisco.com>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
To close https://github.com/apache/kyuubi/issues/5464.
To support JDBC engine use MySQL Dialect (kyuubi.engine.jdbc.type=mysql).
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No.
Closes#5588 from Kwafoor/kyuubi_5464.
Closes#5464
1019a6118 [wangjunbo] [KYUUBI #5464]rename function name `getProviderClass` to `getDriverClass`
9901bbad4 [wangjunbo] [KYUUBI #5464]handle properly to keep compatiblity
b33d79ed2 [wangjunbo] [KYUUBI #5464]handle properly to keep compatiblity
86e6ee2b3 [wangjunbo] [KYUUBI #5464]handle properly to keep compatiblity
d76cb3275 [wangjunbo] [KYUUBI #5464]update the docs
4a1acffd1 [wangjunbo] [KYUUBI #5464]update the docs
1aff55ecd [wangjunbo] [KYUUBI #5464]update the docs of kyuubi.engine.type
84202ea0c [wangjunbo] [KYUUBI #5464] update the docs of kyuubi.engine.type
e3c1e94db [wangjunbo] [KYUUBI #5464] fix check
cdf820da0 [wangjunbo] [KYUUBI #5464] fix check
ff0f30ad8 [wangjunbo] [KYUUBI #5464] fix check
01321dc44 [wangjunbo] [KYUUBI #5464] JDBC Engine supports MySQL
756f5303c [wangjunbo] [KYUUBI #5464] JDBC Engine supports MySQL
Authored-by: wangjunbo <wangjunbo@qiyi.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
flink setting need additional prefix flink. to override settings
## Issue References 🔗
This pull request fixes#5737
## Describe Your Solution 🔧
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.
## Types of changes 🔖
- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklists
## 📝 Author Self Checklist
- [ ] My code follows the [style guidelines](https://kyuubi.readthedocs.io/en/master/contributing/code/style.html) of this project
- [ ] I have performed a self-review
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
## 📝 Committer Pre-Merge Checklist
- [x] Pull request title is okay.
- [x] No license issues.
- [x] Milestone correctly set?
- [ ] Test coverage is ok
- [x] Assignees are selected.
- [x] Minimum number of approvals
- [x] No changes are requested
**Be nice. Be informative.**
Closes#5745 from dik111/update-doc.
Closes#5737
27ac6c1db [yuwei zhan] Update AllKyuubiConfiguration about flink jdbc url
a1b1b2cba [yuwei zhan] Update settings.md
Lead-authored-by: Yuwei Zhan <195812208@qq.com>
Co-authored-by: yuwei zhan <195812208@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Describe Your Solution 🔧
A new feature introduced from #5714 supports kyuubi to clean up spark driver pods automatically, but all pod would be clean up without considering app's terminated state.
This PR make user can chose which pods should be delete by setting up a cleanup strategy.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Test locally.
---
# Checklists
## 📝 Author Self Checklist
- [x] My code follows the [style guidelines](https://kyuubi.readthedocs.io/en/master/contributing/code/style.html) of this project
- [x] I have performed a self-review
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
## 📝 Committer Pre-Merge Checklist
- [x] Pull request title is okay.
- [ ] No license issues.
- [ ] Milestone correctly set?
- [ ] Test coverage is ok
- [ ] Assignees are selected.
- [ ] Minimum number of approvals
- [ ] No changes are requested
**Be nice. Be informative.**
Closes#5728 from liaoyt/master.
Closes#5731
d2cc8cb55 [yeatsliao] regenerate docs
4caf8b152 [yeatsliao] rename conf 'KUBERNETES_SPARK_DELETE_DRIVER_POD_ON_TERMINATION' to 'KUBERNETES_SPARK_CLEANUP_TERMINATED_DRIVER_POD'
4d970fa14 [yeatsliao] [K8S] Support to cleanup the spark driver pod with specified clean up strategy
Authored-by: yeatsliao <liaoyt66066@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
Infer the batch user from session or metadata, user do not need to specify the proxy user anymore.
This pr also align the behavior of BatchesResource with that of SessionsResource and OperationsResource(no proxy user parameter).
For Kyuubi Batch, Session and Operation, these resources have the user attiribute.
So we only need to check whether the authentication user has the permission to access the resource.
## Issue References 🔗
This pull request fixes #
## Describe Your Solution 🔧
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklists
## 📝 Author Self Checklist
- [x] My code follows the [style guidelines](https://kyuubi.readthedocs.io/en/master/contributing/code/style.html) of this project
- [x] I have performed a self-review
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my feature works
- [x] New and existing unit tests pass locally with my changes
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
## 📝 Committer Pre-Merge Checklist
- [x] Pull request title is okay.
- [x] No license issues.
- [x] Milestone correctly set?
- [ ] Test coverage is ok
- [x] Assignees are selected.
- [x] Minimum number of approvals
- [x] No changes are requested
**Be nice. Be informative.**
Closes#5717 from turboFei/hive_server2_proxy_user.
Closes#5717
70ad7e76d [fwang12] comment
c721a751a [fwang12] ignore
da92bd5a1 [fwang12] fix ut
9a197d005 [fwang12] doc
c8ed5f9cf [fwang12] ut
cef9e329c [fwang12] do not use proxy user
Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: fwang12 <fwang12@ebay.com>
# 🔍 Description
This pr is to support to build the application state from pod container instead of pod.
The use case is that:
For spark on k8s, the pod contains two containers.
- one is the spark driver main container
- the another one is a sidecar container
If the spark driver main container completed but the sidecar container not, the batch application state is always RUNNING as the pod state is `NotReady`.
It is not expected.
So in this pr, I want to introduce a new option config to support to build application state from the pod container.
## Issue References 🔗
This pull request fixes #
## Describe Your Solution 🔧
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklists
## 📝 Author Self Checklist
- [x] My code follows the [style guidelines](https://kyuubi.readthedocs.io/en/master/contributing/code/style.html) of this project
- [x] I have performed a self-review
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
## 📝 Committer Pre-Merge Checklist
- [x] Pull request title is okay.
- [x] No license issues.
- [x] Milestone correctly set?
- [ ] Test coverage is ok
- [ ] Assignees are selected.
- [ ] Minimum number of approvals
- [ ] No changes are requested
**Be nice. Be informative.**
Closes#5711 from turboFei/k8s_pod_state.
Closes#5711
9e0575b5f [fwang12] save
eaec5d2bb [fwang12] comment
61ae5f723 [fwang12] comment
1eac84f2d [fwang12] save
bca19be3a [fwang12] nit
38115b752 [fwang12] [K8S] Support to build application state and error from pod container
Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: fwang12 <fwang12@ebay.com>
# 🔍 Description
## Describe Your Solution 🔧
As title, support to cleanup the application pod after application terminates for retain period
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Test locally.
---
# Checklists
## 📝 Author Self Checklist
- [x] My code follows the [style guidelines](https://kyuubi.readthedocs.io/en/master/contributing/code/style.html) of this project
- [x] I have performed a self-review
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
## 📝 Committer Pre-Merge Checklist
- [x] Pull request title is okay.
- [x] No license issues.
- [x] Milestone correctly set?
- [ ] Test coverage is ok
- [x] Assignees are selected.
- [x] Minimum number of approvals
- [ ] No changes are requested
**Be nice. Be informative.**
Closes#5714 from turboFei/kill_k8s_pod.
Closes#5714
1e0787868 [fwang12] doc
0c9ff1a44 [fwang12] cleanup pod
ab95d4c03 [fwang12] save
Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: fwang12 <fwang12@ebay.com>
# 🔍 Description
## Issue References 🔗
This pull request fixes#5701
## Describe Your Solution 🔧
This PR updates the PR template to ensure that PR authors provide informative PR descriptions for reviewers to speed up the review process.
It's also suitable for improving our code quality.
Besides, `docs/contributing/code/style.rst` is also changed to add a link to the doc style so that we can keep the PR template neat.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
See the screenshot below.
#### Behavior With This Pull Request 🎉
It's able to check it from the description of this PR directly.
#### Related Unit Tests
No
---
# Checklists
## 📝 Author Self Checklist
- [x] My code follows the [style guidelines](https://kyuubi.readthedocs.io/en/master/contributing/code/style.html) of this project
- [x] I have performed a self-review
- [ ] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my feature works
- [x] New and existing unit tests pass locally with my changes
- [x] This patch was authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
## 📝 Committer Pre-Merge Checklist
- [x] Pull request title is okay.
- [x] No license issues.
- [x] Milestone correctly set?
- [x] Test coverage is ok
- [x] Assignees are selected.
- [x] Minimum number of approvals
- [x] No changes are requested
**Be nice. Be informative.**
Closes#5702 from yaooqinn/pr.
Closes#5701
19bdea9a4 [Kent Yao] not
d7d9d8cfa [Kent Yao] Enhance PR Template to make PR desc informative
be026e4a6 [Kent Yao] Enhance PR Template to make PR desc informative
Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
close#5582
JDBC Engine supports configurable default fetchSize
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
NO
Closes#5614 from lsm1/branch-kyuubi-5582.
Closes#5582
dd58a6d51 [senmiaoliu] fix style
d0eb7c5dc [senmiaoliu] set jdbc fetch size in operation manager
4eee26b7b [senmiaoliu] rename conf key
4d8b74796 [senmiaoliu] add conf for JDBC engine default fetch size
Authored-by: senmiaoliu <senmiaoliu@trip.com>
Signed-off-by: Shaoyun Chen <csy@apache.org>
### _Why are the changes needed?_
We need a configuration to be an alternative to `hive.server2.proxy.user`. `hive.server2.proxy.user` is derived from the hive server, and it's perfect for keeping the behavior the same as the hive. But hive thrift protocol is only one of the FrontendServices. There are also other FrontendServices not necessary to follow `hive.server2.proxy.user`. So we need `kyuubi.proxy.user`.
In this commit, support for using the `kyuubi.proxy.user` configuration has been implemented in scenarios where the original `hive.server2.proxy.user` configuration was used.In the future, this feature will continue to be improved according to other scenarios where it is used.
Close#5683
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No.
Closes#5653 from mrtisttt/add-kyuubi-proxy-user.
Closes#5683
6d60e7632 [mrtisttt] Rollback unnecessary modifications
7298e4137 [mrtisttt] fix doc violations
19f6f4084 [mrtisttt] For API side, shorten the name of kyuubi.session.proxy.user and change it to proxyUser.
0fbcc704d [mrtisttt] Change kyuubi.proxy.user to kyuubi.session.proxy.user and improve documentation
c2a810ce2 [mrtisttt] fix format violations
a6413e8aa [mrtisttt] Compatible with scenarios where hive.server2.proxy.user is used
b687c21b9 [mrtisttt] Add kyuubi.proxy.user as an alternative to hive.server2.proxy.user
Authored-by: mrtisttt <mrtisttt@126.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
close#5654
This pr is to introduce a config for rewriting pod name, which can help users forcibly rewrite driver or executor pod name prefix.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No.
Closes#5672 from zhaohehuhu/dev-1111.
Closes#5654
6a450e444 [hezhao2] introduce new config to force rewrite pod name
Authored-by: hezhao2 <hezhao2@cisco.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
It corrects the fetching op log Rest API documentation to match request parameter case in code, solving issue #5599
### _How was this patch tested?_
No tests needed.
### _Was this patch authored or co-authored using generative AI tooling?_
No AI tool has been used for this patch.
Closes#5673 from GianlucaPrincipini/fix-fetching-op-log-param-doc.
Closes#5599
f134af050 [GianlucaPrincipini] Change maxRows "param" to "maxrows" in documentation
Authored-by: GianlucaPrincipini <gianluca.principini@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Support to config customized ticket cache location in kyuubi connection URL.
Before, we can only set it via environment variable `KRB5CCNAME`.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No.
Closes#5564 from turboFei/ticket_cache.
Closes#5564
140ae6ec2 [fwang12] since 1.8.0
caaf33a44 [fwang12] check is blank
0fae9f9c5 [fwang12] doc
05b9e9df8 [fwang12] ticket cache
Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
This PR aims to improve out-of-box UX.
Currently, administrators' checking is enforced even when security is disabled, and we are going to deliver the preview of Web UI in 1.8, it may confuse users who use the default configuration to launch Kyuubi sever and explore Kyuubi WebUI (because the user "anonymous" can not visit all users' sessions/engines without being configured as an administrator).
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No
Closes#5568 from pan3793/admin.
Closes#5568
2eccf659f [Cheng Pan] code dedup and fix test
76f4fd4a3 [Cheng Pan] fix test
d7ca94650 [Cheng Pan] Check administrator only when security is enabled
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
When using Kyuubi Hive JDBC to Hive Server2, TProtocolVersion.HIVE_CLI_SERVICE_PROTOCOL_V10 is used and can not be changed.
When we connected to Hive Server2 with version lower than 2.2.0, we got the following error:
```
org.apache.kyuubi.shade.org.apache.thrift.TApplicationException:
Required field 'client_protocol' is unset!
Struct:TOpenSessionReq(client_protocol:null, configuration:{kyuubi.client.version=1.7.3, set:hiveconf:hive.server2.thrift.resultset.default.fetch.size=1000, kyuubi.client.ipAddress=172.16.19.113, use:database=default})
```
In this PR, we introduced a session conf `clientProtocolVersion`.
By adding `clientProtocolVersion=8` to jdbc url, the error got fixed.
Changes of `kyuubi_jdbc.rst`
<img width="867" alt="image" src="https://github.com/apache/kyuubi/assets/88070094/8f98edf9-15c4-4d1b-9299-83b24136352b">
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Manual tests against Hive Server2 version 2.1.1-cdh6.3.0
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No.
Closes#5480 from zhouyifan279/protocol-version.
Closes#5480
0ee7b1f68 [zhouyifan279] Support setting kyuubi hive jdbc client protocol version
61b8038d3 [zhouyifan279] Support setting kyuubi hive jdbc client protocol version
ed4c29f2b [Cheng Pan] Update docs/deployment/migration-guide.md
dc16a05fc [zhouyifan279] Support setting kyuubi hive jdbc client protocol version
1a6bfd88c [zhouyifan279] Support setting kyuubi hive jdbc client protocol version
cb00edc37 [zhouyifan279] Support setting kyuubi hive jdbc client protocol version
c99fc4802 [zhouyifan279] Support setting kyuubi hive jdbc client protocol version
Lead-authored-by: zhouyifan279 <zhouyifan279@gmail.com>
Co-authored-by: Cheng Pan <pan3793@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
close#5423
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
NO
Closes#5485 from lsm1/branch-5423.
Closes#5423
0bd3b0233 [senmiaoliu] mention in the migration guide
dddafeb9b [senmiaoliu] support multi session conf advisor
Authored-by: senmiaoliu <senmiaoliu@trip.com>
Signed-off-by: ulyssesyou <ulyssesyou@apache.org>
### _Why are the changes needed?_
Both the default value of `kyuubi.zookeeper.embedded.data.log.dir` and `kyuubi.zookeeper.embedded.data.dir` are `embedded_zookeeper`. It is used based on the current dir.
If the configuration dir is a relative path , it is resolved relative to `KYUUBI_HOME`.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No
Closes#5412 from biangjuang/master.
Closes#5412
17e502c03 [Cheng Pan] Update kyuubi-zookeeper/src/main/scala/org/apache/kyuubi/zookeeper/ZookeeperConf.scala
e0aa01c82 [Cheng Pan] Update kyuubi-zookeeper/src/main/scala/org/apache/kyuubi/zookeeper/ZookeeperConf.scala
bb6af93df [Cheng Pan] Update docs/deployment/migration-guide.md
53bd61bc3 [lawulu] Resolve the relative zk configuration dir based on KYUUBI_HOME
Lead-authored-by: lawulu <biangjuang@gmail.com>
Co-authored-by: Cheng Pan <pan3793@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Kyuubi 1.8 added the support for JDK 17 and Flink 1.18.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
<img width="1139" alt="image" src="https://github.com/apache/kyuubi/assets/26535726/2316fce3-20fc-49dc-a967-a40f9ea8f706">
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No
Closes#5523 from pan3793/doc-ver.
Closes#5523
d9024ccc7 [Cheng Pan] Update the Kyuubi supported components version
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Refer to spark and flink settings conf, support configure Trino session conf in kyuubi-default.conf
issue : https://github.com/apache/kyuubi/issues/5282
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No
Closes#5283 from ASiegeLion/kyuubi-master-trino.
Closes#5282
87a3f57b4 [Cheng Pan] Apply suggestions from code review
effdd79f4 [liupeiyue] [KYUUBI #5282]Add trino's session conf to kyuubi-default.xml
399a200f7 [liupeiyue] [KYUUBI #5282]Add trino's session conf to kyuubi-default.xml
7462b32c2 [liupeiyue] [KYUUBI #5282]Add trino's session conf to kyuubi-default.xml--Update documentation
5295f5f94 [liupeiyue] [KYUUBI #5282]Add trino's session conf to kyuubi-default.xml
Lead-authored-by: liupeiyue <liupeiyue@yy.com>
Co-authored-by: Cheng Pan <pan3793@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
As https://github.com/apache/kyuubi/pull/4000 adds the Apache Phoenix support for the JDBC engine without upgrading the configuration's description.
This PR aims to update the outdated configuration description to reflect the capabilities of the Kyuubi JDBC engine.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No.
Closes#5294 from ymZhao1001/doc0915.
Closes#5294
8d61c36f9 [yangming] update jdbc doc
d082b4d34 [yangming] run dev/reformat and dev/gen/gen_all_config_docs.sh
5b814d86b [zhaoyangming] add jdbc Phoenix dialect
Lead-authored-by: yangming <261635393@qq.com>
Co-authored-by: zhaoyangming <zhaoyangming@deepexi.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Add Kyuubi Code Program to Doc for long-term management and sponsorship.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
no
Closes#5500 from yaooqinn/program.
Closes#5500
14f144747 [Kent Yao] Add Kyuubi Code Program to Doc
Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
This PR aims to shade the kyuubi spark authz plugin to simplify the user's use.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No
Closes#5427 from Yikf/shade-authz.
Closes#5427
d2f7ea8d1 [yikaifei] fix
695133de4 [Kent Yao] Update docs/security/authorization/spark/install.md
f3a653133 [Kent Yao] Update docs/security/authorization/spark/build.md
963cab372 [yikaifei] bundle
2068c98fc [yikaifei] relocation
6c6e50ea7 [yikaifei] Shade spark authz plugin
Lead-authored-by: yikaifei <yikaifei@apache.org>
Co-authored-by: Kent Yao <yao@apache.org>
Signed-off-by: yikaifei <yikaifei@apache.org>
### _Why are the changes needed?_
close#5408
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
NO
Closes#5433 from lsm1/branch-kyuubi-5408.
Closes#5408
8c6cc223d [senmiaoliu] fix style
20415dc44 [senmiaoliu] use com.mysql.cj.jdbc.Driver first
Authored-by: senmiaoliu <senmiaoliu@trip.com>
Signed-off-by: Shaoyun Chen <csy@apache.org>
### _Why are the changes needed?_
Follow #5329 and close#5328:
1. Add new config `kyuubi.metadata.store.jdbc.priority.enabled` to control whether enable priority scheduling, due to users may experience performance issues when using MySQL5.7 as metastore backend and enabling kyuubi batch v2 priority feature.
2. When priority scheduling is enabled, `KyuubiBatchService` picks metadata job with `ORDER BY priority DESC, create_time ASC`.
3. Insert metadata with priority field, default priority value is `10`.
4. Add new config `kyuubi.batch.priority` for each batch priority.
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No
Closes#5352 from zwangsheng/KYUUBI#5328.
Closes#5328
687ed1ed6 [Cheng Pan] Update kyuubi-server/src/main/scala/org/apache/kyuubi/server/metadata/jdbc/JDBCMetadataStore.scala
58621b557 [zwangsheng] fix comments
1bf81e75c [zwangsheng] fix style
7ed2551b3 [zwangsheng] update default priority desc & improve UT
21ceccb01 [zwangsheng] fix doc
27fc5e825 [zwangsheng] enrich desc
c0bbc0dfd [zwangsheng] fix style
6b8d0f091 [zwangsheng] fix comment
67eb2524d [zwangsheng] fix comment
e1705c34d [zwangsheng] Add config to control whether pick order by priority or not
129a46729 [zwangsheng] Add unit test for pickBatchForSubmitting
fcaf85d92 [zwangsheng] Fix unit test
f7ca2219e [zwangsheng] Fix unit test
8d4b276ff [wangsheng] fix code style
4c6b99090 [wangsheng] fix comments
654ad843a [zwangsheng] [KYUUBI #5328][V2] Kyuubi Server Pick Metadata job with priority
Lead-authored-by: zwangsheng <binjieyang@apache.org>
Co-authored-by: wangsheng <2213335496@qq.com>
Co-authored-by: Cheng Pan <pan3793@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Close#5381
change default metrics reporter to prometheus since Kyuubi 1.8
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
Closes#5344 from zhaohehuhu/Improvement-0928.
Closes#5381
84f4c8208 [hezhao2] reset METRICS_REPORTERS for test case
b9ee5f711 [Cheng Pan] Update kyuubi-server/src/test/scala/org/apache/kyuubi/events/handler/ServerJsonLoggingEventHandlerSuite.scala
86165a6fe [Cheng Pan] Update kyuubi-server/src/test/scala/org/apache/kyuubi/events/handler/ServerJsonLoggingEventHandlerSuite.scala
a3605b626 [hezhao2] set METRICS_PROMETHEUS_PORT to 0 for test cases
f1a4d2861 [hezhao2] restore version number for kyuubi.metrics.reporters in doc
dae40e1a2 [hezhao2] change default metrics reporter to prometheus
Lead-authored-by: hezhao2 <hezhao2@cisco.com>
Co-authored-by: Cheng Pan <pan3793@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- Update the regeneration hits for `setting.md` in `develop.md`, as the config doc`settings.md` is checked and able to regenearted by `dev/gen/gen_all_config_docs.sh` script.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No.
Closes#5409 from bowenliang123/regen-settings.
Closes#5409
985f39e51 [Bowen Liang] update docs
Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
### _Why are the changes needed?_
The Apache Spark Community found a performance regression with log4j2. See https://github.com/apache/spark/pull/36747.
This PR to fix the performance issue on our side.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No.
Closes#5400 from ITzhangqiang/KYUUBI_5365.
Closes#5365
dbb9d8b32 [ITzhangqiang] [KYUUBI #5365] Don't use Log4j2's extended throwable conversion pattern in default logging configurations
Authored-by: ITzhangqiang <itzhangqiang@163.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
To close#5362 .
Considering the maintenance burden of the Kyuubi community and easy cross-support for data lake projects.
Drop support EOLs of Spark 3 for the coming Spark 4.x era in kyuubi v1.9.
We will still do bugfix release for these spark3.0.x users.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No
Closes#5363 from AngersZhuuuu/KYUUBI-5362.
Closes#5362
d34cd6e2b [Angerszhuuuu] Update build.md
99f414bd4 [Angerszhuuuu] Update build.md
a5129e4f6 [Angerszhuuuu] Update build.md
6ee008cc5 [Angerszhuuuu] Update README.md
af792cc42 [Angerszhuuuu] Update master.yml
69b333161 [Angerszhuuuu] Merge branch 'master' into KYUUBI-5362
528554e9c [Angerszhuuuu] Update IcebergCatalogPrivilegesBuilderSuite.scala
427ebd48d [Angerszhuuuu] Update DataMaskingForJDBCV2Suite.scala
64809a54f [Angerszhuuuu] update
f7d89fd9b [Angerszhuuuu] [KYUUBI-5362] Kyuubi remove Authz test for spark3.0.3
Authored-by: Angerszhuuuu <angers.zhu@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
There were obvious errors in the case code, so these cases were fixed.
### _How was this patch tested?_
Just fix case code, no test need. Of course, I have run the case code and have confirmed that there is no problem.
Closes#5347 from mrtisttt/fix-hive-jdbc-example-code.
Closes#5347
ec6ecf758 [mrtisttt] [DOCS] Fix java code errors in quick_start_with_jdbc
Authored-by: mrtisttt <mrtisttt@126.com>
Signed-off-by: Fu Chen <cfmcgrady@gmail.com>
### _Why are the changes needed?_
In the `Getting Started with Hive JDBC`documentation, section`Connect to non-kerberized Kyuubi Server`, the code description is incorrect:
```
The below java code is using a keytab file to login and connect to Kyuubi server by JDBC.
```
Obviously, the sections and code here are describing how to connect to a kyuubi service without kerberos authentication.
### _How was this patch tested?_
Just fix code description, no test need.
Closes#5345 from mrtisttt/fix-non-kerberized-code-description.
Closes#5345
fa0418f00 [mrtisttt] [DOCS] Fix non-kerberized code description in quick_start_with_jdbc
Authored-by: mrtisttt <mrtisttt@126.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
close#4994
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
Closes#5157 from lsm1/branch-kyuubi_4994.
Closes#4994
02e7eb5fa [senmiaoliu] use abbr a
6d001e519 [senmiaoliu] replace 'allengine' with 'all'
f9c548299 [senmiaoliu] fix style
df64e773e [senmiaoliu] fix style
6a7d40e63 [senmiaoliu] remove allenginecommand
6306dd8a2 [senmiaoliu] support list all engine
Authored-by: senmiaoliu <senmiaoliu@trip.com>
Signed-off-by: Shaoyun Chen <csy@apache.org>
### _Why are the changes needed?_
For the secured YARN cluster, the Kyuubi Server's user typically has no permission to kill the application. Proxy user or admin should be used instead.
https://docs.cloudera.com/documentation/enterprise/latest/topics/cm_mc_yarn_acl.html#concept_yarn_app_acls__section_killing_an_app
> For YARN, the following three groups of users are allowed to kill a running application:
> - The application owner
> - A cluster administrator defined in yarn.admin.acl
> - A queue administrator defined in aclAdministerApps for the queue in which the application is running
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
Verified ADMIN mode in internal deployment. (output message is formatted for readable)
```
Error: Batch e351185f-1ed8-437a-91bf-da2174e611e2 failed:
{
"id":"e351185f-1ed8-437a-91bf-da2174e611e2",
"user":"da_music",
"batchType":"SPARK",
"name":"SparkPi",
"appStartTime":0,
"appId":"application_1694730881181_58306",
"appUrl":"http://xxxx-rm-2.xxxx:8088/cluster/app/application_1694730881181_58306",
"appState":"KILLED",
"appDiagnostic":"Application application_1694730881181_58306 was killed by user yarn at 10.49.59.149",
"kyuubiInstance":"kyuubi-1.kyuubi-headless.spark.svc.cluster.local:10099",
"state":"CANCELED",
"createTime":1695102138188,
"endTime":1695102163341,
"batchInfo":{}
}
```
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No.
Closes#5306 from pan3793/kill-proxy-user.
Closes#5306
2b2e54307 [Cheng Pan] address comments
e7e9a9c57 [Cheng Pan] nit
9cf2afc61 [Cheng Pan] polish
ff82d1230 [Cheng Pan] polish
bf0057b41 [Cheng Pan] ApplicationManager supports proxy user
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
As title
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No.
Closes#5273 from pan3793/doc-release.
Closes#5273
c4f96c69f [Cheng Pan] 1
2266af358 [Cheng Pan] nit
1d52c1a86 [Cheng Pan] Bump version for web-ui and update release docs
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
<img width="1080" alt="image" src="https://github.com/apache/kyuubi/assets/3898450/cc118574-c752-449a-9b20-04d2cc59b118">
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No
Closes#5227 from hanna-liashchuk/patch-2.
Closes#5227
baf391ba9 [Hanna Liashchuk] fix DOC style
e46dfddc3 [Cheng Pan] Update docs/deployment/engine_on_kubernetes.md
0746a350a [hanna-liashchuk] upd
233bba00b [hanna-liashchuk] Added deploy mode info
Lead-authored-by: Hanna Liashchuk <g.liashchuk@temabit.com>
Co-authored-by: hanna-liashchuk <47921651+hanna-liashchuk@users.noreply.github.com>
Co-authored-by: Cheng Pan <pan3793@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
As titled.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No.
Closes#5265 from link3280/docs_jm_ha.
Closes#5265
947854a5a [Paul Lin] Improve language
f7ea8dc51 [Paul Lin] Improve language
c40b94bf1 [Paul Lin] Add notice about Jobmanager HA to Flink deployment docs
Authored-by: Paul Lin <paullin3280@gmail.com>
Signed-off-by: Paul Lin <paullin3280@gmail.com>
### _Why are the changes needed?_
close#5244
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No
Closes#5251 from lsm1/branch-kyuubi-5244.
Closes#5244
bcadaa53a [senmiaoliu] rename
a6c92773d [senmiaoliu] fix style
1f38fa711 [senmiaoliu] fix style
3ff57ff8b [senmiaoliu] Make engineAliveMaxFailCount configurable
Authored-by: senmiaoliu <senmiaoliu@trip.com>
Signed-off-by: Shaoyun Chen <csy@apache.org>
### _Why are the changes needed?_
Add Flink KDF doc to close the issue #5118
<img width="1094" alt="image" src="https://github.com/apache/kyuubi/assets/3898450/23cd886e-c94a-406b-81a2-eb2caf4c1775">
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No
Closes#5233 from zhaohehuhu/Improvement-0901.
Closes#5118
fcd45b360 [hezhao2] code reformat
46bb9700c [hezhao2] Add Flink KDF doc
Authored-by: hezhao2 <hezhao2@cisco.com>
Signed-off-by: Shaoyun Chen <csy@apache.org>
Closes#3444
### _Why are the changes needed?_
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3558 from iodone/kyuubi-3444.
Closes#3444
acaa72afe [odone] remove plugin dependency from kyuubi spark engine
739f7dd5b [odone] remove plugin dependency from kyuubi spark engine
1146eb6e0 [odone] kyuubi-3444
Authored-by: odone <odone.zhang@gmail.com>
Signed-off-by: ulyssesyou <ulyssesyou@apache.org>
### _Why are the changes needed?_
Improvement: https://github.com/apache/kyuubi/issues/4540
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No
Closes#5182 from zhuyaogai/issue-4540.
Closes#4540
73652ae6d [Fantasy-Jay] Merge remote-tracking branch 'origin/issue-4540' into issue-4540
ae7a3b5bb [Fantasy-Jay] Support to deny some user to make connection.
46ee2f158 [Fantasy-Jay] Support to limit some user to make connection.
3d7e5e534 [Fantasy-Jay] Support to limit some user to make connection.
1f173ffe4 [Fantasy-Jay] Merge branch 'master' into issue-4540
1d3a3d47f [Fantasy-Jay] Support to limit some user to make connection.
e4a1b3eb3 [Fantasy-Jay] Support to limit some user to make connection.
1c120a992 [Fantasy-Jay] Support to limit some user to make connection.
6b657aa99 [Fantasy-Jay] Support to limit some user to make connection.
506c4751b [Fantasy-Jay] Support to limit some user to make connection.
f99072ef5 [Fantasy-Jay] Support to limit some user to make connection.
Authored-by: Fantasy-Jay <13631435453@163.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- Use `Set` collection for order-insensitive configs instead of `Seq`
- kyuubi.frontend.thrift.binary.ssl.disallowed.protocols
- kyuubi.authentication
- kyuubi.authentication.ldap.groupFilter
- kyuubi.authentication.ldap.userFilter
- kyuubi.kubernetes.context.allow.list
- kyuubi.kubernetes.namespace.allow.list
- kyuubi.session.conf.ignore.list
- kyuubi.session.conf.restrict.list
- kyuubi.session.local.dir.allow.list
- kyuubi.batch.conf.ignore.list
- kyuubi.engine.deregister.exception.classes
- kyuubi.engine.deregister.exception.messages
- kyuubi.operation.plan.only.excludes
- kyuubi.server.limit.connections.user.unlimited.list
- kyuubi.server.administrators
- kyuubi.metrics.reporters
- Support skipping blank elements
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No.
Closes#5185 from bowenliang123/conf-toset.
Closes#5185
c656af78a [liangbowen] conf to set
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
As titled.
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
Closes#5134 from link3280/KYUUBI-4806.
Closes#4806
a1b74783c [Paul Lin] Optimize code style
546cfdf5b [Paul Lin] Update externals/kyuubi-flink-sql-engine/src/main/scala/org/apache/kyuubi/engine/flink/operation/FlinkOperation.scala
b6eb7af4f [Paul Lin] Update externals/kyuubi-flink-sql-engine/src/main/scala/org/apache/kyuubi/engine/flink/result/ResultSet.scala
1563fa98b [Paul Lin] Remove explicit StartRowOffset for Flink
4e61a348c [Paul Lin] Add comments
c93294650 [Paul Lin] Improve code style
6bd0c8e69 [Paul Lin] Use dedicated thread pool
15412db3a [Paul Lin] Improve logging
d6a2a9cff [Paul Lin] [KYUUBI #4806][FLINK] Implement incremental result fetching
Authored-by: Paul Lin <paullin3280@gmail.com>
Signed-off-by: Paul Lin <paullin3280@gmail.com>
### _Why are the changes needed?_
close#5170
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No
Closes#5171 from iodone/kyuubi-5170.
Closes#5170
7cd0e2cbf [odone] identifier compatible with sprk3.4
Authored-by: odone <odone.zhang@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
close#5162
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No
Closes#5163 from lsm1/features/kyuubi-5162.
Closes#5162
c5bba8c20 [senmiaoliu] fix
af5e816fb [Kent Yao] Update docs/contributing/code/developer.md
ceb44358d [senmiaoliu] Add statements about contributions co-authored with AIGC in developer guide page
Lead-authored-by: senmiaoliu <senmiaoliu@trip.com>
Co-authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
This PR provides a new configuration `kyuubi.zookeeper.embedded.client.use.hostname` to control whether use hostname or IP address on binding, to improve the out-of-box experience of K8s case, similar to `kyuubi.frontend.connection.url.use.hostname`.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
Closes#5158 from pan3793/zk-ip.
Closes#5158
260dae987 [Cheng Pan] nit
3ac0c430e [Cheng Pan] doc
de50f65e8 [Cheng Pan] false
890b13bd9 [Cheng Pan] Allow embedded Zookeeper binding IP address
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Correct a typo in this doc
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
Closes#5153 from zhaohehuhu/Improvement-0810.
Closes#5153
b2b593d94 [hezhao2] Update quick start hive jdbc instruction
Authored-by: hezhao2 <hezhao2@cisco.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- Move the configuration docs to the top level of docs, which is most commonly used and referenced
- update relevant doc links
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
Closes#5154 from bowenliang123/config-doc-first.
Closes#5154
b49ed3f8b [liangbowen] nit
db7f0d14d [liangbowen] update doc links
f8fd697a2 [liangbowen] move config docs to the top level
7448e4487 [liangbowen] change title of settings doc
40214ddd8 [liangbowen] move config doc in the front of deployment
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
- allow using the session's user/password to connect database in the JDBC engine
- it is allowed to be applied on any share level of engines, since every kyuubi session maintains a dedicated JDBC connection in the current implementation
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
Closes#5120 from bowenliang123/jdbc-user.
Closes#5120
7b8ebd137 [liangbowen] Use session's user and password to connect to database in JDBC engine
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
Support SSL for trino engine.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [X] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3374 from hddong/support-trino-password.
Closes#3374
f39daaf78 [Cheng Pan] improve
6308c4cf7 [hongdongdong] Support SSL for trino engine
Lead-authored-by: Cheng Pan <chengpan@apache.org>
Co-authored-by: hongdongdong <hongdd@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
