### Why are the changes needed?
fix typos in docs
### How was this patch tested?
NO
### Was this patch authored or co-authored using generative AI tooling?
No
Closes#6964 from wenxin-cn/fix-typos-in-docs.
Closes#6964
5a50a927a [Kent Yao] style
be899c21f [10172] fix typos in docs
Lead-authored-by: wenxin-cn <wen.xin@datasw.com>
Co-authored-by: Kent Yao <yao@apache.org>
Co-authored-by: 10172 <wen.xin@datasw.com>
Signed-off-by: Kent Yao <yao@apache.org>
# 🔍 Description
Flink 1.20 is out, [Release Note](https://nightlies.apache.org/flink/flink-docs-release-1.20/release-notes/flink-1.20/)
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
CI is updated to cover Flink 1.20
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6557 from pan3793/flink-1.20.
Closes#6557
a414094b9 [Cheng Pan] remove rc
8ee4cf5cd [Cheng Pan] fix url
fbaf66071 [Cheng Pan] docs
ddbd10ffe [Cheng Pan] Support Flink 1.20
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
`master` and `branch-1.9` already added basic support for Spark 4.0.0-preivew1.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Review
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6432 from pan3793/docs-4.0.
Closes#6432
73177aab3 [Cheng Pan] Mention support for Spark 4.0 in docs
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
This pull request fixes#6250
## Describe Your Solution 🔧
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [x] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Pass CI
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6273 from liuxiaocs7/issue-6250.
Closes#6250
c6ba1e88a [liuxiao] remove unused import
db887ef9b [liuxiao] inline method
769da013b [Cheng Pan] Update externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/spark/sql/kyuubi/SparkDatasetHelper.scala
21dbd37a7 [liuxiao] remove unused import
e869d571e [liuxiao] update for miss
7d755a879 [liuxiao] Drop support for Spark 3.1
Lead-authored-by: liuxiao <liuxiao2103@qq.com>
Co-authored-by: Cheng Pan <pan3793@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
This pull request fixes#6249.
## Describe Your Solution 🔧
According to the plan, we will no longer support Flink 1.16, and this PR will try to remove Flink 1.16.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [x] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Pass GHA
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6259 from slfan1989/drop_flink1.16_support.
Closes#6249
808acc2fb [Cheng Pan] Update docs/deployment/migration-guide.md
fc5ecf6fd [Shilun Fan] [KYUUBI #6249] Fix CheckStyle.
8d8f9ded4 [Shilun Fan] [KYUUBI #6249] Fix CheckStyle.
7a1d974b3 [Shilun Fan] [KYUUBI #6249] Drop support for Flink 1.16.
Lead-authored-by: Shilun Fan <slfan1989@apache.org>
Co-authored-by: Cheng Pan <pan3793@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
Discussion in mailing thread: https://lists.apache.org/thread/tnmz71o3rypy7qvs3899p3jkkq4xqb4r
I propose to rename the `bin/beeline` to `bin/kyuubi-beeline`, while for compatibility, we may still want to keep the alias `bin/beeline` for a while.
In a real Hadoop cluster, it’s likely to add `$HIVE_HOME/bin`, `$SPARK_HOME/bin`, `$KYUUBI_HOME/bin` to the `$PATH`, at the current state, when performing `beeline`, which one is called depends on the declaration order.
It does not matter for Spark’s `bin/beeline` because it’s a vanilla Hive BeeLine, but in Kyuubi, we have made some improvements based on vanilla Hive BeeLine, so the behavior is not exactly same as Hive’s BeeLine.
An identical name would solve this problem. And I saw some vendors[1] who shippes Kyuubi already have done the same thing.
[1] https://help.aliyun.com/zh/emr/emr-on-ecs/user-guide/connect-to-kyuubi
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Manual test.
```
$ bin/beeline -u 'jdbc:kyuubi://0.0.0.0:10009/'
Warning: beeline is deprecated and will be removed in the future, please use kyuubi-beeline instead.
Connecting to jdbc:kyuubi://0.0.0.0:10009/
Connected to: Spark SQL (version 3.4.1)
Driver: Kyuubi Project Hive JDBC Client (version 1.10.0-SNAPSHOT)
Beeline version 1.10.0-SNAPSHOT by Apache Kyuubi
0: jdbc:kyuubi://0.0.0.0:10009/>
```
```
$ bin/kyuubi-beeline -u 'jdbc:kyuubi://0.0.0.0:10009/'
Connecting to jdbc:kyuubi://0.0.0.0:10009/
Connected to: Spark SQL (version 3.4.1)
Driver: Kyuubi Project Hive JDBC Client (version 1.10.0-SNAPSHOT)
Beeline version 1.10.0-SNAPSHOT by Apache Kyuubi
0: jdbc:kyuubi://0.0.0.0:10009/>
```
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6239 from pan3793/kyuubi-beeline.
Closes#6239
cec8f56e2 [Cheng Pan] docs
b3446baf1 [Cheng Pan] docs
46a115077 [Cheng Pan] Remove `bin/beeline` to `bin/kyuubi-beeline`
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
Kyuubi actually supports Flink 1.19 since 1.9.0, the docs are outdated.
## Describe Your Solution 🔧
Update docs
## Types of changes 🔖
- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Review
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6191 from pan3793/flink-1.19-docs.
Closes#6191
a5a0e4491 [Cheng Pan] Update docs to mention support of Flink 1.19
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
Historically, we provided a few standalone YAML files to allow users to deploy Kyuubi on K8s, and later a Helm Chart was introduced, which is a simple and mature way for production use cases.
## Describe Your Solution 🔧
Remove the legacy standalone YAML files way and update docs to guide user to use Helm chart to deploy Kyuubi on K8s.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Review
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6080 from pan3793/k8s-cleanup.
Closes#6080
3a9fa9be3 [Cheng Pan] comment
8f681d28c [Cheng Pan] Improve instruction for deploy Kyuubi on K8s
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
This is a regular dependency upgrading,
## Describe Your Solution 🔧
Upgrade `trino-client` from 363 to 411. 411 is the latest version which uses okhttp 3.x, hence it does not have kotlin runtime dependencies.
This PR also updates the docs, especially the Trino cluster version requirement.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Pass GA.
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#5975 from pan3793/trino-411.
Closes#5975
2b57df34d [Cheng Pan] fix
c498a5bb3 [Cheng Pan] fix
21948ca4f [Cheng Pan] Fix compile
e4f1397cc [Cheng Pan] license
66583ca16 [Cheng Pan] Bump trino-client 411
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
The CDH Hive 2.1.1-cdh6.3.2 is the latest free version of CDH, which is adopted widely, it would be great if the Kyuubi Hive engine could support this version.
## Describe Your Solution 🔧
Actually, the Kyuubi Hive engine could work with CDH Hive 2.1.1-cdh6.3.2, this pull request just adds an integration test and fixes one test case.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklists
## 📝 Author Self Checklist
- [x] My code follows the [style guidelines](https://kyuubi.readthedocs.io/en/master/contributing/code/style.html) of this project
- [x] I have performed a self-review
- [ ] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my feature works
- [x] New and existing unit tests pass locally with my changes
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
## 📝 Committer Pre-Merge Checklist
- [x] Pull request title is okay.
- [x] No license issues.
- [x] Milestone correctly set?
- [x] Test coverage is ok
- [x] Assignees are selected.
- [x] Minimum number of approvals
- [x] No changes are requested
**Be nice. Be informative.**
Closes#5830 from pan3793/hive-211-cdh6.
Closes#5830
afa80bd82 [Cheng Pan] nit
89c9ae96e [Cheng Pan] nit
e402e49cd [Cheng Pan] nit
3c126a2a6 [Cheng Pan] Test Hive engine with CDH Hive 2.1.1-cdh6.3.2
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Kyuubi 1.8 added the support for JDK 17 and Flink 1.18.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
<img width="1139" alt="image" src="https://github.com/apache/kyuubi/assets/26535726/2316fce3-20fc-49dc-a967-a40f9ea8f706">
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No
Closes#5523 from pan3793/doc-ver.
Closes#5523
d9024ccc7 [Cheng Pan] Update the Kyuubi supported components version
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Kyuubi requires Flink 1.16/1.17 now
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4723 from pan3793/flink-1.17.
Closes#4367
1e64dcbf1 [Cheng Pan] [KYUUBI #4367][DOCS][FOLLOWUP] Kyuubi requries Flink 1.16/1.17
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Kyuubi requires Spark 3.1 and above since 1.7.0
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4722 from pan3793/spark-3.1.
Closes#4722
e2b4c9ca2 [Cheng Pan] [DOCS] Kyuubi requires Spark 3.1 and above now
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Update docs to mention that Kyuubi requires Flink 1.15+ now
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4721 from pan3793/doc.
Closes#4387
4fdd9082b [Cheng Pan] [KYUUBI #4387][DOCS][FOLLOUP] Update Flink version requirements
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
close#4338 .
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4339 from a49a/bump-spark332.
Closes#4338
6c741d82 [Luning Wang] [KYUUBI #4338] Bump Spark from 3.3.1 to 3.3.2
Authored-by: Luning Wang <wang4luning@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- DISCLAIMER file on top-level folder of project was already removed after graduation in (https://github.com/apache/kyuubi/issues/4020)
- fix build/dist for `cp` non-existed DISCLAIMER file
- remove DISCLAIMER file reference in dev/dist script, doc
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4086 from bowenliang123/desclaimer-ref.
Closes#4086
18be7489 [liangbowen] remove DISCLAIMER file reference in dev/dist script, doc, docker , labeler.yml
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
The current quick start is too wordy and not user-friendly.
In this PR, we simply the guide to let newcomers easier build a quick start deployment with Kyuubi and Spark only. The copied logs are removed the make the doc shorter.
<img width="1787" alt="image" src="https://user-images.githubusercontent.com/8326978/201630828-013f3b8b-b88b-4a35-b1ae-e142cb153370.png">
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3804 from yaooqinn/qstart.
Closes#3804
d5bff5d0 [Kent Yao] [DOCS] Make the quick start less wordy and easier
696b2610 [Kent Yao] [DOCS] Make the quick start less wordy and easier
b7e30d95 [Kent Yao] [DOCS] Make the quick start less wordy and easier
Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
