### Why are the changes needed?
Recently, GHA fails frequently with downloading engines failure, this adds logs to display the cached engine archives.
### How was this patch tested?
I will monitor GHA failure after merging.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes#7131 from pan3793/gha-cache.
Closes#7131
87a38e0d6 [Cheng Pan] Print cached engine archives
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### Why are the changes needed?
Test Spark 4.0.0 RC1
https://lists.apache.org/thread/3sx86qhnmot1p519lloyprxv9h7nt2xh
### How was this patch tested?
GHA.
### Was this patch authored or co-authored using generative AI tooling?
No
Closes#6928 from pan3793/spark-4.0.0.
Closes#6928
a910169bd [Cheng Pan] Bump Spark 4.0.0
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### Why are the changes needed?
Github has provide subtask creation & auto-linking features, and they are more advanced
### How was this patch tested?
https://github.com/apache/kyuubi/issues/7030
### Was this patch authored or co-authored using generative AI tooling?
no
Closes#7032 from yaooqinn/sb.
Closes#7032
3fd6ccd90 [Kent Yao] remove more
cbf691033 [Kent Yao] Remove Umbrella/Subtask issue template
Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
### Why are the changes needed?
Test Spark 3.5.5 Release Notes
https://spark.apache.org/releases/spark-release-3-5-5.html
### How was this patch tested?
Pass GHA.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes#6939 from pan3793/spark-3.5.5.
Closes#6939
8c0288ae5 [Cheng Pan] ga
78b0e72db [Cheng Pan] nit
686a7b0a9 [Cheng Pan] fix
d40cc5bba [Cheng Pan] Bump Spark 3.5.5
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### Why are the changes needed?
Spark 3.5.4 is released https://spark.apache.org/releases/spark-release-3-5-4.html
### How was this patch tested?
Pas GHA
### Was this patch authored or co-authored using generative AI tooling?
No
Closes#6842 from pan3793/spark-3.5.4.
Closes#6842
0fb7ad8a0 [Cheng Pan] ga
8eacc9c97 [Cheng Pan] Spark 3.5.4 RC2
0721fa401 [Cheng Pan] fix
49e98a201 [Cheng Pan] maven repo
951db0c82 [Cheng Pan] Spark 3.5.4
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### Why are the changes needed?
Enable gluten CI for kyuubi apache repo
### How was this patch tested?
Review
### Was this patch authored or co-authored using generative AI tooling?
No
Closes#6846 from wForget/gluten_ci.
Closes#6846
22ab3ada8 [wforget] Enable gluten CI for kyuubi apache repo
Authored-by: wforget <643348094@qq.com>
Signed-off-by: wforget <643348094@qq.com>
### Why are the changes needed?
The GHA cache `engine-archives` seems frequently to be evicted because of exceeding the project cache capacity(10GiB), and I found the K8s IT job produces fresh large cache objects frequently, we should disable that to let `engine-archives` survive, to avoid downloading from the fragile `archive.apache.org` each time.
```
Connect to archive.apache.org:443 [archive.apache.org/65.108.204.189, archive.apache.org/2a01:4f9:1a:a084:0:0:0:2] failed: Network is unreachable (connect failed)
```
<img width="1306" alt="Xnip2024-12-04_17-36-03" src="https://github.com/user-attachments/assets/7e00ecad-8492-4a3e-b669-03b56a05747c">
PS: I deleted bunches of `buildkit-blob-*` manually to save `engine-archives`
### How was this patch tested?
Monitor GHA after merging.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes#6837 from pan3793/k8s-it.
Closes#6837
8bab514dc [Cheng Pan] [INFRA] Disable K8s CI image GHA cache
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### Why are the changes needed?
From my review experience in the past few months, the contributors have less interest in filling the current rich PR template without explicit requests from reviewers.
Instead of keeping a rich template to ask the contributors to answer bunches of questions, I prefer to leave only the most important questions to keep the template clean and short.
### How was this patch tested?
It's not a code change, it requires manual review.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes#6821 from pan3793/pr-template.
Closes#6821
b1bbdd6c1 [Cheng Pan] nit
ac9a9a59b [Cheng Pan] nit
6fece594c [Cheng Pan] Restore to the plain PULL_REQUEST_TEMPLATE
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
# 🔍 Description
## Issue References 🔗
This pull request fixes #
## Describe Your Solution 🔧
- batching removals to avoid repeatedly reading and updating the apt-get databases
- 50% speed up the execution time for freeing disk space from 2 minutes or more , to about 1 minutues.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6765 from bowenliang123/speedup-freedisk.
Closes#6765
dd475c243 [Bowen Liang] update (+1 squashed commit) Squashed commits: [688793f47] update
Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Kent Yao <yao@apache.org>
# 🔍 Description
## Issue References 🔗
This pull request fixes #
## Describe Your Solution 🔧
- as titled
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6775 from bowenliang123/gha-cache-v4.
Closes#6775
de7b9fdf0 [Bowen Liang] actions/dependency-review-action@v4
c9e41923a [Bowen Liang] actions/cache@v4
Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
# 🔍 Description
## Issue References 🔗
This pull request fixes #
## Describe Your Solution 🔧
- skip explicitly set the parameter `check-latest` of Github action `actions/setup-java` to false, which is its default value.
- docs: https://github.com/actions/setup-java?tab=readme-ov-file#check-latest
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6768 from bowenliang123/action-java-latest-false.
Closes#6768
a0ecd4688 [Bowen Liang] update
Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
# 🔍 Description
## Issue References 🔗
`Spark Engine On Kubernetes Integration Test` always fails after #6739
The default java version of Ubuntu 24.04 is OpenJDK 21: https://discourse.ubuntu.com/t/ubuntu-24-04-lts-noble-numbat-release-notes/39890#openjdk , this is a version higher than that supported by spark 3.5.X.
## Describe Your Solution 🔧
Specify JDK 11 for spark engine on kubernetes it.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
test ga: https://github.com/wForget/kyuubi/actions/runs/11441231714/job/31828818049
---
# Checklist 📝
- [X] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6763 from wForget/minor.
Closes#6763
aee39339f [wforget] Specify JDK 11 for spark engine on kubernetes it
Authored-by: wforget <643348094@qq.com>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
# 🔍 Description
## Issue References 🔗
This pull request fixes #
## Describe Your Solution 🔧
- as titled
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6741 from bowenliang123/gha-upload-artifact-v4.
Closes#6741
83a0ee90e [Bowen Liang] update GHA actions/upload-artifact from v3 to v4
Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
# 🔍 Description
## Issue References 🔗
This pull request fixes #
## Describe Your Solution 🔧
- The Ubuntu 24.04 image for Actions is now generally available since Sep 2024 : https://github.blog/changelog/2024-09-25-actions-new-images-and-ubuntu-latest-changes/
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6739 from bowenliang123/ubuntu-2404.
Closes#6739
2d120d1f6 [Bowen Liang] bump ci image from ubuntu 22.04 to 24.04
Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
# 🔍 Description
## Issue References 🔗
This pull request fixes #
## Describe Your Solution 🔧
- inspired by comments from #6664 : https://github.com/apache/kyuubi/pull/6664#issuecomment-2344351119
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6687 from bowenliang123/mvn-args.
Closes#6687
6a9170b60 [Cheng Pan] Update .github/workflows/master.yml
3573131d8 [liangbowen] export MAVEN_ARGS
36ced0532 [liangbowen] update
a93fa28ee [liangbowen] MAVEN_ARGS
Lead-authored-by: Bowen Liang <liangbowen@gf.com.cn>
Co-authored-by: liangbowen <liangbowen@gf.com.cn>
Co-authored-by: Cheng Pan <pan3793@gmail.com>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
# 🔍 Description
## Issue References 🔗
This pull request fixes #
## Describe Your Solution 🔧
- update GHA docker/build-push-action from v5 to v6 to support build mmary
https://docs.docker.com/build/ci/github-actions/build-summary/
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6740 from bowenliang123/docker-push-v6.
Closes#6740
674cc15fc [Bowen Liang] update GHA docker/build-push-action from v5 to v6 to support build summary
Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
Upgrade default flink version to 1.20.0
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [X] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6755 from wForget/minor.
Closes#6755
e968502a6 [wforget] comment
c0094c760 [wforget] comment
b74a0bb75 [wforget] comment
1aea2216c [wforget] Upgrade default flink version to 1.20.0
Authored-by: wforget <643348094@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
This pull request fixes #
## Describe Your Solution 🔧
- build python dependencies for docs building to latest versions
- no display or behaviour changes
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6752 from bowenliang123/doc-req.
Closes#6752
ffd8782bd [Bowen Liang] update
c328c7584 [Bowen Liang] bump doc build requirements
Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
# 🔍 Description
## Issue References 🔗
This pull request fixes #
## Describe Your Solution 🔧
- bump `pnpm` to v9.11, release note: https://github.com/pnpm/pnpm/releases/tag/v9.11.0
- updated pnpm lockfile by running `pnpm install --fix-lockfile`
- bump `nodejs` version from v18 to v20 (LTS version, released on 18 Apr 2023)
- update related github actions, aligning the pnpm and nodejs versions
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6712 from bowenliang123/pnpm9.
Closes#6712
ca6e425f4 [liangbowen] nit
a9ea92216 [liangbowen] bump pnpm to v9
Lead-authored-by: Bowen Liang <liangbowen@gf.com.cn>
Co-authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
# 🔍 Description
Spark 4.0.0-preview2 RC1 passed the vote
https://lists.apache.org/thread/4ctj2mlgs4q2yb4hdw2jy4z34p5yw2b1
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Pass GHA.
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6699 from pan3793/spark-4.0.0-preview2.
Closes#6699
2db1f645d [Cheng Pan] 4.0.0-preview2
42055bb1e [Cheng Pan] fix
d29c0ef83 [Cheng Pan] disable delta test
98d323b95 [Cheng Pan] fix
2e782c00b [Cheng Pan] log4j-slf4j2-impl
fde4bb6ba [Cheng Pan] spark-4.0.0-preview2
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
This pull request fixes #
## Describe Your Solution 🔧
- bump superlinter Github Action to v7
- skip linting `.env` files as the rules are too strict in unorder keys
- remove deprecated `ERROR_ON_MISSING_EXEC_BIT` option
- no major change
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
- pass the style CI workflow
#### Related Unit Tests
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6682 from bowenliang123/superlinter-7.
Closes#6682
0ff321127 [Bowen Liang] skip linting env files
cab7aa810 [liangbowen] Remove deprecated ERROR_ON_MISSING_EXEC_BIT
67515df5b [liangbowen] bump superlinter to v7
Lead-authored-by: Bowen Liang <liangbowen@gf.com.cn>
Co-authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
# 🔍 Description
## Issue References 🔗
This pull request fixes#5314
## Describe Your Solution 🔧
To support Java 21, some dependency packages need to be upgraded.Here is the detailed explanation.
1. To build with Java 21, Scala binary version needs to be upgraded to 2.13.11 or above.
2. The versions of `spotless` and `googlejavaformat` need to be upgraded to higher versions. However, this will lose support for Java 8. In order to continue supporting Java 8, code style checks will be ignored in the Java 21 profile.
3. Since the Flink engine does not support Scala 2.13, the Scala version of the Flink module is fixed at 2.12.
Previous discussions were at https://github.com/apache/kyuubi/pull/6306.
---
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
---
**Be nice. Be informative.**
Closes#6597 from dupen01/issue-java21.
Closes#5314
0c5a9e455 [Cheng Pan] simplify
54cbadf79 [dupeng] java21+scala2.13.11+flink_2.12+CI(spark4,21,2,13)
Lead-authored-by: dupeng <dunett@163.com>
Co-authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
To support Java 21, the Scala version needs to be upgraded to 2.13.11 or above.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6603 from dupen01/scala213.
Closes#6603
541f86054 [Cheng Pan] exclude SparkLocalClusterTest
13c11bf52 [Perl Du] add ignoreClasses comments
d5b67971b [Cheng Pan] Update pom.xml
582de9b90 [dupeng] ignore org.jline.terminal.impl.ffm.*
364eea00c [dupeng] upgrade maven.plugin.silencer.version from 1.7.13 to 1.7.17
99a1cc40a [dupeng] upgrade scala version to 2.13.14
54b0f4d3f [dupeng] upgrade scala version to 2.13.11
Lead-authored-by: dupeng <dunett@163.com>
Co-authored-by: Cheng Pan <pan3793@gmail.com>
Co-authored-by: Perl Du <34719039+dupen01@users.noreply.github.com>
Co-authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
The current Hive 2.3.10 integration tests are broken, the root cause is CALCITE-1224
```
2024-08-20T12:20:55,405 ERROR [HiveTBinaryFrontendHandler-Pool: Thread-49] org.apache.kyuubi.util.KyuubiUncaughtExceptionHandler - Uncaught exception in thread HiveTBinaryFrontendHandler-Pool: Thread-49
java.lang.NoSuchMethodError: com.fasterxml.jackson.core.JsonGenerator.writeStartArray(Ljava/lang/Object;I)V
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:78) ~[jackson-databind-2.12.0.jar:2.12.0]
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18) ~[jackson-databind-2.12.0.jar:2.12.0]
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480) ~[jackson-databind-2.12.0.jar:2.12.0]
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319) ~[jackson-databind-2.12.0.jar:2.12.0]
at com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4485) ~[jackson-databind-2.12.0.jar:2.12.0]
at com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3699) ~[jackson-databind-2.12.0.jar:2.12.0]
at org.apache.hive.service.cli.operation.SQLOperation.getTaskStatus(SQLOperation.java:518) ~[hive-service-2.3.10.jar:2.3.10]
at org.apache.hive.service.cli.operation.Operation.getStatus(Operation.java:143) ~[hive-service-2.3.10.jar:2.3.10]
at org.apache.kyuubi.engine.hive.operation.HiveOperation.runInternal(HiveOperation.scala:62) ~[kyuubi-hive-sql-engine_2.12-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
at org.apache.kyuubi.operation.AbstractOperation.run(AbstractOperation.scala:175) ~[kyuubi-hive-sql-engine_2.12-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
at org.apache.kyuubi.session.AbstractSession.runOperation(AbstractSession.scala:103) ~[kyuubi-hive-sql-engine_2.12-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
at org.apache.kyuubi.engine.hive.session.HiveSessionImpl.runOperation(HiveSessionImpl.scala:59) ~[kyuubi-hive-sql-engine_2.12-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
at org.apache.kyuubi.session.AbstractSession.$anonfun$executeStatement$1(AbstractSession.scala:133) ~[kyuubi-hive-sql-engine_2.12-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
at org.apache.kyuubi.session.AbstractSession.withAcquireRelease(AbstractSession.scala:84) ~[kyuubi-hive-sql-engine_2.12-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
at org.apache.kyuubi.session.AbstractSession.executeStatement(AbstractSession.scala:130) ~[kyuubi-hive-sql-engine_2.12-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
at org.apache.kyuubi.service.AbstractBackendService.executeStatement(AbstractBackendService.scala:66) ~[kyuubi-hive-sql-engine_2.12-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
at org.apache.kyuubi.service.TFrontendService.ExecuteStatement(TFrontendService.scala:253) ~[kyuubi-hive-sql-engine_2.12-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
at org.apache.kyuubi.shaded.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1670) ~[kyuubi-hive-sql-engine_2.12-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
at org.apache.kyuubi.shaded.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1650) ~[kyuubi-hive-sql-engine_2.12-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
at org.apache.kyuubi.shaded.thrift.ProcessFunction.process(ProcessFunction.java:38) ~[kyuubi-hive-sql-engine_2.12-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
at org.apache.kyuubi.shaded.thrift.TBaseProcessor.process(TBaseProcessor.java:38) ~[kyuubi-hive-sql-engine_2.12-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
at org.apache.kyuubi.service.authentication.TSetIpAddressProcessor.process(TSetIpAddressProcessor.scala:35) ~[kyuubi-hive-sql-engine_2.12-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
at org.apache.kyuubi.shaded.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:250) ~[kyuubi-hive-sql-engine_2.12-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_422]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_422]
at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_422]
```
## Describe Your Solution 🔧
Unpack `avatica-1.8.0.jar` and delete Jackson classes to workaround CALCITE-1224
## Types of changes 🔖
- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
This PR should recover CI.
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6635 from pan3793/hive-2310-fix.
Closes#6635
334bc5c39 [Cheng Pan] nit
22ae945fb [Cheng Pan] fix
dec03196d [Cheng Pan] Fix Hive 2.3.10 integration tests
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
Spark 3.5.2 was released recently.
Release Notes is available at https://spark.apache.org/releases/spark-release-3-5-2.html
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Pass GHA.
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6609 from pan3793/spark-3.5.2.
Closes#6609
587cf1dd3 [Cheng Pan] Bump Spark 3.5.2
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
I found the [previous change](https://github.com/apache/kyuubi/pull/6606) breaks compatibility in some shell interpreters, for example `[ ! -f "$MVN_BIN" ]`, `[ $MVN_DETECTED_VERSION != $MVN_VERSION ]` are not supported in ubuntu dash, which fails the CI https://github.com/apache/kyuubi/actions/runs/10347069575/job/28636666290
## Types of changes 🔖
- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
This change should recover the CI.
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6610 from pan3793/mvn-followup.
Closes#6606
5381b60ac [Cheng Pan] 1
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
This PR disables Javadoc/Scaladoc by default, while still enabling that on Nexus deploy and CI, to speed up the normal building by developers daily performed.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Apple M1 Max, Java 17
```
$ time build/mvn clean install -DskipTests
```
before: 1145.67s user 62.41s system 276% cpu 7:16.56 total
after: 530.33s user 35.82s system 214% cpu 4:23.69 total
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6599 from pan3793/javadoc.
Closes#6599
660f8dc02 [Cheng Pan] Disable Javadoc/Scaladoc by default
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
# 🔍 Description
Flink 1.20 is out, [Release Note](https://nightlies.apache.org/flink/flink-docs-release-1.20/release-notes/flink-1.20/)
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
CI is updated to cover Flink 1.20
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6557 from pan3793/flink-1.20.
Closes#6557
a414094b9 [Cheng Pan] remove rc
8ee4cf5cd [Cheng Pan] fix url
fbaf66071 [Cheng Pan] docs
ddbd10ffe [Cheng Pan] Support Flink 1.20
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
This pull request aims to remove building support for Spark 3.2, while still keeping the engine support for Spark 3.2.
Mailing list discussion: https://lists.apache.org/thread/l74n5zl1w7s0bmr5ovxmxq58yqy8hqzc
- Remove Maven profile `spark-3.2`, and references on docs, release scripts, etc.
- Keep the cross-version verification to ensure that the Spark SQL engine built on the default Spark version (3.5) still works well on Spark 3.2 runtime.
- Merge `kyuubi-extension-spark-common` into `kyuubi-extension-spark-3-3`
- Remove `log4j.properties` as Spark moves to Log4j2 since 3.3 (SPARK-37814)
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [x] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Pass GHA.
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6545 from pan3793/deprecate-spark-3.2.
Closes#6545
54c172528 [Cheng Pan] fix
f4602e805 [Cheng Pan] Deprecate and remove building support for Spark 3.2
2e083f89f [Cheng Pan] fix style
458a92c53 [Cheng Pan] nit
929e1df36 [Cheng Pan] Deprecate and remove building support for Spark 3.2
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
This pull request fixes#6537
## Describe Your Solution 🔧
Upgrade Flink from 1.19.0 to 1.19.1.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6538 from SteNicholas/KYUUBI-6537.
Closes#6537
b39d308d6 [SteNicholas] [KYUUBI #6537] Upgrade Flink from 1.19.0 to 1.19.1
Authored-by: SteNicholas <programgeek@163.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
This pull request fixes #
## Describe Your Solution 🔧
Bump minikube and kubernetes version of integration test to fix failure of integration test in CI.
- minikube: v1.29.0->v1.33.1
- kubernetes: v1.26.1->v1.30.0
Docker version 25 loads images into various distributions of Kubernetes has been impossible due to what appears to be mismatching hashes of manifests/images as follows:
```
X Exiting due to GUEST_IMAGE_LOAD: save to dir: caching images: caching image "/home/runner/.minikube/cache/images/amd64/apache/kyuubi_latest": write: unable to calculate manifest: blob sha256:5bbc241b2d6dcc55b5a63c080556ad458ba1395e93af3204d12e72c9a192eb06 not found
```
Minikube and kubernetes version should be bumped to fix failure of integration test which refers to https://github.com/moby/moby/issues/47207.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6466 from cfmcgrady/bump-minikube.
Closes#6466
c4bf4b56e [Fu Chen] Bump minikube and kubernetes version of integration test
Authored-by: Fu Chen <cfmcgrady@gmail.com>
Signed-off-by: Fu Chen <cfmcgrady@gmail.com>
# 🔍 Description
[`jcl-over-slf4j`](https://www.slf4j.org/legacy.html#jcl-over-slf4j) is a drop-in replacement of `commons-logging`, the latter one should not be present in the final classpath, otherwise, there are potential class conflict issues.
The current dep check is problematic, this PR also changes it to always perform "install" to fix the false negative report.
## Types of changes 🔖
- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Simply delete `commons-logging-1.1.3.jar` from `apache-kyuubi-1.9.1-bin.tgz` and everything goes well.
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6458 from pan3793/commons-logging.
Closes#6458
114ec766a [Cheng Pan] fix
79d4121a1 [Cheng Pan] fix
6633e83ee [Cheng Pan] fix
21127ed0b [Cheng Pan] always perform install on dep check
98b13dfcf [Cheng Pan] Remove commons-logging from binary release
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
This PR makes KSHC support Spark 4.0, and also makes sure that the KSHC jar compiled against Spark 3.5 is binary compatible with Spark 4.0.
We are ready to enable CI for Spark 4.0, except for authZ module.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Pass GHA.
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6453 from pan3793/spark4-ci.
Closes#6453
695e3d7f7 [Cheng Pan] Update pom.xml
2eaa0f88a [Cheng Pan] Update .github/workflows/master.yml
b1f540a34 [Cheng Pan] cross test
562839982 [Cheng Pan] fix
9f0c2e1be [Cheng Pan] fix
45f182462 [Cheng Pan] kshc
227ef5bae [Cheng Pan] fix
690a3b8b2 [Cheng Pan] Revert "fix"
87fe7678b [Cheng Pan] fix
60f55dbed [Cheng Pan] CI for Spark 4.
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
This PR adds Spark 4.0 cross-test for the Spark SQL engine - built against Spark 3.5, run with Spark 4.0.
As Spark 4.0 requires Java 17 or 21, the cross test uses Java 17.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Pass GHA.
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6452 from pan3793/xtest-4.0.
Closes#6452
c49883756 [Cheng Pan] Add cross test for Spark 4.0
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
Kyuubi uses the Hudi Spark bundle jar in authZ module for testing, Hudi 0.15 brings Spark 3.5 and Scala 2.13 support, it also removes hacky for profile `spark-3.5`.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Pass GHA.
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6451 from pan3793/hudi-0.15.
Closes#6451
98d6e97c5 [Cheng Pan] fix
2d31307da [Cheng Pan] remove spark-authz-hudi-test
8896f8c3f [Cheng Pan] Enable hudi test
7e9a7c7ae [Cheng Pan] Bump Hudi 0.15.0
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
This pull request adds cross-version tests for Kyuubi Spark TPC-DS Connector and TPC-H Connector.
## Describe Your Solution 🔧
Add TPC-DS Connector and TPC-H Connector into GitHub Actions job `spark-connector-cross-version-test`.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6441 from zhouyifan279/tcp-ds/h-cross-version.
Closes#6441
c2abc468a [zhouyifan279] Kyuubi Spark TPC-DS/H Connector cross version test
Authored-by: zhouyifan279 <zhouyifan279@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
This pull request closes#6247
This also closes#6431
## Describe Your Solution 🔧
Add a job `spark-connector-cross-version-test` in GitHub Actions to:
1. Build KSHC package with maven opt `-Pspark-3.5`
2. Run KSHC tests with maven opt `-Pspark-3.3` and `-Pspark-3.4` and KSHC package built in step 1
3. Fix the binary-compatible issue via reflection.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Pass GHA.
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6436 from zhouyifan279/kshc-cross-version-test.
Closes#6247
d3ac2ef47 [zhouyifan279] Tune the KSHC code to fix binary-compatible issues
4e14edcb5 [zhouyifan279] Fix invalid unit-tests-log name
56ca45d18 [zhouyifan279] Fix invalid unit-tests-log name
4c5ab7b9e [zhouyifan279] Update test log name
8a84e8812 [zhouyifan279] Add matrix scala
17cb67155 [zhouyifan279] [KYUUBI #6247] KSHC cross-version test
Authored-by: zhouyifan279 <zhouyifan279@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
`DeltaTest` was duplicated, while the second one should be `PaimonTest`
## Types of changes 🔖
- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Review and Pass GHA.
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6423 from pan3793/excl-tag.
Closes#6423
c94218a5d [Cheng Pan] Fix Scalatest exclusion tags
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
Hive 2.3.10 is available now, and it will be the latest version of the 2.3 serial.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Pass GA.
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6328 from pan3793/hive-2310.
Closes#6328
82a9fb70c [Cheng Pan] Update .github/workflows/master.yml
10af5e13b [Cheng Pan] hive-2.3.10-rc-1
2dbb2b54e [Cheng Pan] test Hive 2.3.10 RC0
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
This pull request fixes#6281
## Describe Your Solution 🔧
The change initialize a CI job to run unit testing on python client, including:
- Set up Github Action based on docker-compose
- Update test cases and test succeeded for dialect `presto` and `trino`
- Temporary disabled hive related test due to test cases are not valid, not about connection
- Update dev dependencies to support python 3.10
- Speed up testing with `pytest-xdist` plugin
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
Not able to ran unit test in local and on CI
#### Behavior With This Pull Request 🎉
Able to run and partially cover a couple of test cases
#### Related Unit Tests
No
## Additional notes
Next action is about fixing failing test cases or considering skipping some of them if necessary
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6343 from sudohainguyen/ci/init.
Closes#6281
682e575c4 [Harry] Remove xdist out of scope
dc42ca1ff [Harry] Pin pytest packages version
469f1d955 [Harry] Pin ubuntu version
00cef476a [Harry] Use v4 checkout action
96ef83148 [Harry] Remove unnecessary steps
732344a2c [Harry] Add step to tear down containers
1e2c2481a [Harry] Resolved trino and presto test
5b33e3924 [Harry] Make tests runnable
1be033ba3 [Harry] Remove randome flag which causes failed test run
2bc6dc036 [Harry] Switch action setup provider to docker
ea2a76319 [Harry] Initialize github action for python unit testing
Authored-by: Harry <quanghai.ng1512@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
Previously, we were asked to limit the job concurrency to 10, while the new policy is more relaxed.
> All workflows SHOULD have a job concurrency level less than or equal to 15. Just because 20 is the max, doesn't mean you should strive for 20.
https://infra.apache.org/github-actions-policy.html
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Monitor CI
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6363 from pan3793/ci-con.
Closes#6363
101121aa9 [Cheng Pan] [INFRA] Increase CI concurrency
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
This pull request fixes#6323
## Describe Your Solution 🔧
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6333 from lsm1/branch-bump-spark-3.4.3.
Closes#6323
cc324fd77 [senmiaoliu] bump spark 3.4.3
Authored-by: senmiaoliu <senmiaoliu@trip.com>
Signed-off-by: Shaoyun Chen <csy@apache.org>
# 🔍 Description
## Issue References 🔗
This pull request fixes CI step name which makes confusing, followup: https://github.com/apache/kyuubi/pull/6180
## Describe Your Solution 🔧
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.
## Types of changes 🔖
- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6280 from liuxiaocs7/patch-1.
Closes#6180
d8002d4ea [Liu Xiao] Update the setup JDK name
Lead-authored-by: Liu Xiao <liuxiao2103@qq.com>
Co-authored-by: Liu Xiao <42756849+liuxiaocs7@users.noreply.github.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
This pull request fixes#6250
## Describe Your Solution 🔧
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [x] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Pass CI
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6273 from liuxiaocs7/issue-6250.
Closes#6250
c6ba1e88a [liuxiao] remove unused import
db887ef9b [liuxiao] inline method
769da013b [Cheng Pan] Update externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/spark/sql/kyuubi/SparkDatasetHelper.scala
21dbd37a7 [liuxiao] remove unused import
e869d571e [liuxiao] update for miss
7d755a879 [liuxiao] Drop support for Spark 3.1
Lead-authored-by: liuxiao <liuxiao2103@qq.com>
Co-authored-by: Cheng Pan <pan3793@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
This pull request fixes#6249.
## Describe Your Solution 🔧
According to the plan, we will no longer support Flink 1.16, and this PR will try to remove Flink 1.16.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [x] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Pass GHA
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6259 from slfan1989/drop_flink1.16_support.
Closes#6249
808acc2fb [Cheng Pan] Update docs/deployment/migration-guide.md
fc5ecf6fd [Shilun Fan] [KYUUBI #6249] Fix CheckStyle.
8d8f9ded4 [Shilun Fan] [KYUUBI #6249] Fix CheckStyle.
7a1d974b3 [Shilun Fan] [KYUUBI #6249] Drop support for Flink 1.16.
Lead-authored-by: Shilun Fan <slfan1989@apache.org>
Co-authored-by: Cheng Pan <pan3793@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
This pull request adds an issue template for 2024H1 Kyuubi Code Contribution Task
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Review
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6240 from pan3793/code-task-tmpl.
Closes#6240
f811f31bc [Cheng Pan] [INFRA] Add issue template for 2024H1 Kyuubi Code Contribution Task
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
This pull request fixes#6134
## Describe Your Solution 🔧
Add profile and ci for flink 1.19.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [X] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [X] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6190 from wForget/KYUUBI-6134-FOLLOWUP.
Closes#6134
6c5c8824f [wforget] [KYUUBI #6134][FOLLOWUP] Add profile and ci for flink 1.19
Authored-by: wforget <643348094@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
This pull request fixes [Kyuubi Spark Nightly test]( https://github.com/apache/kyuubi/actions/runs/8259256940/job/22592865897) because Spark 4.0 drops the support of Java 8/11 and Scala 2.12
## Describe Your Solution 🔧
Switch to Java 17 and Scala 2.13
## Types of changes 🔖
- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Monitor CI
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6180 from pan3793/spark-nightly.
Closes#6180
781d3aeda [Cheng Pan] Kyuubi Spark nightly testing should use JDK 17
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
`kyuubi-gluten-it` is always enabled and uses a different Spark version, which mess the code jumping in IDEA.
## Describe Your Solution 🔧
- conditional enable `kyuubi-gluten-it` module via profile `gluten-it`
- refactor POM and GHA workflow to reduce duplicated definition
## Types of changes 🔖
- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Review, and wait for daily GHA results.
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6178 from pan3793/gluten-it.
Closes#6178
77b7bb809 [Cheng Pan] fix
c62ce40f8 [Cheng Pan] Improve Gluten intergartion test
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
