# 🔍 Description
## Issue References 🔗
This pull request fixes #
## Describe Your Solution 🔧
Preparing v1.11.0-SNAPSHOT after branch-1.10 cut
```shell
build/mvn versions:set -DgenerateBackupPoms=false -DnewVersion="1.11.0-SNAPSHOT"
(cd kyuubi-server/web-ui && npm version "1.11.0-SNAPSHOT")
```
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6769 from bowenliang123/bump-1.11.
Closes#6769
6db219d28 [Bowen Liang] get latest_branch by sorting version in branch name
465276204 [Bowen Liang] update package.json
81f2865e5 [Bowen Liang] bump
Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
# 🔍 Description
Upgrade default flink version to 1.20.0
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [X] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6755 from wForget/minor.
Closes#6755
e968502a6 [wforget] comment
c0094c760 [wforget] comment
b74a0bb75 [wforget] comment
1aea2216c [wforget] Upgrade default flink version to 1.20.0
Authored-by: wforget <643348094@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
This pull request fixes #
## Describe Your Solution 🔧
- to fix CVE-2024-8184 reported in https://github.com/apache/kyuubi/security/dependabot/75
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6742 from bowenliang123/jetty-9.4.56.
Closes#6742
1e8fee555 [Bowen Liang] update
96db527d6 [Bowen Liang] bumpu jetty to 9.4.56
Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
# 🔍 Description
## Issue References 🔗
This pull request fixes #
## Describe Your Solution 🔧
- to fix CVE-2024-8184 reported in https://github.com/apache/kyuubi/security/dependabot/72
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6743 from bowenliang123/protobuf-3.25.5.
Closes#6743
a9a574041 [Bowen Liang] bump protobuf from 3.25.4 to 3.25.5
Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
# 🔍 Description
## Issue References 🔗
This pull request fixes #
## Describe Your Solution 🔧
Bump testcontainers-scala from 0.41.0 (released in Aug 2023) to 0.41.4 (released in Jun 2024)
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6745 from bowenliang123/testcontainers-scala-0.41.4.
Closes#6745
06551cb8d [Bowen Liang] bump testcontainers-scala from 0.41.0 to 0.41.4
Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
# 🔍 Description
## Issue References 🔗
This pull request fixes #
## Describe Your Solution 🔧
- Bump commons-lang3 from 3.13.0 ( 2023-07-23 ) to 3.17.0 ( 2024-08-24 ), https://commons.apache.org/proper/commons-lang/changes-report.html#a3.17.0
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6746 from bowenliang123/commonslang3-3.17.
Closes#6746
d72467bfb [Bowen Liang] bump commons-lang3 from 3.13.0 to 3.17.0
Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
# 🔍 Description
## Issue References 🔗
This pull request fixes #
## Describe Your Solution 🔧
- Bump guava from 32.1.3 (Oct 11, 2023) to 33.3.1 (Sep 23, 2024 ) : https://github.com/google/guava/releases/tag/v33.3.1
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6747 from bowenliang123/guava-33.3.1.
Closes#6747
39abf1b23 [Bowen Liang] bump guava from 32.1.3 to 33.3.1
Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
# 🔍 Description
## Describe Your Solution 🔧
Bump vertx-core from 4.5.1 to 4.5.3 to fix CVE-2024-1300 and CVE-2024-1023
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Build and ran locally
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6737 from Madhukar525722/vertx.
Closes#6737
7c758e082 [madlnu] Bump vertx-grpc from 4.5.1 to 4.5.3
Authored-by: madlnu <madlnu@visa.com>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
# 🔍 Description
## Issue References 🔗
This pull request fixes #
## Describe Your Solution 🔧
- bump SQLlite from 3.42.0 (May 2023) to 3.46 (May 2024)
- SQLlite changelog: https://sqlite.org/changes.html
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6733 from bowenliang123/sqllite-346.
Closes#6733
a86bea232 [Bowen Liang] update
Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
# 🔍 Description
## Issue References 🔗
This pull request fixes #
## Describe Your Solution 🔧
- bump `pnpm` to v9.11, release note: https://github.com/pnpm/pnpm/releases/tag/v9.11.0
- updated pnpm lockfile by running `pnpm install --fix-lockfile`
- bump `nodejs` version from v18 to v20 (LTS version, released on 18 Apr 2023)
- update related github actions, aligning the pnpm and nodejs versions
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6712 from bowenliang123/pnpm9.
Closes#6712
ca6e425f4 [liangbowen] nit
a9ea92216 [liangbowen] bump pnpm to v9
Lead-authored-by: Bowen Liang <liangbowen@gf.com.cn>
Co-authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
# 🔍 Description
Spark 4.0.0-preview2 RC1 passed the vote
https://lists.apache.org/thread/4ctj2mlgs4q2yb4hdw2jy4z34p5yw2b1
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Pass GHA.
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6699 from pan3793/spark-4.0.0-preview2.
Closes#6699
2db1f645d [Cheng Pan] 4.0.0-preview2
42055bb1e [Cheng Pan] fix
d29c0ef83 [Cheng Pan] disable delta test
98d323b95 [Cheng Pan] fix
2e782c00b [Cheng Pan] log4j-slf4j2-impl
fde4bb6ba [Cheng Pan] spark-4.0.0-preview2
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
`paimon-spark-3.5` is a bundled jar but still pulls transitive deps.
## Describe Your Solution 🔧
Explicitly excluding transitive deps of paimon.
## Types of changes 🔖
- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Pass GHA and manually checked.
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6698 from pan3793/exclude-paimon.
Closes#6698
6048b1156 [Cheng Pan] Exclude transitive deps from paimon
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
## Describe Your Solution 🔧
- Bump `scala-collection-compat` to 2.12.0
- `scala-collection-compat` is used by the authz plugin testing Iceberg
- Currently Iceberg 1.6.1 uses `scala-collection-compat` 2.12.0,
according to: https://github.com/apache/iceberg/blob/apache-iceberg-1.6.1/gradle/libs.versions.toml#L78
- release note: https://github.com/scala/scala-collection-compat/releases/tag/v2.12.0
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
- pass the Authz tests
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6677 from pionCham/bump-scala-collection-compat-version.
Closes#6677
fd2738dce [chengpeiming] bump scala collection compat to 2.12.0
Authored-by: chengpeiming <chengpeiming@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
# 🔍 Description
## Issue References 🔗
- Scala 2.12.19 release https://github.com/scala/scala/releases/tag/v2.12.19
## Describe Your Solution 🔧
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6676 from pionCham/bump-scala-version.
Closes#6676
7f5e17e98 [chengpeiming] change dependencyList
8736f2c57 [chengpeiming] bump scala version to 2.12.19
36f9984f5 [chengpeiming] bump scala version
Authored-by: chengpeiming <chengpeiming@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
# 🔍 Description
## Issue References 🔗
This pull request fixes #
## Describe Your Solution 🔧
- Apache Commons Lang2 is no longer actively maintained and not used by Kyuubi modules
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6672 from bowenliang123/remove-commonlang2.
Closes#6672
34cda170a [liangbowen] remove common lang2
Lead-authored-by: Bowen Liang <liangbowen@gf.com.cn>
Co-authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
# 🔍 Description
## Issue References 🔗
This pull request fixes #
## Describe Your Solution 🔧
- bump Maven from 3.8.x to 3.9.x
- Maven 3.9 release notes:
- 3.9.0: https://maven.apache.org/docs/3.9.0/release-notes.html
- 3.9.9: https://maven.apache.org/docs/3.9.9/release-notes.html
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6664 from bowenliang123/maven39.
Closes#6664
074b7ba00 [Bowen Liang] bump maven to 3.9
Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
# 🔍 Description
## Issue References 🔗
Apache Iceberg 1.6.1 release https://github.com/apache/iceberg/releases/tag/apache-iceberg-1.6.1
## Describe Your Solution 🔧
In the project POM file, I have updated the Apache Iceberg version from 1.6.0 to 1.6.1
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6659 from pionCham/upgrate-iceberg-version.
Closes#6659
923019440 [chengpeiming] Update the docs for Spark connector
433981e66 [chengpeiming] Supplement other configurations
1617e36fe [chengpeiming] Upgrate iceberg.version in pom.xml
Authored-by: chengpeiming <chengpeiming@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
# 🔍 Description
## Issue References 🔗
This pull request fixes#5402
## Describe Your Solution 🔧
When facing out-of-control memory management in Spark engine, we typically use JVMkill as a remedy by killing the process and generating a heap dump for post-analysis. However, even with jvmkill protection, we may still encounter issues caused by JVM running out of memory, such as repeated execution of Full GC without performing any useful work during the pause time. Since the JVM does not exhaust 100% of resources, JVMkill will not be triggered.
So introducing JVMQuake provides more granular monitoring of GC behavior, enabling early detection of memory management issues and facilitating fast failure.
You can use the following configuration to enable jvmQuake plugins：
```
spark.plugins=org.apache.spark.kyuubi.jvm.quake.KyuubiJVMQuakePlugin
```
| configuration | default | comment |
| ---- | ---- | ---- |
| spark.driver.jvmQuake.enabled | false | when true, enable driver jvmQuake |
| spark.executor.jvmQuake.enabled | false | when true, enable executor jvmQuake |
| spark.driver.jvmQuake.heapDump.enabled | false | when true, enable jvm heap dump when jvmQuake rearch the threshold |
| spark.executor.jvmQuake.heapDump.enabled | false | when true, enable jvm heap dump when jvmQuake rearch the threshold |
| spark.jvmQuake.dumpThreshold | 100 | The number of seconds to dump memory |
| spark.jvmQuake.killThreshold | 200 | The number of seconds to kill process |
| spark.jvmQuake.exitCode | 502 | The exit code of kill process |
| spark.jvmQuake.heapDumpPath | /tmp/kyuubi_jvm_quake/apps | The path of heap dump |
| spark.jvmQuake.checkInterval | 3 | The number of seconds to check jvmQuake |
| spark.jvmQuake.runTimeWeight | 1.0 | The weight of rum time |
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6572 from yoock/features/kyuubi-jvm-quake.
Closes#5402
84361ce8f [王龙] add jvm quake
Authored-by: 王龙 <wanglong16@xiaomi.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
This pull request fixes #
## Describe Your Solution 🔧
- bump Paimon from 0.7.x-incubating to 0.8.2
- Paimon is currently used for Authz plugin testing only for Paimon Spark catalog
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6644 from bowenliang123/paimon08.
Closes#6644
6944099e4 [liangbowen] update paimon to 0.8
Lead-authored-by: Bowen Liang <liangbowen@gf.com.cn>
Co-authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
# 🔍 Description
## Issue References 🔗
This pull request fixes#5314
## Describe Your Solution 🔧
To support Java 21, some dependency packages need to be upgraded.Here is the detailed explanation.
1. To build with Java 21, Scala binary version needs to be upgraded to 2.13.11 or above.
2. The versions of `spotless` and `googlejavaformat` need to be upgraded to higher versions. However, this will lose support for Java 8. In order to continue supporting Java 8, code style checks will be ignored in the Java 21 profile.
3. Since the Flink engine does not support Scala 2.13, the Scala version of the Flink module is fixed at 2.12.
Previous discussions were at https://github.com/apache/kyuubi/pull/6306.
---
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
---
**Be nice. Be informative.**
Closes#6597 from dupen01/issue-java21.
Closes#5314
0c5a9e455 [Cheng Pan] simplify
54cbadf79 [dupeng] java21+scala2.13.11+flink_2.12+CI(spark4,21,2,13)
Lead-authored-by: dupeng <dunett@163.com>
Co-authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
To support Java 21, the Scala version needs to be upgraded to 2.13.11 or above.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6603 from dupen01/scala213.
Closes#6603
541f86054 [Cheng Pan] exclude SparkLocalClusterTest
13c11bf52 [Perl Du] add ignoreClasses comments
d5b67971b [Cheng Pan] Update pom.xml
582de9b90 [dupeng] ignore org.jline.terminal.impl.ffm.*
364eea00c [dupeng] upgrade maven.plugin.silencer.version from 1.7.13 to 1.7.17
99a1cc40a [dupeng] upgrade scala version to 2.13.14
54b0f4d3f [dupeng] upgrade scala version to 2.13.11
Lead-authored-by: dupeng <dunett@163.com>
Co-authored-by: Cheng Pan <pan3793@gmail.com>
Co-authored-by: Perl Du <34719039+dupen01@users.noreply.github.com>
Co-authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
Spark 3.5.2 was released recently.
Release Notes is available at https://spark.apache.org/releases/spark-release-3-5-2.html
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Pass GHA.
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6609 from pan3793/spark-3.5.2.
Closes#6609
587cf1dd3 [Cheng Pan] Bump Spark 3.5.2
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
This pull request fixes#6584
## Describe Your Solution 🔧
It involves bumping the version from 1.0.2 to 1.1.1 for flip-tables
## Test Plan 🧪
Build locally
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6614 from Madhukar525722/flip_tables.
Closes#6584
dee9fd7a9 [madlnu] [KYUUBI #6584] Upgrade flip-tables from 1.0.2 to 1.1.1
Authored-by: madlnu <madlnu@visa.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
This PR upgrades gRPC, Protobuf, and Guava to the relatively up-to-date versions, to support the latest Netty 4.1.111.
https://groups.google.com/g/grpc-io/c/R4JB2h7y5N4
> For Netty 4.1.111 compat, it is best to use grpc-java 1.63.2, 1.64.2, 1.65.1, and later.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Pass GHA.
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6607 from pan3793/grpc.
Closes#6607
a7c299629 [Cheng Pan] update dependencyList
c4cac4372 [Cheng Pan] Bump gRPC 1.65.1 Protobuf 3.25.4 Guava 32.1.3
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
As title.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Pass GHA.
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6601 from pan3793/delta-4.0.
Closes#6601
711f2e071 [Cheng Pan] Bump Delta Lake 4.0.0rc1 and enable Delta Lake test for Spark 4.0
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
It was wrong applied to all goals previously, which caused all `test-compile`s are skipped.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Let's monitor what CI happens
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6600 from pan3793/scaladoc.
Closes#6600
86ea6fdd0 [Cheng Pan] scala213
2a76a93d5 [Cheng Pan] Correct usage of maven.scaladoc.skip
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
This PR disables Javadoc/Scaladoc by default, while still enabling that on Nexus deploy and CI, to speed up the normal building by developers daily performed.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Apple M1 Max, Java 17
```
$ time build/mvn clean install -DskipTests
```
before: 1145.67s user 62.41s system 276% cpu 7:16.56 total
after: 530.33s user 35.82s system 214% cpu 4:23.69 total
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6599 from pan3793/javadoc.
Closes#6599
660f8dc02 [Cheng Pan] Disable Javadoc/Scaladoc by default
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
# 🔍 Description
## Issue References 🔗
This pull request fixes #
## Describe Your Solution 🔧
- bump apache parent pom from 31 to 33:
- 32: https://github.com/apache/maven-apache-parent/releases/tag/apache-32
- [[MPOM-264](https://issues.apache.org/jira/browse/MPOM-264)] - Parameterize maven-compiler-plugin with parameter "release" when running on JDK 9+
- 33: https://github.com/apache/maven-apache-parent/releases/tag/apache-33
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6571 from bowenliang123/mvn-parent-33.
Closes#6571
bb367efdf [Bowen Liang] bump parent pom to 33
Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
# 🔍 Description
## Issue References 🔗
This pull request fixes #
## Describe Your Solution 🔧
- Bump commons-io from 2.11.0(2021-07-09) to 2.16.1(2024-04-04) for accumulate updates, release note: https://commons.apache.org/proper/commons-io/changes-report.html#a2.16.1
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6586 from bowenliang123/commonsio-216.
Closes#6586
4fbb05650 [Bowen Liang] bump commons-io to 2.16.1
Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
# 🔍 Description
Flink 1.20 is out, [Release Note](https://nightlies.apache.org/flink/flink-docs-release-1.20/release-notes/flink-1.20/)
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
CI is updated to cover Flink 1.20
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6557 from pan3793/flink-1.20.
Closes#6557
a414094b9 [Cheng Pan] remove rc
8ee4cf5cd [Cheng Pan] fix url
fbaf66071 [Cheng Pan] docs
ddbd10ffe [Cheng Pan] Support Flink 1.20
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
This pull request fixes #
## Describe Your Solution 🔧
- bump `scala-maven-plugin` from 4.8.0 to 4.9.2, with `zinc` bumped from 1.8.0(Nov 11, 2022) to 1.10.0(May 6, 2024)
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6364 from bowenliang123/scala-maven-490.
Closes#6364
554379702 [Bowen Liang] bump scala-maven-plugin to 4.9.2
Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
# 🔍 Description
Upgrade to latest Apache Iceberg 1.6.0
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Pass existing GHA.
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6558 from pan3793/iceberg-1.6.0.
Closes#6558
75e42938f [Cheng Pan] nit
ba508b6bb [Cheng Pan] nit
0180521aa [Cheng Pan] Bump Iceberg 1.6.0
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
This pull request aims to remove building support for Spark 3.2, while still keeping the engine support for Spark 3.2.
Mailing list discussion: https://lists.apache.org/thread/l74n5zl1w7s0bmr5ovxmxq58yqy8hqzc
- Remove Maven profile `spark-3.2`, and references on docs, release scripts, etc.
- Keep the cross-version verification to ensure that the Spark SQL engine built on the default Spark version (3.5) still works well on Spark 3.2 runtime.
- Merge `kyuubi-extension-spark-common` into `kyuubi-extension-spark-3-3`
- Remove `log4j.properties` as Spark moves to Log4j2 since 3.3 (SPARK-37814)
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [x] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Pass GHA.
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6545 from pan3793/deprecate-spark-3.2.
Closes#6545
54c172528 [Cheng Pan] fix
f4602e805 [Cheng Pan] Deprecate and remove building support for Spark 3.2
2e083f89f [Cheng Pan] fix style
458a92c53 [Cheng Pan] nit
929e1df36 [Cheng Pan] Deprecate and remove building support for Spark 3.2
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
Bump Kyuubi Shaded 0.4.1
The only change is #6496
## Types of changes 🔖
- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Pass GHA.
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6543 from pan3793/shaded-041.
Closes#6543
3b674af95 [Cheng Pan] Remove staging repo
cae175517 [Cheng Pan] fix
549310067 [Cheng Pan] Bump Kyuubi Shaded 0.4.1
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
This pull request fixes #
## Describe Your Solution 🔧
- this bump helps to recognize Java 21-23 class formats: https://github.com/mojohaus/extra-enforcer-rules/releases
- reduce warning message when checking java bytecode versions
```
Warning: Unknown bytecodeVersion for com.fasterxml.jackson.core:jackson-core:jar:2.15.4:compile : META-INF/versions/21/com/fasterxml/jackson/core/io/doubleparser/FastDoubleSwar.class: got null class-file-version
```
## Types of changes 🔖
- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6548 from bowenliang123/mojo-rules-180.
Closes#6548
f53e4725d [Bowen Liang] update
Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
# 🔍 Description
## Issue References 🔗
This pull request fixes#6537
## Describe Your Solution 🔧
Upgrade Flink from 1.19.0 to 1.19.1.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6538 from SteNicholas/KYUUBI-6537.
Closes#6537
b39d308d6 [SteNicholas] [KYUUBI #6537] Upgrade Flink from 1.19.0 to 1.19.1
Authored-by: SteNicholas <programgeek@163.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
This pull request fixes #
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
---
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6529 from avishnus/kubernetes.
Closes#6529
2a82df557 [Cheng Pan] Update dev/dependencyList
a4855467c [Cheng Pan] Update pom.xml
1ef0c40aa [Cheng Pan] Update dev/dependencyList
4b45b0038 [Cheng Pan] Update dev/dependencyList
d0c5174e8 [Cheng Pan] Update dev/dependencyList
bc2f6b8eb [Cheng Pan] Update dev/dependencyList
f76a7f487 [avishnus] Upgrading kubernetes-client
Lead-authored-by: Cheng Pan <pan3793@gmail.com>
Co-authored-by: avishnus <avishnus@visa.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
Upgrade the Netty version to fix the CVEs
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
GHAs
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6528 from Madhukar525722/netty_upgrade.
Closes#6528
6c083c1fa [madlnu] Bump Netty 4.1.108.Final
Authored-by: madlnu <madlnu@visa.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
This pull request fixes#6515
## Describe Your Solution 🔧
Bumped up dropwizard metrics from 4.2.23 to 4.2.26
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6527 from avishnus/metrics.
Closes#6515
253036ca3 [avishnus] Bumping up dropwizard metrics to 4.2.26
Authored-by: avishnus <avishnus@visa.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
[`jcl-over-slf4j`](https://www.slf4j.org/legacy.html#jcl-over-slf4j) is a drop-in replacement of `commons-logging`, the latter one should not be present in the final classpath, otherwise, there are potential class conflict issues.
The current dep check is problematic, this PR also changes it to always perform "install" to fix the false negative report.
## Types of changes 🔖
- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Simply delete `commons-logging-1.1.3.jar` from `apache-kyuubi-1.9.1-bin.tgz` and everything goes well.
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6458 from pan3793/commons-logging.
Closes#6458
114ec766a [Cheng Pan] fix
79d4121a1 [Cheng Pan] fix
6633e83ee [Cheng Pan] fix
21127ed0b [Cheng Pan] always perform install on dep check
98b13dfcf [Cheng Pan] Remove commons-logging from binary release
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
This PR follows SPARK-45508 to modify the JVM args by adding `--add-opens=java.base/jdk.internal.ref=ALL-UNNAMED`, without this arg should not affect Spark engines as `spark-submit` will handle that by launching a sub-process and adding those JVM args.
The change benefits Spark in-process cases in UT, a little.
This PR also renames maven properties `maven.plugin.surefire.argLine` back to `extraJavaTestArgs` as it is used in both surefire and scalatest plugins, and makes it always takes effect instead of only JDK 17, then we don't need a copy when adding a new profile `java-21`
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Pass GHA.
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6456 from pan3793/add-open.
Closes#6456
ef7310a2e [Cheng Pan] add opens
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
This PR makes KSHC support Spark 4.0, and also makes sure that the KSHC jar compiled against Spark 3.5 is binary compatible with Spark 4.0.
We are ready to enable CI for Spark 4.0, except for authZ module.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Pass GHA.
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6453 from pan3793/spark4-ci.
Closes#6453
695e3d7f7 [Cheng Pan] Update pom.xml
2eaa0f88a [Cheng Pan] Update .github/workflows/master.yml
b1f540a34 [Cheng Pan] cross test
562839982 [Cheng Pan] fix
9f0c2e1be [Cheng Pan] fix
45f182462 [Cheng Pan] kshc
227ef5bae [Cheng Pan] fix
690a3b8b2 [Cheng Pan] Revert "fix"
87fe7678b [Cheng Pan] fix
60f55dbed [Cheng Pan] CI for Spark 4.
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
Improve data lake dependency management by extracting the following Maven properties:
- `delta.artifact`
- `hudi.artifact`
- `iceberg.artifact`
- `paimon.artifact`
It often takes a while for the downstream data lakes to support the new Spark versions, extracting those properties makes it easy to override in the new profile on the Kyuubi project's `pom.xml` to workaround before data lakes jars are available.
One use case is a19bb7c18e
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Pass GHA.
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6427 from pan3793/datalake-dep.
Closes#6427
74a9300e0 [Cheng Pan] Improve datalake dependency management
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
Kyuubi uses the Hudi Spark bundle jar in authZ module for testing, Hudi 0.15 brings Spark 3.5 and Scala 2.13 support, it also removes hacky for profile `spark-3.5`.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Pass GHA.
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6451 from pan3793/hudi-0.15.
Closes#6451
98d6e97c5 [Cheng Pan] fix
2d31307da [Cheng Pan] remove spark-authz-hudi-test
8896f8c3f [Cheng Pan] Enable hudi test
7e9a7c7ae [Cheng Pan] Bump Hudi 0.15.0
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
`DeltaTest` was duplicated, while the second one should be `PaimonTest`
## Types of changes 🔖
- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Review and Pass GHA.
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6423 from pan3793/excl-tag.
Closes#6423
c94218a5d [Cheng Pan] Fix Scalatest exclusion tags
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
This is a regular dependency upgrade.
https://github.com/eclipse-ee4j/jersey/releases/tag/2.40
Though 2.40 is not the latest version of the Jersey 2 serial, using a version adopted by Spark 3.5 is not harmful.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Pass GHA.
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6418 from pan3793/jersey-240.
Closes#6418
e925acdbd [Cheng Pan] update dependencyList
a853dc289 [Cheng Pan] Bump Jersey to 2.40
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
Leverage [Flatten Maven Plugin](https://www.mojohaus.org/flatten-maven-plugin/) to generate resolved `pom.xml`.
This recovers a dev case for Scala 2.13.
We usually perform `clean install` with `-am` and `-DskipTests` one time and perform `test` after changing code on specific modules, e.g.
```
build/mvn -Pscala-2.13 -pl externals/kyuubi-spark-sql-engine clean install -am -DskipTests
build/mvn -Pscala-2.13 -pl externals/kyuubi-spark-sql-engine test
```
Currently, the second step wrongly resolves to `*_2.12` dependencies due to the installed `pom.xml` referring to `*_${scala.binary.version}` while the default value of `scala.binary.version` is `2.12`.
## Types of changes 🔖
- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
This change is verified locally by the above case.
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6416 from pan3793/flatten.
Closes#6416
db7020c92 [Cheng Pan] Generate flattened pom.xml
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
We can not manage Jersey dependencies globally because SPARK-47118 (4.0.0) upgrades Jersey from 2 to 3, using an unmatched Jersey version in modules like Spark SQL engine would fail Spark to start thus abort UT.
## Types of changes 🔖
- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Tested with Spark 4.0
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6415 from pan3793/server-jersey.
Closes#6415
804d6367b [Cheng Pan] nit
fcb6ee573 [Cheng Pan] manage jersey in kyuubi-server module
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
