Bumps [io.netty:netty-codec-http2](https://github.com/netty/netty) from 4.1.93.Final to 4.1.100.Final.
<details>
<summary>Commits</summary>
<ul>
<li><a href="58df783eb4"><code>58df783</code></a> [maven-release-plugin] prepare release netty-4.1.100.Final</li>
<li><a href="58f75f665a"><code>58f75f6</code></a> Merge pull request from GHSA-xpw8-rcwv-8f8p</li>
<li><a href="491144865a"><code>4911448</code></a> Do not fail when compressing empty HttpContent (<a href="https://redirect.github.com/netty/netty/issues/13655">#13655</a>)</li>
<li><a href="caca5e5a1e"><code>caca5e5</code></a> When read PoolSubpage's variant fields, it should lock on PoolSubpage's head ...</li>
<li><a href="d97f2a5606"><code>d97f2a5</code></a> Update checkout action to latest version (<a href="https://redirect.github.com/netty/netty/issues/13649">#13649</a>)</li>
<li><a href="275341f01c"><code>275341f</code></a> Fix issue with unrecognized JVM option while running with Java 11 (<a href="https://redirect.github.com/netty/netty/issues/13648">#13648</a>)</li>
<li><a href="5db037beed"><code>5db037b</code></a> Speedup max direct memory estimation via Unsafe (<a href="https://redirect.github.com/netty/netty/issues/13643">#13643</a>)</li>
<li><a href="ce5c78cec1"><code>ce5c78c</code></a> Update actions to the latest version (<a href="https://redirect.github.com/netty/netty/issues/13644">#13644</a>)</li>
<li><a href="d7a8169f1b"><code>d7a8169</code></a> [maven-release-plugin] prepare for next development iteration</li>
<li><a href="8e3fe28425"><code>8e3fe28</code></a> [maven-release-plugin] prepare release netty-4.1.99.Final</li>
<li>Additional commits viewable in <a href="https://github.com/netty/netty/compare/netty-4.1.93.Final...netty-4.1.100.Final">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
You can trigger a rebase of this PR by commenting `dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `dependabot rebase` will rebase this PR
- `dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `dependabot merge` will merge this PR after your CI passes on it
- `dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `dependabot cancel merge` will cancel a previously requested merge and block automerging
- `dependabot reopen` will reopen this PR if it is closed
- `dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency
- `dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/apache/kyuubi/network/alerts).
</details>
> **Note**
> Automatic rebases have been disabled on this pull request as it has been open for over 30 days.
Closes#5395 from dependabot[bot]/dependabot/maven/io.netty-netty-codec-http2-4.1.100.Final.
Closes#5395
51028b289 [Cheng Pan] remove obsolete comments
a8f916c0f [Bowen Liang] update dependencyList
6813ccb4e [dependabot[bot]] Bump io.netty:netty-codec-http2 from 4.1.93.Final to 4.1.100.Final
Lead-authored-by: Bowen Liang <liangbowen@gf.com.cn>
Co-authored-by: Cheng Pan <chengpan@apache.org>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Update codecov token to fix stale code coverage action
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
no
Closes#5501 from yaooqinn/token.
Closes#5501
1f4053231 [Kent Yao] java-17
fcd8391e7 [Kent Yao] java-17
5ccead77d [Kent Yao] upgrade jacoco
5cb4a8374 [Kent Yao] Revert "rm"
45d7a2134 [Kent Yao] rm
0a3dd2f42 [Kent Yao] Update codecov token
Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
- Bump Kafka client version from 3.4.0 to 3.5.1
- Bump Docker image for Kafka 3 to 3.5.0 (equivalent to Confluent Platform 7.5.x)
- https://docs.confluent.io/platform/current/installation/versions-interoperability.html#cp-and-apache-ak-compatibility
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No.
Closes#5486 from bowenliang123/bump-kafka.
Closes#5486
017ea860d [Bowen Liang] update dependencyList
20ad988ed [liangbowen] bump kafka client version 3.5.1
Lead-authored-by: liangbowen <liangbowen@gf.com.cn>
Co-authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Close https://github.com/apache/kyuubi/issues/5484
Kyuubi provides a basic new Web UI which is built on top of Vue3, we can remove the legacy dummy Web UI in 1.8.
The new UI hosts at `http://<host>:<port>/ui/` and the legacy UI hosts at `http://<host>:<port>/`, we should
1. Remove the legacy UI routing from Jetty
2. Remove all files related to legacy UI
3. Redirect `http://<host>:<port>/` to `http://<host>:<port>/ui/`
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
building with the command `build/dist --web-ui`, then `cd dist` and perform `bin/kyuubi run`
access http://0.0.0.0:10099 could correctly redirect to http://0.0.0.0:10099/ui/
<img width="1428" alt="image" src="https://github.com/apache/kyuubi/assets/26535726/1e8a67f6-e4db-415e-8a47-dd7c41b487cf">
swagger is render correctly too.
<img width="1428" alt="image" src="https://github.com/apache/kyuubi/assets/26535726/1cb4ba31-9965-4468-b7c3-b0319ba959e6">
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No.
Closes#5516 from pan3793/5484.
Closes#5484
9d58ef758 [Cheng Pan] address comment and fix test
6d4c0981b [Cheng Pan] [KYUUBI #5484] Remove legacy Web UI
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
- With this PR, the code of module `kyuubi-kubernetes-it` will be auto-reformated by running `dev/reformat` script.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No.
Closes#5443 from bowenliang123/reformat-k8sit.
Closes#5443
39a323efc [Bowen Liang] reformat k8s it module
Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
### _Why are the changes needed?_
It is basically copied from `kyuubi-extension-spark-3-4`.
### _How was this patch tested?_
Compiled successfully:
```
build/mvn clean install -DskipTests -Pflink-provided,spark-provided,hive-provided,spark-3.5
```
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No
Closes#5336 from wForget/dev_spark_3_5.
Closes#5336
7ba99804a [wforget] remove iceberg.version in spark-3.5 profile
a18ce166a [wforget] Regenerate KyuubiEnsureRequirements based on EnsureRequirements in spark 3.5
4725c4701 [wforget] fix iceberg version
f5a8ea934 [wforget] Bump iceberg 1.4.0
06558dcfa [wforget] make kyuubi-spark-authz plugin compatible with Spark3.5
90d0e4c70 [wforget] make kyuubi-spark-authz plugin compatible with Spark3.5
4bc8d24d6 [wforget] add ci
1b3f2d916 [wforget] Make kyuubi spark extension compatible with Spark3.5
Authored-by: wforget <643348094@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
upgrade snakeyaml from 1.33 to 2.2 reducing direct CVE vulnerabilities, see (https://bitbucket.org/snakeyaml/snakeyaml/wiki/Changes)
[CVE-2022-1471](https://nvd.nist.gov/vuln/detail/CVE-2022-1471)
SnakeYaml's Constructor() class does not restrict types which can be instantiated during deserialization. Deserializing yaml content provided by an attacker can lead to remote code execution. We recommend using SnakeYaml's SafeConsturctor when parsing untrusted content to restrict deserialization. We recommend upgrading to version 2.0 and beyond.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No
Closes#5293 from dev-lpq/snakeyaml_critical.
Closes#5293
5b2412d8e [pengqli] upgrade snakeyaml from 1.33 to 2.2
Authored-by: pengqli <pengqli@cisco.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- Extract common assertion method for verifying file contents
- Ensure integrity of the file by comparing the line count
- Correct the script name for Spark engine KDF doc generation from `gen_kdf.sh` to `gen_spark_kdf_docs.sh`
- Add `gen_hive_kdf_docs.sh` script for Hive engine KDF doc generation
- Fix incorrect hints for Ranger spec file generation
- shows the line number of the incorrect file content
- Streamingly read file content by line with buffered support
- Regeneration hints:
<img width="656" alt="image" src="https://github.com/apache/kyuubi/assets/1935105/d1a7cb70-8b63-4fe9-ae27-80dadbe84799">
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No.
Closes#5275 from bowenliang123/doc-regen-hint.
Closes#5275
9af97ab86 [Bowen Liang] implicit source position
07020c74d [liangbowen] assertFileContent
Lead-authored-by: liangbowen <liangbowen@gf.com.cn>
Co-authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
### _Why are the changes needed?_
- Apache Commons Lang 3.13.0 release note: https://commons.apache.org/proper/commons-lang/changes-report.html#a3.13.0
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No.
Closes#5241 from bowenliang123/commonslang3-3.13.
Closes#5241
99e5ffbd9 [Bowen Liang] update dependencyList
f2bf9c554 [Bowen Liang] bump commons lang3 to 3.13.0
Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
### _Why are the changes needed?_
- Change hardcoded Scala's version 2.12 in Maven module's `artifactId` to placeholder `scala.binary.version` which is defined in project parent pom as 2.12
- Preparation for Scala 2.13/3.x support in the future
- No impact on using or building Maven modules
- Some ignorable warning messages for unstable artifactId will be thrown by Maven.
```
Warning: Some problems were encountered while building the effective model for org.apache.kyuubi:kyuubi-server_2.12🫙1.8.0-SNAPSHOT
Warning: 'artifactId' contains an expression but should be a constant
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No.
Closes#5175 from bowenliang123/artifactId-scala.
Closes#5177
2eba29cfa [liangbowen] use placeholder of scala binary version for artifactId
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- extract development scripts for regenerating and verifying the golden files
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
Closes#5121 from bowenliang123/gen-golden.
Closes#5121
a8fb20046 [liangbowen] add golden result file path hint in comment
444e5aff9 [liangbowen] nit
eec4fe84f [liangbowen] use the generation scripts for running test suites.
5a97785e6 [liangbowen] fix class name in gen_ranger_spec_json.sh
3be9aacf5 [liangbowen] comments
32993de43 [liangbowen] comments
ca0090909 [liangbowen] update scripts
6439ad3e9 [liangbowen] update scripts
f26d77935 [liangbowen] add scripts for golden files' generation
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
- Show prompt if milestone or assignees not set when merging pull requests.
<img width="611" alt="image" src="https://github.com/apache/kyuubi/assets/1935105/4f4df661-14ab-45e4-bcfe-9050549048e6">
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
Closes#5152 from bowenliang123/merge-check.
Closes#5152
3a1731d40 [liangbowen] nit
20eb0e2ad [liangbowen] print
51a268e71 [liangbowen] check and print milestone and assignees of pull requests
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
https://hadoop.apache.org/release/3.3.6.html
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
Closes#5116 from pan3793/hadoop-3.3.6.
Closes#5116
c3717e7fb [Cheng Pan] Bump Hadoop 3.3.6
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
It was planned but actually delayed, remove this dummy module to save CI and avoid confusing users and release managers.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
Closes#5113 from pan3793/remove-kudu.
Closes#5113
ff8fd2e6a [Cheng Pan] Remove Spark Kudu connector
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Now the kubernetes client version in apache/spark is 6.7.2(https://github.com/apache/spark/pull/41490), I think it is better to align with that.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
Closes#5059 from turboFei/k8s_client_version.
Closes#5059
79955d17a [fwang12] deps
4a68cc820 [fwang12] bump
Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
The main change is copy code from `kyuubi-extension-spark-common_2.12` and `kyuubi-extension-spark-3-3`. The reason copy `kyuubi-extension-spark-common_2.12` is that, Spark3.4 v1 writes does not have ctas...
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
Closes#5018 from ulysses-you/extension.
Closes#5018
1ca4951e1 [ulysses-you] fix
fb20bb1f0 [ulysses-you] fix
338a3120a [ulysses-you] rebase
241a436fd [ulysses-you] fix
70618f368 [ulysses-you] fix
5f5e2b7c8 [ulysses-you] fix
301ae5c04 [ulysses-you] fix
bb9416811 [ulysses-you] fix
a38595d0a [ulysses-you] copy
9dec27d6c [ulysses-you] copy
Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: ulyssesyou <ulyssesyou@apache.org>
### _Why are the changes needed?_
This pr amins to make KSHC support Apache Spark 3.4.
- KSHC support Apache Spark 3.4
- Make Apache kyuubi `codecov` module contain the spark-3.4 profile. so that Apache kyubbi CI can cover some modules.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
Closes#4999 from Yikf/kudu-spark3.4.
Closes#4999
6a35e54b8 [Cheng Pan] Update extensions/spark/kyuubi-spark-connector-hive/src/main/scala/org/apache/kyuubi/spark/connector/hive/HiveConnectorUtils.scala
66bb742eb [Cheng Pan] Update extensions/spark/kyuubi-spark-connector-hive/src/main/scala/org/apache/kyuubi/spark/connector/hive/HiveConnectorUtils.scala
7be517c7f [Cheng Pan] Update extensions/spark/kyuubi-spark-connector-hive/src/main/scala/org/apache/kyuubi/spark/connector/hive/HiveConnectorUtils.scala
ae23133d1 [Cheng Pan] Update extensions/spark/kyuubi-spark-connector-hive/src/main/scala/org/apache/kyuubi/spark/connector/hive/HiveConnectorUtils.scala
dda5e6521 [Cheng Pan] Update extensions/spark/kyuubi-spark-connector-hive/src/main/scala/org/apache/kyuubi/spark/connector/hive/HiveConnectorUtils.scala
e43a25dff [Cheng Pan] Update extensions/spark/kyuubi-spark-connector-hive/src/main/scala/org/apache/kyuubi/spark/connector/hive/HiveConnectorUtils.scala
54f52f16d [Cheng Pan] Update extensions/spark/kyuubi-spark-connector-hive/src/main/scala/org/apache/kyuubi/spark/connector/hive/HiveConnectorUtils.scala
0955b544b [Cheng Pan] Update pom.xml
38a1383d9 [yikaifei] codecov module should contain the spark 3.4 profile
Lead-authored-by: Cheng Pan <pan3793@gmail.com>
Co-authored-by: yikaifei <yikaifei@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- Bump Scala to 2.12.18
- release note: https://github.com/scala/scala/releases/tag/v2.12.18
- Bump silencer-plugin from 1.7.10 to 1.7.13 for compatibility
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
Closes#5001 from bowenliang123/scala-2.12.18.
Closes#5001
e4a5b9b0d [liangbowen] update dependencyList
e847bbced [liangbowen] Bump silencer-plugin from 1.7.10 to 1.7.13
daceece66 [liangbowen] Bump scala from 2.12.17 to 2.12.18
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
Upgrade Netty to the latest Arrow-compatible version: 4.1.93.Final
Currently, we can not upgrade to 4.1.94.Final or above because of https://github.com/netty/netty/pull/13408
```
java.lang.NoSuchMethodError: 'io.netty.buffer.PoolThreadCache io.netty.buffer.PooledByteBufAllocatorL$InnerAllocator.threadCache()'
at io.netty.buffer.PooledByteBufAllocatorL$InnerAllocator.newDirectBufferL(PooledByteBufAllocatorL.java:164)
at io.netty.buffer.PooledByteBufAllocatorL$InnerAllocator.directBuffer(PooledByteBufAllocatorL.java:214)
at io.netty.buffer.PooledByteBufAllocatorL.allocate(PooledByteBufAllocatorL.java:58)
at org.apache.arrow.memory.NettyAllocationManager.<init>(NettyAllocationManager.java:77)
at org.apache.arrow.memory.NettyAllocationManager.<init>(NettyAllocationManager.java:84)
at org.apache.arrow.memory.NettyAllocationManager$1.create(NettyAllocationManager.java:34)
at org.apache.arrow.memory.BaseAllocator.newAllocationManager(BaseAllocator.java:354)
at org.apache.arrow.memory.BaseAllocator.newAllocationManager(BaseAllocator.java:349)
at org.apache.arrow.memory.BaseAllocator.bufferWithoutReservation(BaseAllocator.java:337)
at org.apache.arrow.memory.BaseAllocator.buffer(BaseAllocator.java:315)
at org.apache.arrow.memory.BaseAllocator.buffer(BaseAllocator.java:279)
at org.apache.arrow.vector.BaseVariableWidthVector.allocateBytes(BaseVariableWidthVector.java:462)
at org.apache.arrow.vector.BaseVariableWidthVector.allocateNew(BaseVariableWidthVector.java:420)
at org.apache.arrow.vector.BaseVariableWidthVector.allocateNew(BaseVariableWidthVector.java:380)
at org.apache.spark.sql.execution.arrow.ArrowWriter$.$anonfun$create$1(ArrowWriter.scala:42)
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
Closes#4992 from pan3793/netty.
Closes#4992
9dd8f9ad0 [Cheng Pan] nit
e6953144e [Cheng Pan] Bump Netty 4.1.93.Final
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Apache Derby is no longer active. One major drawback of Derby is that it does not support multiple connections to single db files, making it hard to analyze the data on local development.
SQLite may be the most popular embedded DBMS in the world. It lives almost in every smartphone (at least Android and iOS integrate SQLite), which means SQLite is quite stable and may be a good choice for standalone production deployment.
SQLite provides a CLI command `sqlite3` which is easy to use to connect a data file and run queries. Multi connections to a single db file is allowed, which helps a lot to analyze the data when the Kyuubi server is running.
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4950 from pan3793/sqlite.
Closes#4950
738c39e6a [Cheng Pan] Update docs/deployment/migration-guide.md
5facdad9c [Cheng Pan] Update docs/deployment/migration-guide.md
b9883489c [Cheng Pan] migrate default metadata store to sqlite
c785e1a77 [Cheng Pan] migrate jdbc auth test to sqlite
23f63b932 [Cheng Pan] introduce sqlite deps
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Kyuubi Server Proxy Engine UI
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4795 from zwangsheng/KYUUBI_3420.
Closes#3420
079dc1c60 [zwangsheng] fix frontend unit test case
6e71b4518 [Cheng Pan] fix
cf7ca5145 [Cheng Pan] Update kyuubi-server/src/main/scala/org/apache/kyuubi/server/api/v1/ApiRootResource.scala
9a91d62a0 [Cheng Pan] polish
a5dcfae18 [zwangsheng] fix
5d4a8c239 [zwangsheng] Rebase
71d22fc9a [zwangsheng] fix
3b0152f33 [zwangsheng] [KYUUBI #3420][UI] Proxy Engnie UI
Lead-authored-by: zwangsheng <2213335496@qq.com>
Co-authored-by: Cheng Pan <chengpan@apache.org>
Co-authored-by: Cheng Pan <pan3793@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
This PR aims to migrate the vanilla Zookeeper and Curator to the Kyuubi Shaded Zookeeper. It's the first step to adapting JDK 17.
There is a known issue [ZOOKEEPER-3779](https://issues.apache.org/jira/browse/ZOOKEEPER-3779) that Zookeeper 3.4 client can not run on JDK 14 and above, in https://github.com/apache/kyuubi-shaded/pull/5, we fixed this issue by a surgical.
With the above fixing, zk-3.4 and zk-3.6 clients both work well on JDK 17, we just randomly pick some cases to make sure zk-3.6 is tested
zk-3.4 client supports zk-3.4+ server, but zk-3.6 client only supports zk-3.5+ server; in the meanwhile, zk-3.4 is adopted widely, (CDH 5/6, HDP, EMR created before 2023).
We are sticky to zk-3.4 to ensure that Kyuubi can be out-of-box in the most existing Hadoop cluster but also provide zk-3.6 as an alternative(simply replace the kyuubi-shaded-zk-3.4 jar w/ kyuubi-shaded-zk-3.6, or build w/ -Pzookeeper-3.6) for users who concerns that zk-3.4 is EOL.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4852 from pan3793/shaded-zk.
Closes#4852
d960cc945 [Cheng Pan] remove staging repo
1b3622080 [Cheng Pan] Switch to Kyuubi Shaded Zookeeper
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
https://arrow.apache.org/release/12.0.0.html
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4827 from cfmcgrady/arrow-12.0.0.
Closes#4827
324a8ddbf [Fu Chen] bump arrow from 11.0.0 to 12.0.0
Authored-by: Fu Chen <cfmcgrady@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
spark bump jackson from 2.14.2 to 2.15.0 in https://github.com/apache/spark/pull/40933
to fix
https://github.com/apache/kyuubi/actions/runs/4943800010/jobs/8838642303
```
Caused by: org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 1.0 failed 1 times, most recent failure: Lost task 0.0 in stage 1.0 (TID 1) (localhost executor driver): java.lang.NoClassDefFoundError: com/fasterxml/jackson/core/StreamReadConstraints
at org.apache.spark.sql.catalyst.json.JSONOptions.buildJsonFactory(JSONOptions.scala:195)
at org.apache.spark.sql.catalyst.json.JsonInferSchema.$anonfun$infer$1(JsonInferSchema.scala:83)
at org.apache.spark.rdd.RDD.$anonfun$mapPartitions$2(RDD.scala:855)
at org.apache.spark.rdd.RDD.$anonfun$mapPartitions$2$adapted(RDD.scala:855)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:364)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:328)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:92)
at org.apache.spark.TaskContext.runTaskWithListeners(TaskContext.scala:161)
at org.apache.spark.scheduler.Task.run(Task.scala:139)
at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:554)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1514)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:557)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.ClassNotFoundException: com.fasterxml.jackson.core.StreamReadConstraints
at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 16 more
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4824 from cfmcgrady/jackson-2.15.0.
Closes#4824
7a8a3de89 [Fu Chen] update dev/dependencyList
2d01b4b9d [Fu Chen] bump jackson
Authored-by: Fu Chen <cfmcgrady@gmail.com>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
- to fix 2 Dependabot alerts
1. https://github.com/apache/kyuubi/security/dependabot/19
2. https://github.com/apache/kyuubi/security/dependabot/18
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4808 from bowenliang123/jetty-9.4.51.
Closes#4808
84789292b [liangbowen] update dependencyList
d1ebdc68a [liangbowen] Revert "update dependencyList"
e4e26b7b5 [liangbowen] update dependencyList
6cfb37dbb [liangbowen] bump jetty to 9.4.51
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: bowenliang <bowenliang@apache.org>
### _Why are the changes needed?_
- introduce new event logger type `KAFKA`
- send server events to the Kafka topic with initializing and closing Kafka producer properly with server's lifecyle
- use Kafka 3.4.0 as the client version, and tested with Kakfa servers of 2.8.x and 3.4.x
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4733 from bowenliang123/kafka-logger.
Closes#4733
b5220d234 [liangbowen] introduce kafka server event logger
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
Upgrade Hadoop from 3.3.4 to 3.3.5, close#4652
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4689 from huapan123456/master.
Closes#4652
810af9848 [Cheng Pan] fix
380786e9f [Cheng Pan] nit
7c5f18ef9 [Cheng Pan] nit
2d308943b [Cheng Pan] nit
76869d822 [Cheng Pan] handle HDFS-16591
36142c787 [huapan] fix(KYUUBI #4652): the position of JaasConfiguration has changed on hadoop-clint-api-3.3.5
1b5de6dfe [huapan] feat(KYUUBI #4652): upgrade hadoop-version from 3.3.4 to 3.3.5
f9d34c716 [huapan] feat(KYUUBI #4652): upgrade hadoop-client-api from 3.3.4 to 3.3.5
Lead-authored-by: Cheng Pan <chengpan@apache.org>
Co-authored-by: huapan <huapan@52tt.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
close#4325
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4417 from yehere/kyuubi-4325.
Closes#4325
7b2864b53 [yehere] [KYUUBI #4325] Support replace preparedStatement for Trino-jdbc,update the dependency file, run './build/dependency.sh --replace'
749b1c15c [yehere] [KYUUBI #4325] Support replace preparedStatement for Trino-jdbc,Code optimization
33ea9ba2b [yehere] [KYUUBI #4325] Support replace preparedStatement for Trino-jdbc,Code optimization
568418a21 [yehere] [KYUUBI #4325] Support replace preparedStatement for Trino-jdbc,Add test case for DEALLOCATE PREPARE
358a8e3b8 [yehere] [KYUUBI #4325] Support replace preparedStatement for Trino-jdbc,Support DEALLOCATE PREPARE
7d4a32402 [yehere] [KYUUBI #4325] Support replace preparedStatement for Trino-jdbc
91392add6 [yehere] [KYUUBI #4325] Support replace preparedStatement for Trino-jdbc
63bf8c462 [yehere] [KYUUBI #4325] Support replace preparedStatement for Trino-jdbc
f5b7fb786 [yehere] [KYUUBI #4325] Support replace preparedStatement for Trino-jdbc
b0476a79d [yehere] [KYUUBI #4325] Support replace preparedStatement for Trino-jdbc
1a8f147a0 [yehere] [KYUUBI #4325] Support replace preparedStatement for Trino-jdbc
Authored-by: yehere <867171931@qq.com>
Signed-off-by: ulyssesyou <ulyssesyou@apache.org>
### _Why are the changes needed?_
- log4j 2.20.0 release notes: https://logging.apache.org/log4j/2.x/release-notes/2.20.0.html
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4447 from bowenliang123/log4j-2.20.0.
Closes#4447
c9bc9aa95 [liangbowen] bump log4j from 2.19.0 to 2.20.0
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
to close#4357.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4366 from bowenliang123/jersey-2.39.
Closes#4357
bd214e8d [liangbowen] bump jersey from 2.38 to 2.39
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- Include `**/README.md` markdown files in spotless style check
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4312 from bowenliang123/reformat-readme.
Closes#4312
1fda1bdeb [Bowen Liang] Merge branch 'master' into reformat-readme
2ca8b4c81 [liangbowen] merge master
876f52a4c [liangbowen] include `**/README.md` in spotless style check
Lead-authored-by: liangbowen <liangbowen@gf.com.cn>
Co-authored-by: Bowen Liang <bowenliang@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Close#3935
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4232 from iodone/kyuubi-3935.
Closes#3935
936ea1f8 [odone] address
e7bd01a1 [odone] support trino client connect kyuubi trino server
9ea8b6af [odone] [WIP] trion request/response implementation
Authored-by: odone <odone.zhang@gmail.com>
Signed-off-by: ulyssesyou <ulyssesyou@apache.org>
### _Why are the changes needed?_
- Jackson 2.14.2 release note: https://github.com/FasterXML/jackson/wiki/Jackson-Release-2.14.2
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4286 from runzhliu/patch-4.
Closes#4286
6902e2ec [runzhliu] Update dependencyList
3cde2004 [runzhliu] Merge branch 'apache:master' into patch-4
8ac6eccd [runzhliu] bump the jackson maven dependecy version to 2.14.2
Authored-by: runzhliu <runzhliu@163.com>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
This PR proposes to enhance the LDAP support, which mainly referring the code introduced in HIVE-14713.
Currently, Kyuubi has very limited LDAP support, and the implementation is from the early Hive codebase. Hive enhanced the LDAP support in later versions, considering the Hive ecosystem is quite mature, I think we'd better to porting this functionality and keep the same behavior w/ Hive first, and we can improve it if meet certain requirements/issues in the future.
Basically, this PR introduces the following configurations
```
kyuubi.authentication.ldap.url (since 1.0.0)
kyuubi.authentication.ldap.domain (since 1.0.0)
kyuubi.authentication.ldap.guidKey (since 1.2.0)
kyuubi.authentication.ldap.base.dn (since 1.0.0 deprecated)
kyuubi.authentication.ldap.baseDN
kyuubi.authentication.ldap.groupMembershipKey
kyuubi.authentication.ldap.userMembershipKey
kyuubi.authentication.ldap.groupClassKey
kyuubi.authentication.ldap.groupDNPattern
kyuubi.authentication.ldap.userDNPattern
kyuubi.authentication.ldap.groupFilter
kyuubi.authentication.ldap.userFilter
kyuubi.authentication.ldap.customLDAPQuery
kyuubi.authentication.ldap.binddn
kyuubi.authentication.ldap.bindpw
```
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
This PR ports all LDAP-related UT&IT from Hive codebase
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4152 from pan3793/ldap.
Closes#4152
d251c959 [Cheng Pan] nit
6d14f44b [Cheng Pan] nit
6b3d116c [Cheng Pan] nit
ab47d822 [Cheng Pan] nit
a56e8702 [Cheng Pan] nit
4624619a [Cheng Pan] nit
b82c0c05 [Cheng Pan] LDAP test password uses alphanumeric
86a01cca [Cheng Pan] Enhance LDAP authentication
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4230 from turboFei/upgrade_http_version.
Closes#4230
10fe8a34 [fwang12] Bump httpclient and httpcore version
Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
- REST client support for creating batch with uploading resource file (https://kyuubi.readthedocs.io/en/master/client/rest/rest_api.html#post-batches-with-uploading-resource) introduced in #4144
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4221 from bowenliang123/restclient-batch-upload.
Closes#4221
9ec83766 [liangbowen] add MultiPart class to avoid exposing httpclient classes to RestClient interface
83051843 [liangbowen] fix content type
056edb84 [liangbowen] nit
74ad5642 [liangbowen] style
68dcdb38 [liangbowen] update dependencyList
a274d392 [liangbowen] client for creating batch with uploading resource file
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
- Bump snakeyaml from 1.31 to 1.33 reducing 2 direct CVE vulnerabilities, see (https://bitbucket.org/snakeyaml/snakeyaml/wiki/Changes)
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4207 from bowenliang123/snakeyaml-1.33.
Closes#4207
080024ce5 [liangbowen] bump snakeyaml from 1.31 to 1.33
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- use `Jackson` instead of `Gson` in Authz tests, where it is the only place `Gson` is used in project
- remove `Gson` as the dependency
- Notice: Gson project is in maintenance mode, (see https://github.com/google/gson/pull/2085)
- Gson is still on `dependencyList`, as depended in kyuubi-ha
<img width="640" alt="image" src="https://user-images.githubusercontent.com/1935105/213956635-ce079638-15c3-4de2-9f46-a93dfe3db199.png">
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4201 from bowenliang123/remove-gson.
Closes#4201
96df2513 [liangbowen] update dependencyList
81303d6e [liangbowen] imports
a73a1f98 [liangbowen] use jackson instead of gson in authz testing and remove gson as dependency
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
to close#4106 .
1. add `POST /batches`API in `BatchesResource` of REST API, which consumes `multipart/form-data` media type to support uploading `resourcefile`. And a `batchRequest` form data part in the JSON format string of `BatchRequest`, as in the required request body of `POST /batches`
2. the uploaded `resourceFile` is saved to a temp local file which will be cleaned up after job execution at the end of `submitAndMonitorBatchJob`
3. the local temp copy of `resourceFile` will be used as `resource` in `BatchJobSubmission`, eg. as <application-jar> for spark-submit
Todos in follow-up:
1. add a related description in Rest API doc
4. add `multipart` media type support to `RestClient` and implement `createBatchWithUploadingResource` in `BatchRestApi`
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4144 from bowenliang123/rest-batch-upload.
Closes#4106
f6723a02 [Bowen Liang] Merge branch 'master' into rest-batch-upload
0dd67245 [liangbowen] correct dependencyList
6365a0cd [liangbowen] introducing resource file upload support in batch creation
Lead-authored-by: liangbowen <liangbowen@gf.com.cn>
Co-authored-by: Bowen Liang <bowenliang@apache.org>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
- Bump Netty from `4.1.84.Final` to `4.1.87.Final` (release note: https://netty.io/news/2023/01/12/4-1-87-Final.html)
- with 2 CVE （ including 1 in high risk level) fixed in 4.1.86.Final for 4.1.85 and before, `CVE-2022-41915` and `CVE-2022-41881`
(https://netty.io/news/2022/12/12/4-1-86-Final.html)
- exclude `netty-handler-ssl-ocsp` which is released with `netty-all` since `4.1.86.Final`, as no SSL ocsp related feature used in kyuubi server
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4190 from bowenliang123/netty-4.1.87.
Closes#4190
97198aa9 [liangbowen] exclude netty-handler-ssl-ocsp from netty-all
44c3fab2 [liangbowen] update dependencyList
26a9ca7f [liangbowen] bump netty from 4.1.84 to 4.1.87
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
- bump `com.google.guava:guava` from 30.1 to 31.1 (released in Mar 2022, https://github.com/google/guava/releases/tag/v31.1)
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4174 from bowenliang123/guava-31.1.
Closes#4174
4376cd8a6 [liangbowen] update dependencyList
09b7f876e [liangbowen] bump guava from 30.1 to 31.1
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
Upgrade all outdated Apache Commons series dependencies released in 2020 to the latest versions released in 2021.
- bump `commons-io` from 2.8.0 to 2.11.0 (release notes: https://commons.apache.org/proper/commons-io/changes-report.html#a2.11.0)
- bump `commons-lang3` from 3.10 to 3.12.0 (release notes: https://commons.apache.org/proper/commons-lang/changes-report.html#a3.12.0)
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4132 from bowenliang123/bump-apache-commons.
Closes#4132
2b8c7150 [liangbowen] update dependencyList
385ecc11 [liangbowen] bump commons-lang3 from 3.10 to 3.12.0
a452b611 [liangbowen] bump commons-io from 2.8.0 to 2.11.0
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Release notes for Jersey:
- https://github.com/eclipse-ee4j/jersey/releases/tag/2.38
- https://github.com/eclipse-ee4j/jersey/releases/tag/2.37
Changes in this PR:
- `jakarta.activation-api` is now removed from `dependencyList`
- `jakarta.activation-api` added to `kyuubi-kubernetes-it` module's dependency in test scope for satisfying `hadoop-client-minicluster`
Facts:
- Before : with jersey 2.36,
- `kyuubi-kubernetes-it` relies on `hadoop-client-minicluster` in test scope
- `hadoop-client-minicluster` uses `DataSource` and other classes in package `javax.activation` without depending on the depencency
- `javax.activation` package is actually provied by `jakarta.activation-api` included by `jersey-media-json-jackson`
- Investigation: with jersey 2.38
- `jakarta.activation-api` was removed in Jersey 2.38 compared to 2.36 (https://github.com/eclipse-ee4j/jersey/compare/2.36...2.38)
- from the blame history, it's clear that `jakarta.activation-api` was explictly excluded from `jersey-media-json-jackson`
(https://github.com/eclipse-ee4j/jersey/blob/2.38/media/json-jackson/pom.xml), which in order to apply changes from `maven-enforcer-plugin` in (7e7d339281 (diff-f47eb7c28b888c017334e057492d4d2eab5be3e0e02bea1ff10765542bf9b9c8R101))
- Solution:
- added `jakarta.activation-api` to `kyuubi-kubernetes-it` module's dependency in test scope statisfying `hadoop-client-minicluster`
- confirm `jakarta.activation-api` is no longer directly or indirectly dependent in maven runtime scope
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4082 from bowenliang123/jersey-2.38.
Closes#4082
88c7cd80 [liangbowen] move jakarta.activation-api dependency closer to its dependant hadoop-client-minicluster
ce6df817 [liangbowen] add jakarta.activation-api dependency in test scope to kubernetes-integration-tests_2.1 module
8422dc2f [Bowen Liang] Merge branch 'master' into jersey-2.38
606cad0a [liangbowen] Bump Jersey from 2.36 to 2.38
Lead-authored-by: liangbowen <liangbowen@gf.com.cn>
Co-authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4069 from lightning-L/kyuubi-4020.
Closes#4020
97406ca0 [Tianlin Liao] [KYUUBI #4020] remove incubating from kyuubi source code
Authored-by: Tianlin Liao <tiliao@ebay.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
This PR aims to bump [antlr4](https://github.com/antlr/antlr4) from 4.8 to 4.9.3.
Version 4.8 was released on Jan 21, 2020, It's been nearly three years, and there have been a lot of bug fixes during that time, see more at [change list](https://github.com/antlr/antlr4/compare/4.8...4.9.3).
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4039 from Yikf/bump-antlr4.
Closes#4039
24aed3a7 [Yikf] Bump Antlr4 from 4.8 to 4.9.3
Authored-by: Yikf <yikaifei1@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Close https://github.com/apache/incubator-kyuubi/issues/3926, we intend to introduce the parser module based on antlr4 in the Apache kyuubi server. Through this module, we can achieve:
1. Expand kyuubi capabilities, for example, implement `SHOW KYUUBI_SESSIONS` through SQL.
2. Sql translation for mysql front-end protocol.
3. Sql translation for trino front-end protocol.
This issue is the first step of the parser module, which introduces the parser module and implements `SHOW KYUUBI_SESSION`, and then improves the Apache kyuubi parser module based on this initial pr.
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3944 from Yikf/parser-init.
Closes#3926
4463b298 [Yikf] [KYUUBI #3926] Introduce antlr4 to parse query statement
Authored-by: Yikf <yikaifei1@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
close#3905
### _Why are the changes needed?_
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3977 from iodone/kyuubi-3905.
Closes#3977Closes#3905
4e4a22b1 [odone] trino server init and run
Authored-by: odone <odone.zhang@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
to close#3911 .
Bump Jackson from 2.13.4 to 2.14.1.
- From 2.14, Java8 is required
- v2.14.1 fixed heap consumption increased from 2.13.x to 2.14.0
Relase notes for 2.14 and 2.14.1
https://github.com/FasterXML/jackson/wiki/Jackson-Release-2.14https://github.com/FasterXML/jackson/wiki/Jackson-Release-2.14.1
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3912 from bowenliang123/3911-jackson-214.
Closes#3911
5832471e9 [liangbowen] update dependencyList
669249117 [liangbowen] Bump Jackson from 2.13.4 to 2.14.1
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
Introduce code style check support for Maven's pom.xml with sortPom in spotless maven plugin.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3843 from bowenliang123/spotless-pom.
Closes#3842
3c654597 [liangbowen] apply to pom.xml
fd1536f7 [liangbowen] set expandEmptyElements to true
e498423f [liangbowen] apply spotless:apply to all pom.xml
e46bcfec [liangbowen] add pom style check support in spotless
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3834 from leoluan2009/slf4j.
Closes#3834
e17b08ef [Xuedong Luan] Upgrade SLF4J from 1.7.35 to 1.7.36
Authored-by: Xuedong Luan <luanxuedong2009@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
to close#3828.
Python code style checking support.
1. reuse Spotless maven plugin for Python style check
2. add style check to CI style workflow
3. add python linting support to `dev/reformat`. checks whether `black` installed in PATH.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3823 from bowenliang123/spotless-python.
Closes#3828
4a4de885 [liangbowen] simplify empty tags
0bb9ec7c [liangbowen] simplify empty tag in pom
9dd39531 [liangbowen] lint python code with black via spotless
f85020fa [liangbowen] typo
4c93bce0 [liangbowen] install python 3.9 first
23fc4b96 [liangbowen] ci install black version from added `spotless.python.black.version` property
73f746b0 [Bowen Liang] Update dev/reformat
46667a00 [liangbowen] update style.yml
9c20b434 [liangbowen] update style.yml
21017e5e [liangbowen] update style.yml
8272c0bc [liangbowen] add python style to style checking for CI
e102726c [liangbowen] add profile spotless in dev/reformat if black found in path
062e9bf2 [liangbowen] add python scan for spotless. add new profile `spotless-python` for python file path.
Lead-authored-by: liangbowen <liangbowen@gf.com.cn>
Co-authored-by: Bowen Liang <bowen.liang.123@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
**Fix the following bug:**
```
2022-11-16 13:08:45.540 ERROR org.apache.kyuubi.server.KyuubiTBinaryFrontendService: Error opening session:
org.apache.kyuubi.KyuubiSQLException: Error opening session for test_user client ip 127.0.0.1, due to com/google/common/util/concurrent/internal/InternalFutureFailureAccess
at org.apache.kyuubi.KyuubiSQLException$.apply(KyuubiSQLException.scala:69) ~[kyuubi-common_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.kyuubi.session.KyuubiSessionManager.openSession(KyuubiSessionManager.scala:102) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.kyuubi.service.AbstractBackendService.openSession(AbstractBackendService.scala:45) ~[kyuubi-common_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.kyuubi.server.KyuubiServer$$anon$1.org$apache$kyuubi$server$BackendServiceMetric$$super$openSession(KyuubiServer.scala:111) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.kyuubi.server.BackendServiceMetric.$anonfun$openSession$1(BackendServiceMetric.scala:37) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.kyuubi.metrics.MetricsSystem$.timerTracing(MetricsSystem.scala:111) ~[kyuubi-metrics_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.kyuubi.server.BackendServiceMetric.openSession(BackendServiceMetric.scala:37) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.kyuubi.server.BackendServiceMetric.openSession$(BackendServiceMetric.scala:30) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.kyuubi.server.KyuubiServer$$anon$1.openSession(KyuubiServer.scala:111) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.kyuubi.service.TFrontendService.getSessionHandle(TFrontendService.scala:181) ~[kyuubi-common_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.kyuubi.server.KyuubiTBinaryFrontendService.OpenSession(KyuubiTBinaryFrontendService.scala:81) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1497) ~[hive-service-rpc-3.1.3.jar:3.1.3]
at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1482) ~[hive-service-rpc-3.1.3.jar:3.1.3]
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) ~[libthrift-0.9.3.jar:0.9.3]
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) ~[libthrift-0.9.3.jar:0.9.3]
at org.apache.kyuubi.service.authentication.TSetIpAddressProcessor.process(TSetIpAddressProcessor.scala:36) ~[kyuubi-common_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) ~[libthrift-0.9.3.jar:0.9.3]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_222]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_222]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_222]
Caused by: java.lang.NoClassDefFoundError: com/google/common/util/concurrent/internal/InternalFutureFailureAccess
at java.lang.ClassLoader.defineClass1(Native Method) ~[?:1.8.0_222]
at java.lang.ClassLoader.defineClass(ClassLoader.java:763) ~[?:1.8.0_222]
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) ~[?:1.8.0_222]
at java.net.URLClassLoader.defineClass(URLClassLoader.java:468) ~[?:1.8.0_222]
at java.net.URLClassLoader.access$100(URLClassLoader.java:74) ~[?:1.8.0_222]
at java.net.URLClassLoader$1.run(URLClassLoader.java:369) ~[?:1.8.0_222]
at java.net.URLClassLoader$1.run(URLClassLoader.java:363) ~[?:1.8.0_222]
at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_222]
at java.net.URLClassLoader.findClass(URLClassLoader.java:362) ~[?:1.8.0_222]
at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_222]
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[?:1.8.0_222]
at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_222]
at java.lang.ClassLoader.defineClass1(Native Method) ~[?:1.8.0_222]
at java.lang.ClassLoader.defineClass(ClassLoader.java:763) ~[?:1.8.0_222]
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) ~[?:1.8.0_222]
at java.net.URLClassLoader.defineClass(URLClassLoader.java:468) ~[?:1.8.0_222]
at java.net.URLClassLoader.access$100(URLClassLoader.java:74) ~[?:1.8.0_222]
at java.net.URLClassLoader$1.run(URLClassLoader.java:369) ~[?:1.8.0_222]
at java.net.URLClassLoader$1.run(URLClassLoader.java:363) ~[?:1.8.0_222]
at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_222]
at java.net.URLClassLoader.findClass(URLClassLoader.java:362) ~[?:1.8.0_222]
at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_222]
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[?:1.8.0_222]
at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_222]
at java.lang.ClassLoader.defineClass1(Native Method) ~[?:1.8.0_222]
at java.lang.ClassLoader.defineClass(ClassLoader.java:763) ~[?:1.8.0_222]
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) ~[?:1.8.0_222]
at java.net.URLClassLoader.defineClass(URLClassLoader.java:468) ~[?:1.8.0_222]
at java.net.URLClassLoader.access$100(URLClassLoader.java:74) ~[?:1.8.0_222]
at java.net.URLClassLoader$1.run(URLClassLoader.java:369) ~[?:1.8.0_222]
at java.net.URLClassLoader$1.run(URLClassLoader.java:363) ~[?:1.8.0_222]
at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_222]
at java.net.URLClassLoader.findClass(URLClassLoader.java:362) ~[?:1.8.0_222]
at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_222]
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[?:1.8.0_222]
at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_222]
at com.google.common.cache.LocalCache$LoadingValueReference.<init>(LocalCache.java:3472) ~[guava-30.1-jre.jar:?]
at com.google.common.cache.LocalCache$LoadingValueReference.<init>(LocalCache.java:3476) ~[guava-30.1-jre.jar:?]
at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2134) ~[guava-30.1-jre.jar:?]
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045) ~[guava-30.1-jre.jar:?]
at com.google.common.cache.LocalCache.get(LocalCache.java:3951) ~[guava-30.1-jre.jar:?]
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974) ~[guava-30.1-jre.jar:?]
at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4935) ~[guava-30.1-jre.jar:?]
at org.apache.kyuubi.session.FileSessionConfAdvisor.getConfOverlay(FileSessionConfAdvisor.scala:41) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.kyuubi.session.KyuubiSessionImpl.<init>(KyuubiSessionImpl.scala:55) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.kyuubi.session.KyuubiSessionManager.createSession(KyuubiSessionManager.scala:83) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.kyuubi.session.SessionManager.openSession(SessionManager.scala:98) ~[kyuubi-common_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.kyuubi.session.KyuubiSessionManager.openSession(KyuubiSessionManager.scala:95) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
... 18 more
Caused by: java.lang.ClassNotFoundException: com.google.common.util.concurrent.internal.InternalFutureFailureAccess
at java.net.URLClassLoader.findClass(URLClassLoader.java:382) ~[?:1.8.0_222]
at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_222]
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[?:1.8.0_222]
at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_222]
at java.lang.ClassLoader.defineClass1(Native Method) ~[?:1.8.0_222]
at java.lang.ClassLoader.defineClass(ClassLoader.java:763) ~[?:1.8.0_222]
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) ~[?:1.8.0_222]
at java.net.URLClassLoader.defineClass(URLClassLoader.java:468) ~[?:1.8.0_222]
at java.net.URLClassLoader.access$100(URLClassLoader.java:74) ~[?:1.8.0_222]
at java.net.URLClassLoader$1.run(URLClassLoader.java:369) ~[?:1.8.0_222]
at java.net.URLClassLoader$1.run(URLClassLoader.java:363) ~[?:1.8.0_222]
at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_222]
at java.net.URLClassLoader.findClass(URLClassLoader.java:362) ~[?:1.8.0_222]
at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_222]
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[?:1.8.0_222]
at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_222]
at java.lang.ClassLoader.defineClass1(Native Method) ~[?:1.8.0_222]
at java.lang.ClassLoader.defineClass(ClassLoader.java:763) ~[?:1.8.0_222]
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) ~[?:1.8.0_222]
at java.net.URLClassLoader.defineClass(URLClassLoader.java:468) ~[?:1.8.0_222]
at java.net.URLClassLoader.access$100(URLClassLoader.java:74) ~[?:1.8.0_222]
at java.net.URLClassLoader$1.run(URLClassLoader.java:369) ~[?:1.8.0_222]
at java.net.URLClassLoader$1.run(URLClassLoader.java:363) ~[?:1.8.0_222]
at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_222]
at java.net.URLClassLoader.findClass(URLClassLoader.java:362) ~[?:1.8.0_222]
at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_222]
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[?:1.8.0_222]
at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_222]
at java.lang.ClassLoader.defineClass1(Native Method) ~[?:1.8.0_222]
at java.lang.ClassLoader.defineClass(ClassLoader.java:763) ~[?:1.8.0_222]
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) ~[?:1.8.0_222]
at java.net.URLClassLoader.defineClass(URLClassLoader.java:468) ~[?:1.8.0_222]
at java.net.URLClassLoader.access$100(URLClassLoader.java:74) ~[?:1.8.0_222]
at java.net.URLClassLoader$1.run(URLClassLoader.java:369) ~[?:1.8.0_222]
at java.net.URLClassLoader$1.run(URLClassLoader.java:363) ~[?:1.8.0_222]
at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_222]
at java.net.URLClassLoader.findClass(URLClassLoader.java:362) ~[?:1.8.0_222]
at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_222]
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[?:1.8.0_222]
at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_222]
at com.google.common.cache.LocalCache$LoadingValueReference.<init>(LocalCache.java:3472) ~[guava-30.1-jre.jar:?]
at com.google.common.cache.LocalCache$LoadingValueReference.<init>(LocalCache.java:3476) ~[guava-30.1-jre.jar:?]
at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2134) ~[guava-30.1-jre.jar:?]
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045) ~[guava-30.1-jre.jar:?]
at com.google.common.cache.LocalCache.get(LocalCache.java:3951) ~[guava-30.1-jre.jar:?]
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974) ~[guava-30.1-jre.jar:?]
at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4935) ~[guava-30.1-jre.jar:?]
at org.apache.kyuubi.session.FileSessionConfAdvisor.getConfOverlay(FileSessionConfAdvisor.scala:41) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.kyuubi.session.KyuubiSessionImpl.<init>(KyuubiSessionImpl.scala:55) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.kyuubi.session.KyuubiSessionManager.createSession(KyuubiSessionManager.scala:83) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.kyuubi.session.SessionManager.openSession(SessionManager.scala:98) ~[kyuubi-common_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.kyuubi.session.KyuubiSessionManager.openSession(KyuubiSessionManager.scala:95) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
... 18 more
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3813 from ychris78/yuqi_1116.
Closes#3813
7fbfdff8 [yuqi] update dev/dependencyList and LICENSE-binary
f664907d [yuqi] fix bug: Caused by: java.lang.ClassNotFoundException: com.google.common.util.concurrent.internal.InternalFutureFailureAccess
Authored-by: yuqi <1450636878@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
Bumps [protobuf-java](https://github.com/protocolbuffers/protobuf) from 3.21.1 to 3.21.7.
<details>
<summary>Commits</summary>
<ul>
<li><a href="54489e95e0"><code>54489e9</code></a> Updating changelog</li>
<li><a href="5fc03e1647"><code>5fc03e1</code></a> Updating version.json and repo version numbers to: 21.7</li>
<li><a href="a3888f5331"><code>a3888f5</code></a> Clean up TextFormat parser (<a href="https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/10673">#10673</a>)</li>
<li><a href="3b5301c114"><code>3b5301c</code></a> Refactoring Java parsing (21.x) (<a href="https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/10665">#10665</a>)</li>
<li><a href="bea6726c73"><code>bea6726</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/10579">#10579</a> from protocolbuffers/21.x-202209142140</li>
<li><a href="b1924e1de9"><code>b1924e1</code></a> Update version.json to: 21.7-dev</li>
<li><a href="929e13d1a9"><code>929e13d</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/10572">#10572</a> from deannagarcia/21.x</li>
<li><a href="de7597e577"><code>de7597e</code></a> Update python/release.sh to handle delay between twine upload and pip install...</li>
<li><a href="480bd3b970"><code>480bd3b</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/10557">#10557</a> from protocolbuffers/21.x-202209132118</li>
<li><a href="aa8c73d845"><code>aa8c73d</code></a> Updating changelog</li>
<li>Additional commits viewable in <a href="https://github.com/protocolbuffers/protobuf/compare/v3.21.1...v3.21.7">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `dependabot rebase` will rebase this PR
- `dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `dependabot merge` will merge this PR after your CI passes on it
- `dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `dependabot cancel merge` will cancel a previously requested merge and block automerging
- `dependabot reopen` will reopen this PR if it is closed
- `dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
- `dependabot use these labels` will set the current labels as the default for future PRs for this repo and language
- `dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language
- `dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language
- `dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language
You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/apache/incubator-kyuubi/network/alerts).
</details>
Closes#3798 from dependabot[bot]/dependabot/maven/com.google.protobuf-protobuf-java-3.21.7.
Closes#3798
ab92cde0 [Cheng Pan] deplist
5f535b97 [dependabot[bot]] Bump protobuf-java from 3.21.1 to 3.21.7
Lead-authored-by: Cheng Pan <chengpan@apache.org>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Bump Netty to latest version.
https://github.com/netty/netty/pull/12146 explained that netty-tcnative-classes should be optional, otherwise will cause https://github.com/netty/netty/issues/12101
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3796 from pan3793/netty.
Closes#3796
2b299233 [Cheng Pan] license
447a8d08 [Cheng Pan] nit
62abe481 [Cheng Pan] nit
506f76bf [Cheng Pan] Bump Netty from 4.1.73.Final to 4.1.84.Final
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
This pr aims to reduce the source file size.
This PR removes the swagger UI resource file in the folder `kyuubi-server/src/main/resources/org/apache/kyuubi/ui/swagger` which was introduced by [PR-2061](https://github.com/apache/incubator-kyuubi/pull/2061) , and now we rely on Maven dependency `org.webjars::swagger-ui::4.9.1` to package these resource files, we also add a new REST endpoint `/swagger-static/` for downloading these resource files
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3710 from cfmcgrady/reduce-src-size.
Closes#3710
c83bd313 [Fu Chen] update comment
441129b5 [Fu Chen] update LICENSE-binary
14eac59b [Fu Chen] update dev/dependencyList
7971c388 [Fu Chen] bump swagger-ui to 4.9.1
455aa206 [Fu Chen] reduce source file size
Authored-by: Fu Chen <cfmcgrady@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
to close#3602 .
Bump log4j from 2.18.0 to 2.19.0.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3603 from bowenliang123/3602-log4j2-218.
Closes#3602
fef6865c [Bowen Liang] Bump log4j from 2.18.0 to 2.19.0
Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Add `white-list` help run the specified queries.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
`--white-list q4,q15,q19,q25,q42,q49,q60,q66,q68,q72,q76,q78,q79,q89,q97`
Closes#3539 from zwangsheng/feature/tpcds_white_list.
Closes#3539
74657957 [zwangsheng] fix
26ade916 [Binjie Yang] Update dev/kyuubi-tpcds/src/main/scala/org/apache/kyuubi/tpcds/benchmark/RunBenchmark.scala
11a5b5c9 [Binjie Yang] Update dev/kyuubi-tpcds/src/main/scala/org/apache/kyuubi/tpcds/benchmark/RunBenchmark.scala
780410a3 [zwangsheng] fix
19018932 [zwangsheng] naming
1a4d1a25 [zwangsheng] naming
e1bb0069 [zwangsheng] fix
1be2bcb9 [zwangsheng] white
Lead-authored-by: zwangsheng <2213335496@qq.com>
Co-authored-by: Binjie Yang <52876270+zwangsheng@users.noreply.github.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3523 from zwangsheng/bugfix/bechmark_fix_result_dir.
Closes#3522
4f957960 [zwangsheng] rebase
17de184f [zwangsheng] fix
39361cfc [zwangsheng] fix
Authored-by: zwangsheng <2213335496@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
`r` or `results-dir` should point to results-dir
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3522 from zwangsheng/bugfix/bechmark_fix_result_dir.
Closes#3522
39361cfc [zwangsheng] fix
Authored-by: zwangsheng <2213335496@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Close https://github.com/apache/incubator-kyuubi/issues/3443, This pr aims to bump scala version from 2.12.15 to 2.12.16
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3467 from Yikf/bump-scala.
Closes#3443
63b6b381 [yikf] Bump Scala from 2.12.15 to 2.12.16
Authored-by: yikf <yikaifei1@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
close#3460
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3466 from QianyongY/features/kyuubi-3460.
Closes#3460
d3509a14 [yongqian] fix failed
52020fa2 [yongqian] [KYUUBI apache#3460] Bump Jackson from 2.13.3 to 2.13.4
Authored-by: yongqian <yongqian@trip.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
#2040 adds the test to verify the compatibility for Zookeeper Server 3.4, I will rebase this PR after #2040 get merged.
Close#1941
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2041 from pan3793/zk.
Closes#2041
d6722f11 [Cheng Pan] license
eb7101a0 [Cheng Pan] fix
d9ce7c29 [Cheng Pan] Bump Zookeeper 3.6.3
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Fix https://github.com/apache/incubator-kyuubi/issues/3378
This pr aims to improve hive-connector module tests and make CI perform tests for the hive connector
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3379 from Yikf/hive-connector-test.
Closes#3378
72ad050c [yikf] CI test for hive-connector
Authored-by: yikf <yikaifei1@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
This pr is a follow-up, aims to fix CI failure due to https://github.com/apache/incubator-kyuubi/pull/3299#discussion_r953265471
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3319 from Yikf/master.
Closes#3298
d0cba33f [yikf] [KYUUBI #3298][FOLLOWUP] Fix CI failure
Authored-by: yikf <yikaifei1@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
fix https://github.com/apache/incubator-kyuubi/issues/3298
Curently, a serval snippets use `getClass.getClassLoader` to get the classLoader in Apache Kyuubi, this pr aims to unify this behavior in `Utils.getContextOrKyuubiClassLoader`
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3299 from yikf/unify-calssloader.
Closes#3298
ae081ad9 [yikf] Unify the approach of get classLoader in Kyuubi
Authored-by: yikf <yikaifei1@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Preparing v1.7.0-SNAPSHOT with branch-1.6 cut
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3264 from SteNicholas/prepare-1.7.0-snapshot.
Closes#3264
374d56bf [SteNicholas] preparing v1.7.0-SNAPSHOT with branch-1.6 cut
Authored-by: SteNicholas <programgeek@163.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
This PR proposes to upgrade the latest version of jetcd. As https://github.com/etcd-io/jetcd/pull/1074 has been included, https://github.com/apache/incubator-kyuubi/pull/3148 is not required then.
This PR also pin Netty dependecies' version.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3189 from pan3793/etcd.
Closes#3189
1b2317af [Cheng Pan] [BUILD] Bump jetcd 0.7.3 and pin Netty dependencies
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: hongdongdong <hongdongdong@cmss.chinamobile.com>
### _Why are the changes needed?_
Bumping log4j from 2.17.2 to 2.18.0 will reduce the risk of CVEs.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request (Ran ```./build/mvn clean test``` - see screenshots below for more info)
Closes#3187 from ParisaTork/bump-log4j.
Closes#3145
c9f9e4a8 [ParisaTork] Update dependency list
c0d88f39 [ParisaTork] Bump log4j from 2.17.2 to 2.18.0
Authored-by: ParisaTork <47482049+ParisaTork@users.noreply.github.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Bumping gRPC from 1.47.0 to 1.48.0 will reduce the risk of CVEs.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3188 from ParisaTork/bump-gRPC.
Closes#3135
491afa5d [ParisaTork] Bump gRPC from 1.47.0 to 1.48.0
Authored-by: ParisaTork <47482049+ParisaTork@users.noreply.github.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
upgrade prometheus from 0.14.1 to 0.16.0
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
This patch had conflicts when merged, resolved by
Committer: Cheng Pan <chengpan@apache.org>
Closes#3097 from xinsun6/kyuubi_3008.
Closes#3097
97092c86 [xsun4] [KYUUBI apache#3008]Bump prometheus from 0.14.1 to 0.16.0
Authored-by: xsun4 <xsun4@ebay.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Add etcd discovery client for HA
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2767 from hddong/etcd-support.
Closes#2644
6fa37578 [hongdongdong] Rebase and remove from rat-excludes
fc23a2b4 [hongdongdong] upgrade grpc-1.47.0
61c77d27 [hongdongdong] Fix tests
28abd13e [hongdongdong] fix comments
9e81a6a3 [hongdongdong] Rename HA_ZK_ENGINE_REF_ID to HA_ENGINE_REF_ID
aa2b4260 [hongdongdong] [KYUUBI #2644][WIP] Add etcd discovery client for HA
Authored-by: hongdongdong <hongdongdong@cmss.chinamobile.com>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
Handles configuring the JUL -> SLF4J bridge, otherwise, the components which use JUL like Jersey will be not controlled by log4j2 configurations
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3037 from pan3793/jul.
Closes#3037
fda05ff6 [Cheng Pan] update license
31f579fd [Cheng Pan] nit
7da0f9fb [Cheng Pan] Fix dependencyList
3179e6bd [Cheng Pan] Handles configuring the JUL -> SLF4J bridge
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
close#3007
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3015 from lsm1/features/Bump_scopt.
Closes#3007
876659ba [senmiaoliu] fix style
e1a42033 [LSM] Merge pull request #1 from cxzl25/PR_3015_UT
9a34eedb [sychen] fix UT
46e1dff9 [senmiaoliu] uodate dependencyList
8481b14b [senmiaoliu] Bump scopt from 4.0.1 to 4.1.0
Lead-authored-by: senmiaoliu <senmiaoliu@trip.com>
Co-authored-by: sychen <sychen@ctrip.com>
Co-authored-by: LSM <senmiaoliu@trip.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
### _Why are the changes needed?_
close#3010
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
This patch had conflicts when merged, resolved by
Committer: Cheng Pan <chengpan@apache.org>
Closes#3014 from lsm1/features/bump_Jetty.
Closes#3010
ae18e0f6 [senmiaoliu] update dependencyList
b32cd8c3 [senmiaoliu] Bump Jetty from 9.4.41.v20210516 to 9.4.48.v20220622
Authored-by: senmiaoliu <senmiaoliu@trip.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
close#3011
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3016 from lsm1/features/bump_swagger.
Closes#3011
df0a18e4 [senmiaoliu] update dependencyList
ac07032a [senmiaoliu] Bump swagger from 2.1.11 to 2.2.1
6d8a1b8e [senmiaoliu] Bump swagger from 2.1.11 to 2.2.1
Authored-by: senmiaoliu <senmiaoliu@trip.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
fix#3009
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3013 from jiaoqingbo/kyuubi-3009.
Closes#3009
6f47ec6d [jiaoqingbo] [KYUUBI #3009] Bump Jersey from 2.35 to 2.36
Authored-by: jiaoqingbo <1178404354@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
https://github.com/apache/incubator-kyuubi/issues/2904
This reverts commit e250c4ae35.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2915 from pan3793/revert.
Closes#2915Closes#2000
9c789eb1 [Cheng Pan] Revert "[KYUUBI #2000][DEPS] Bump Hadoop 3.3.2"
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
This PR moves engine integration tests except to spark engine from `kyuubi-server` module to dedicated IT modules e.g. `kyuubi-hive-it`, then we can spilt different engine test into jobs to speed up CI tes
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
before
<img width="836" alt="image" src="https://user-images.githubusercontent.com/26535726/173265839-5d0f1ce6-e133-4d60-9a95-c95aa16f817f.png">
after
<img width="825" alt="image" src="https://user-images.githubusercontent.com/26535726/173265903-fb01e910-8f78-4091-91da-dca25be4357d.png">
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2827 from pan3793/hive-it.
Closes#2827
df3798f1 [Cheng Pan] Remove Java 11 test for Hive
e1645338 [Cheng Pan] nit
66524226 [Cheng Pan] nit
3aef2cef [Cheng Pan] review
500c9cff [Cheng Pan] MVN_OPT
3b4ad142 [Cheng Pan] nit
daa6e0ab [Cheng Pan] review
3383d4c1 [Cheng Pan] [BUILD][TEST] Decouple integration tests from kyuubi-server
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Fix#2879
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2883 from zhouyifan279/2879.
Closes#2883
bbd6c2e7 [zhouyifan279] [Bug] java.lang.NoClassDefFoundError: org/apache/hadoop/hive/common/ValidWriteIdList in HiveDelegationTokenProvider#initialize
d5046f36 [zhouyifan279] [Bug] java.lang.NoClassDefFoundError: org/apache/hadoop/hive/common/ValidWriteIdList in HiveDelegationTokenProvider#initialize
Authored-by: zhouyifan279 <zhouyifan279@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
1. reformat code should include all of the modules in the project.
2. stylecheck should include all of the modules in the project.
3. there is a bug in the stage `Upload style check report`.
Since the `grep` returns code 1 when the pattern is not found, the upload report does not work before this pr.
```
> echo "a" > /tmp/a; grep abc /tmp/a; echo $?
1
```
Before this PR
```
Run for log in `find * -name "scalastyle-output.xml"`; do echo "=========$log========="; grep "error" $log; done
=========dev/kyuubi-tpcds/target/scalastyle-output.xml=========
Error: Process completed with exit code 1.
```
After this PR
```
Run for log in `find * -name "scalastyle-output.xml"`;
<?xml version="1.0" encoding="UTF-8"?>
<checkstyle version="5.0">
<file name="/home/runner/work/incubator-kyuubi/incubator-kyuubi/extensions/spark/kyuubi-extension-spark-3-3/src/main/scala/org/apache/kyuubi/sql/KyuubiSparkSQLExtension.scala">
<error column="0" line="21" source="org.scalastyle.scalariform.ImportOrderChecker" severity="error" message="There should at least one a single empty line separating groups 3rdParty and kyuubi."/>
</file>
</checkstyle>
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2780 from cfmcgrady/refine-stylecheck.
Closes#2780
a9c4a71e [Fu Chen] update reformat
f5d4d4e4 [Fu Chen] address comment
0665e7e5 [Fu Chen] address
91d123ee [Fu Chen] fix
f5d6414f [Fu Chen] style.yml
c56fa081 [Fu Chen] dev/reformat
Authored-by: Fu Chen <cfmcgrady@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
Close#2715Close#2374
Introduce jdbc based session state store to share the batch state information across kyuubi instances.
This pr is based on docs: https://docs.google.com/document/d/1_BMekMT88P2vCHCl4ZacWEeWslpIv4pxsJH32PAnWSI and refactor it during implementation.
In this PR, I introduce `StateStore` interface and implement the default `JDBCStateStore`.
The data structures:
1. BatchState - the batch state info for multiple-HA, including batch operation state and application information
2. BatchMeta - the meta data used for batch recovery, including sessionConf, batchRequest information
For JDBCStateStore:
- support to specify data base type:
- 1. DERBY - it will use derby jdbc driver by default
- 2. MYSQL - it will use mysql jdbc by default
- 3. CUSTOM - the user need specify the jdbc driver
- support to init schema
By default, the jdbc url is a derby in-memory database url for single kyuubi instance.
To support multiple HA in production env, we should config the production jdbc url to share batch info across kyuubi instances.
The datasource is managed by HiKariCP.
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2717 from turboFei/kyuubi_2715_hikaricp_v2.
Closes#2717
367b944a3 [Fei Wang] comments
9a7963dbd [Fei Wang] post session type in event
b1c130853 [Fei Wang] move session type to session package
66def0cb6 [Fei Wang] add JDBCStateStoreConf
839b72bf6 [Fei Wang] refactor
644cbcd28 [Fei Wang] filter by session type
58f1c7854 [Fei Wang] refactor
1c0035208 [Fei Wang] add session type
34a087ee5 [Fei Wang] refactor
fba38e473 [Fei Wang] refactor
fcbde25a0 [Fei Wang] refactor
332a67537 [Fei Wang] comments
135ba9b6d [Fei Wang] Revert "revert class utils change"
0e045a483 [Fei Wang] Revert "Revert "move""
05048416e [Fei Wang] Revert "test again"
01a50c35b [Fei Wang] test again
b89159d13 [Fei Wang] Revert "move"
3764a9051 [Fei Wang] move
eef17822c [Fei Wang] revert
4166db261 [Fei Wang] adjust engine idle timeout
5fc95a824 [Fei Wang] revert class utils change
23f88d63d [Fei Wang] refactor
a66cb811d [Fei Wang] refactor
89fa0edb4 [Fei Wang] fix
e5228fefd [Fei Wang] refactor
581908d86 [Fei Wang] exclude sql
3446aaa06 [Fei Wang] UPPER
ab980c677 [Fei Wang] [KYUUBI #2715] Introduce session state store for batch session multiple HA
Authored-by: Fei Wang <fwang12@ebay.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
### _Why are the changes needed?_
To close#2643
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2723 from lightning-L/kyuubi-2643.
Closes#2643
b7c409bcf [Tianlin Liao] unify junit dependency
ae2caca33 [Tianlin Liao] implements AutoClosable for KyuubiRestClient; rename server in rest builder to spnego builder
19e078a32 [Tianlin Liao] reuse ObjectMapper
ef98f4c2a [Tianlin Liao] minor improvements
15d958acf [Tianlin Liao] unify httpclient version parameter
37032a6a9 [Tianlin Liao] fix dependency
6c97745aa [Tianlin Liao] support NONE and NOSASL basic auth type
aa0a1c011 [Tianlin Liao] add optional tag for hadoop-client-api/hadoop-client-runtime
275d1d042 [Tianlin Liao] add integration test in kyuubi-server
b407d0229 [Tianlin Liao] apiBasePath is not necessary for the client builder
db11f10eb [Tianlin Liao] add test case for http response code is not 200
361343fe0 [Tianlin Liao] add builder arguments validation
a2684de86 [Tianlin Liao] remove gson dependency; fix get log api typo
ed3498938 [Tianlin Liao] Define KyuubiRestException and remove kyuubi-common dependency
b46f0c4d3 [Tianlin Liao] add AuthScheme config in Builder
61796d782 [Tianlin Liao] fix sun.security does not exist/not visible error
de9bb26e2 [Tianlin Liao] [KYUUBI #2643] [SUB-TASK][KPIP-4] Implement kyuubi rest sdk for batch job operation
Authored-by: Tianlin Liao <tiliao@ebay.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
