### _Why are the changes needed?_
Introduce code style check support for Maven's pom.xml with sortPom in spotless maven plugin.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3843 from bowenliang123/spotless-pom.
Closes#3842
3c654597 [liangbowen] apply to pom.xml
fd1536f7 [liangbowen] set expandEmptyElements to true
e498423f [liangbowen] apply spotless:apply to all pom.xml
e46bcfec [liangbowen] add pom style check support in spotless
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3834 from leoluan2009/slf4j.
Closes#3834
e17b08ef [Xuedong Luan] Upgrade SLF4J from 1.7.35 to 1.7.36
Authored-by: Xuedong Luan <luanxuedong2009@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
to close#3828.
Python code style checking support.
1. reuse Spotless maven plugin for Python style check
2. add style check to CI style workflow
3. add python linting support to `dev/reformat`. checks whether `black` installed in PATH.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3823 from bowenliang123/spotless-python.
Closes#3828
4a4de885 [liangbowen] simplify empty tags
0bb9ec7c [liangbowen] simplify empty tag in pom
9dd39531 [liangbowen] lint python code with black via spotless
f85020fa [liangbowen] typo
4c93bce0 [liangbowen] install python 3.9 first
23fc4b96 [liangbowen] ci install black version from added `spotless.python.black.version` property
73f746b0 [Bowen Liang] Update dev/reformat
46667a00 [liangbowen] update style.yml
9c20b434 [liangbowen] update style.yml
21017e5e [liangbowen] update style.yml
8272c0bc [liangbowen] add python style to style checking for CI
e102726c [liangbowen] add profile spotless in dev/reformat if black found in path
062e9bf2 [liangbowen] add python scan for spotless. add new profile `spotless-python` for python file path.
Lead-authored-by: liangbowen <liangbowen@gf.com.cn>
Co-authored-by: Bowen Liang <bowen.liang.123@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
**Fix the following bug:**
```
2022-11-16 13:08:45.540 ERROR org.apache.kyuubi.server.KyuubiTBinaryFrontendService: Error opening session:
org.apache.kyuubi.KyuubiSQLException: Error opening session for test_user client ip 127.0.0.1, due to com/google/common/util/concurrent/internal/InternalFutureFailureAccess
at org.apache.kyuubi.KyuubiSQLException$.apply(KyuubiSQLException.scala:69) ~[kyuubi-common_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.kyuubi.session.KyuubiSessionManager.openSession(KyuubiSessionManager.scala:102) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.kyuubi.service.AbstractBackendService.openSession(AbstractBackendService.scala:45) ~[kyuubi-common_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.kyuubi.server.KyuubiServer$$anon$1.org$apache$kyuubi$server$BackendServiceMetric$$super$openSession(KyuubiServer.scala:111) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.kyuubi.server.BackendServiceMetric.$anonfun$openSession$1(BackendServiceMetric.scala:37) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.kyuubi.metrics.MetricsSystem$.timerTracing(MetricsSystem.scala:111) ~[kyuubi-metrics_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.kyuubi.server.BackendServiceMetric.openSession(BackendServiceMetric.scala:37) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.kyuubi.server.BackendServiceMetric.openSession$(BackendServiceMetric.scala:30) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.kyuubi.server.KyuubiServer$$anon$1.openSession(KyuubiServer.scala:111) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.kyuubi.service.TFrontendService.getSessionHandle(TFrontendService.scala:181) ~[kyuubi-common_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.kyuubi.server.KyuubiTBinaryFrontendService.OpenSession(KyuubiTBinaryFrontendService.scala:81) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1497) ~[hive-service-rpc-3.1.3.jar:3.1.3]
at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1482) ~[hive-service-rpc-3.1.3.jar:3.1.3]
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) ~[libthrift-0.9.3.jar:0.9.3]
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) ~[libthrift-0.9.3.jar:0.9.3]
at org.apache.kyuubi.service.authentication.TSetIpAddressProcessor.process(TSetIpAddressProcessor.scala:36) ~[kyuubi-common_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) ~[libthrift-0.9.3.jar:0.9.3]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_222]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_222]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_222]
Caused by: java.lang.NoClassDefFoundError: com/google/common/util/concurrent/internal/InternalFutureFailureAccess
at java.lang.ClassLoader.defineClass1(Native Method) ~[?:1.8.0_222]
at java.lang.ClassLoader.defineClass(ClassLoader.java:763) ~[?:1.8.0_222]
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) ~[?:1.8.0_222]
at java.net.URLClassLoader.defineClass(URLClassLoader.java:468) ~[?:1.8.0_222]
at java.net.URLClassLoader.access$100(URLClassLoader.java:74) ~[?:1.8.0_222]
at java.net.URLClassLoader$1.run(URLClassLoader.java:369) ~[?:1.8.0_222]
at java.net.URLClassLoader$1.run(URLClassLoader.java:363) ~[?:1.8.0_222]
at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_222]
at java.net.URLClassLoader.findClass(URLClassLoader.java:362) ~[?:1.8.0_222]
at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_222]
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[?:1.8.0_222]
at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_222]
at java.lang.ClassLoader.defineClass1(Native Method) ~[?:1.8.0_222]
at java.lang.ClassLoader.defineClass(ClassLoader.java:763) ~[?:1.8.0_222]
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) ~[?:1.8.0_222]
at java.net.URLClassLoader.defineClass(URLClassLoader.java:468) ~[?:1.8.0_222]
at java.net.URLClassLoader.access$100(URLClassLoader.java:74) ~[?:1.8.0_222]
at java.net.URLClassLoader$1.run(URLClassLoader.java:369) ~[?:1.8.0_222]
at java.net.URLClassLoader$1.run(URLClassLoader.java:363) ~[?:1.8.0_222]
at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_222]
at java.net.URLClassLoader.findClass(URLClassLoader.java:362) ~[?:1.8.0_222]
at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_222]
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[?:1.8.0_222]
at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_222]
at java.lang.ClassLoader.defineClass1(Native Method) ~[?:1.8.0_222]
at java.lang.ClassLoader.defineClass(ClassLoader.java:763) ~[?:1.8.0_222]
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) ~[?:1.8.0_222]
at java.net.URLClassLoader.defineClass(URLClassLoader.java:468) ~[?:1.8.0_222]
at java.net.URLClassLoader.access$100(URLClassLoader.java:74) ~[?:1.8.0_222]
at java.net.URLClassLoader$1.run(URLClassLoader.java:369) ~[?:1.8.0_222]
at java.net.URLClassLoader$1.run(URLClassLoader.java:363) ~[?:1.8.0_222]
at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_222]
at java.net.URLClassLoader.findClass(URLClassLoader.java:362) ~[?:1.8.0_222]
at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_222]
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[?:1.8.0_222]
at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_222]
at com.google.common.cache.LocalCache$LoadingValueReference.<init>(LocalCache.java:3472) ~[guava-30.1-jre.jar:?]
at com.google.common.cache.LocalCache$LoadingValueReference.<init>(LocalCache.java:3476) ~[guava-30.1-jre.jar:?]
at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2134) ~[guava-30.1-jre.jar:?]
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045) ~[guava-30.1-jre.jar:?]
at com.google.common.cache.LocalCache.get(LocalCache.java:3951) ~[guava-30.1-jre.jar:?]
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974) ~[guava-30.1-jre.jar:?]
at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4935) ~[guava-30.1-jre.jar:?]
at org.apache.kyuubi.session.FileSessionConfAdvisor.getConfOverlay(FileSessionConfAdvisor.scala:41) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.kyuubi.session.KyuubiSessionImpl.<init>(KyuubiSessionImpl.scala:55) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.kyuubi.session.KyuubiSessionManager.createSession(KyuubiSessionManager.scala:83) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.kyuubi.session.SessionManager.openSession(SessionManager.scala:98) ~[kyuubi-common_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.kyuubi.session.KyuubiSessionManager.openSession(KyuubiSessionManager.scala:95) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
... 18 more
Caused by: java.lang.ClassNotFoundException: com.google.common.util.concurrent.internal.InternalFutureFailureAccess
at java.net.URLClassLoader.findClass(URLClassLoader.java:382) ~[?:1.8.0_222]
at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_222]
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[?:1.8.0_222]
at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_222]
at java.lang.ClassLoader.defineClass1(Native Method) ~[?:1.8.0_222]
at java.lang.ClassLoader.defineClass(ClassLoader.java:763) ~[?:1.8.0_222]
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) ~[?:1.8.0_222]
at java.net.URLClassLoader.defineClass(URLClassLoader.java:468) ~[?:1.8.0_222]
at java.net.URLClassLoader.access$100(URLClassLoader.java:74) ~[?:1.8.0_222]
at java.net.URLClassLoader$1.run(URLClassLoader.java:369) ~[?:1.8.0_222]
at java.net.URLClassLoader$1.run(URLClassLoader.java:363) ~[?:1.8.0_222]
at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_222]
at java.net.URLClassLoader.findClass(URLClassLoader.java:362) ~[?:1.8.0_222]
at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_222]
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[?:1.8.0_222]
at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_222]
at java.lang.ClassLoader.defineClass1(Native Method) ~[?:1.8.0_222]
at java.lang.ClassLoader.defineClass(ClassLoader.java:763) ~[?:1.8.0_222]
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) ~[?:1.8.0_222]
at java.net.URLClassLoader.defineClass(URLClassLoader.java:468) ~[?:1.8.0_222]
at java.net.URLClassLoader.access$100(URLClassLoader.java:74) ~[?:1.8.0_222]
at java.net.URLClassLoader$1.run(URLClassLoader.java:369) ~[?:1.8.0_222]
at java.net.URLClassLoader$1.run(URLClassLoader.java:363) ~[?:1.8.0_222]
at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_222]
at java.net.URLClassLoader.findClass(URLClassLoader.java:362) ~[?:1.8.0_222]
at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_222]
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[?:1.8.0_222]
at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_222]
at java.lang.ClassLoader.defineClass1(Native Method) ~[?:1.8.0_222]
at java.lang.ClassLoader.defineClass(ClassLoader.java:763) ~[?:1.8.0_222]
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) ~[?:1.8.0_222]
at java.net.URLClassLoader.defineClass(URLClassLoader.java:468) ~[?:1.8.0_222]
at java.net.URLClassLoader.access$100(URLClassLoader.java:74) ~[?:1.8.0_222]
at java.net.URLClassLoader$1.run(URLClassLoader.java:369) ~[?:1.8.0_222]
at java.net.URLClassLoader$1.run(URLClassLoader.java:363) ~[?:1.8.0_222]
at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_222]
at java.net.URLClassLoader.findClass(URLClassLoader.java:362) ~[?:1.8.0_222]
at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_222]
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[?:1.8.0_222]
at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_222]
at com.google.common.cache.LocalCache$LoadingValueReference.<init>(LocalCache.java:3472) ~[guava-30.1-jre.jar:?]
at com.google.common.cache.LocalCache$LoadingValueReference.<init>(LocalCache.java:3476) ~[guava-30.1-jre.jar:?]
at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2134) ~[guava-30.1-jre.jar:?]
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045) ~[guava-30.1-jre.jar:?]
at com.google.common.cache.LocalCache.get(LocalCache.java:3951) ~[guava-30.1-jre.jar:?]
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974) ~[guava-30.1-jre.jar:?]
at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4935) ~[guava-30.1-jre.jar:?]
at org.apache.kyuubi.session.FileSessionConfAdvisor.getConfOverlay(FileSessionConfAdvisor.scala:41) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.kyuubi.session.KyuubiSessionImpl.<init>(KyuubiSessionImpl.scala:55) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.kyuubi.session.KyuubiSessionManager.createSession(KyuubiSessionManager.scala:83) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.kyuubi.session.SessionManager.openSession(SessionManager.scala:98) ~[kyuubi-common_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.kyuubi.session.KyuubiSessionManager.openSession(KyuubiSessionManager.scala:95) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
... 18 more
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3813 from ychris78/yuqi_1116.
Closes#3813
7fbfdff8 [yuqi] update dev/dependencyList and LICENSE-binary
f664907d [yuqi] fix bug: Caused by: java.lang.ClassNotFoundException: com.google.common.util.concurrent.internal.InternalFutureFailureAccess
Authored-by: yuqi <1450636878@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
Bumps [protobuf-java](https://github.com/protocolbuffers/protobuf) from 3.21.1 to 3.21.7.
<details>
<summary>Commits</summary>
<ul>
<li><a href="54489e95e0"><code>54489e9</code></a> Updating changelog</li>
<li><a href="5fc03e1647"><code>5fc03e1</code></a> Updating version.json and repo version numbers to: 21.7</li>
<li><a href="a3888f5331"><code>a3888f5</code></a> Clean up TextFormat parser (<a href="https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/10673">#10673</a>)</li>
<li><a href="3b5301c114"><code>3b5301c</code></a> Refactoring Java parsing (21.x) (<a href="https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/10665">#10665</a>)</li>
<li><a href="bea6726c73"><code>bea6726</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/10579">#10579</a> from protocolbuffers/21.x-202209142140</li>
<li><a href="b1924e1de9"><code>b1924e1</code></a> Update version.json to: 21.7-dev</li>
<li><a href="929e13d1a9"><code>929e13d</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/10572">#10572</a> from deannagarcia/21.x</li>
<li><a href="de7597e577"><code>de7597e</code></a> Update python/release.sh to handle delay between twine upload and pip install...</li>
<li><a href="480bd3b970"><code>480bd3b</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/10557">#10557</a> from protocolbuffers/21.x-202209132118</li>
<li><a href="aa8c73d845"><code>aa8c73d</code></a> Updating changelog</li>
<li>Additional commits viewable in <a href="https://github.com/protocolbuffers/protobuf/compare/v3.21.1...v3.21.7">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `dependabot rebase` will rebase this PR
- `dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `dependabot merge` will merge this PR after your CI passes on it
- `dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `dependabot cancel merge` will cancel a previously requested merge and block automerging
- `dependabot reopen` will reopen this PR if it is closed
- `dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
- `dependabot use these labels` will set the current labels as the default for future PRs for this repo and language
- `dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language
- `dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language
- `dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language
You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/apache/incubator-kyuubi/network/alerts).
</details>
Closes#3798 from dependabot[bot]/dependabot/maven/com.google.protobuf-protobuf-java-3.21.7.
Closes#3798
ab92cde0 [Cheng Pan] deplist
5f535b97 [dependabot[bot]] Bump protobuf-java from 3.21.1 to 3.21.7
Lead-authored-by: Cheng Pan <chengpan@apache.org>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Bump Netty to latest version.
https://github.com/netty/netty/pull/12146 explained that netty-tcnative-classes should be optional, otherwise will cause https://github.com/netty/netty/issues/12101
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3796 from pan3793/netty.
Closes#3796
2b299233 [Cheng Pan] license
447a8d08 [Cheng Pan] nit
62abe481 [Cheng Pan] nit
506f76bf [Cheng Pan] Bump Netty from 4.1.73.Final to 4.1.84.Final
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
This pr aims to reduce the source file size.
This PR removes the swagger UI resource file in the folder `kyuubi-server/src/main/resources/org/apache/kyuubi/ui/swagger` which was introduced by [PR-2061](https://github.com/apache/incubator-kyuubi/pull/2061) , and now we rely on Maven dependency `org.webjars::swagger-ui::4.9.1` to package these resource files, we also add a new REST endpoint `/swagger-static/` for downloading these resource files
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3710 from cfmcgrady/reduce-src-size.
Closes#3710
c83bd313 [Fu Chen] update comment
441129b5 [Fu Chen] update LICENSE-binary
14eac59b [Fu Chen] update dev/dependencyList
7971c388 [Fu Chen] bump swagger-ui to 4.9.1
455aa206 [Fu Chen] reduce source file size
Authored-by: Fu Chen <cfmcgrady@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
to close#3602 .
Bump log4j from 2.18.0 to 2.19.0.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3603 from bowenliang123/3602-log4j2-218.
Closes#3602
fef6865c [Bowen Liang] Bump log4j from 2.18.0 to 2.19.0
Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Add `white-list` help run the specified queries.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
`--white-list q4,q15,q19,q25,q42,q49,q60,q66,q68,q72,q76,q78,q79,q89,q97`
Closes#3539 from zwangsheng/feature/tpcds_white_list.
Closes#3539
74657957 [zwangsheng] fix
26ade916 [Binjie Yang] Update dev/kyuubi-tpcds/src/main/scala/org/apache/kyuubi/tpcds/benchmark/RunBenchmark.scala
11a5b5c9 [Binjie Yang] Update dev/kyuubi-tpcds/src/main/scala/org/apache/kyuubi/tpcds/benchmark/RunBenchmark.scala
780410a3 [zwangsheng] fix
19018932 [zwangsheng] naming
1a4d1a25 [zwangsheng] naming
e1bb0069 [zwangsheng] fix
1be2bcb9 [zwangsheng] white
Lead-authored-by: zwangsheng <2213335496@qq.com>
Co-authored-by: Binjie Yang <52876270+zwangsheng@users.noreply.github.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3523 from zwangsheng/bugfix/bechmark_fix_result_dir.
Closes#3522
4f957960 [zwangsheng] rebase
17de184f [zwangsheng] fix
39361cfc [zwangsheng] fix
Authored-by: zwangsheng <2213335496@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
`r` or `results-dir` should point to results-dir
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3522 from zwangsheng/bugfix/bechmark_fix_result_dir.
Closes#3522
39361cfc [zwangsheng] fix
Authored-by: zwangsheng <2213335496@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Close https://github.com/apache/incubator-kyuubi/issues/3443, This pr aims to bump scala version from 2.12.15 to 2.12.16
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3467 from Yikf/bump-scala.
Closes#3443
63b6b381 [yikf] Bump Scala from 2.12.15 to 2.12.16
Authored-by: yikf <yikaifei1@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
close#3460
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3466 from QianyongY/features/kyuubi-3460.
Closes#3460
d3509a14 [yongqian] fix failed
52020fa2 [yongqian] [KYUUBI apache#3460] Bump Jackson from 2.13.3 to 2.13.4
Authored-by: yongqian <yongqian@trip.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
#2040 adds the test to verify the compatibility for Zookeeper Server 3.4, I will rebase this PR after #2040 get merged.
Close#1941
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2041 from pan3793/zk.
Closes#2041
d6722f11 [Cheng Pan] license
eb7101a0 [Cheng Pan] fix
d9ce7c29 [Cheng Pan] Bump Zookeeper 3.6.3
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Fix https://github.com/apache/incubator-kyuubi/issues/3378
This pr aims to improve hive-connector module tests and make CI perform tests for the hive connector
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3379 from Yikf/hive-connector-test.
Closes#3378
72ad050c [yikf] CI test for hive-connector
Authored-by: yikf <yikaifei1@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
This pr is a follow-up, aims to fix CI failure due to https://github.com/apache/incubator-kyuubi/pull/3299#discussion_r953265471
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3319 from Yikf/master.
Closes#3298
d0cba33f [yikf] [KYUUBI #3298][FOLLOWUP] Fix CI failure
Authored-by: yikf <yikaifei1@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
fix https://github.com/apache/incubator-kyuubi/issues/3298
Curently, a serval snippets use `getClass.getClassLoader` to get the classLoader in Apache Kyuubi, this pr aims to unify this behavior in `Utils.getContextOrKyuubiClassLoader`
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3299 from yikf/unify-calssloader.
Closes#3298
ae081ad9 [yikf] Unify the approach of get classLoader in Kyuubi
Authored-by: yikf <yikaifei1@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Preparing v1.7.0-SNAPSHOT with branch-1.6 cut
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3264 from SteNicholas/prepare-1.7.0-snapshot.
Closes#3264
374d56bf [SteNicholas] preparing v1.7.0-SNAPSHOT with branch-1.6 cut
Authored-by: SteNicholas <programgeek@163.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
This PR proposes to upgrade the latest version of jetcd. As https://github.com/etcd-io/jetcd/pull/1074 has been included, https://github.com/apache/incubator-kyuubi/pull/3148 is not required then.
This PR also pin Netty dependecies' version.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3189 from pan3793/etcd.
Closes#3189
1b2317af [Cheng Pan] [BUILD] Bump jetcd 0.7.3 and pin Netty dependencies
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: hongdongdong <hongdongdong@cmss.chinamobile.com>
### _Why are the changes needed?_
Bumping log4j from 2.17.2 to 2.18.0 will reduce the risk of CVEs.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request (Ran ```./build/mvn clean test``` - see screenshots below for more info)
Closes#3187 from ParisaTork/bump-log4j.
Closes#3145
c9f9e4a8 [ParisaTork] Update dependency list
c0d88f39 [ParisaTork] Bump log4j from 2.17.2 to 2.18.0
Authored-by: ParisaTork <47482049+ParisaTork@users.noreply.github.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Bumping gRPC from 1.47.0 to 1.48.0 will reduce the risk of CVEs.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3188 from ParisaTork/bump-gRPC.
Closes#3135
491afa5d [ParisaTork] Bump gRPC from 1.47.0 to 1.48.0
Authored-by: ParisaTork <47482049+ParisaTork@users.noreply.github.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
upgrade prometheus from 0.14.1 to 0.16.0
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
This patch had conflicts when merged, resolved by
Committer: Cheng Pan <chengpan@apache.org>
Closes#3097 from xinsun6/kyuubi_3008.
Closes#3097
97092c86 [xsun4] [KYUUBI apache#3008]Bump prometheus from 0.14.1 to 0.16.0
Authored-by: xsun4 <xsun4@ebay.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Add etcd discovery client for HA
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2767 from hddong/etcd-support.
Closes#2644
6fa37578 [hongdongdong] Rebase and remove from rat-excludes
fc23a2b4 [hongdongdong] upgrade grpc-1.47.0
61c77d27 [hongdongdong] Fix tests
28abd13e [hongdongdong] fix comments
9e81a6a3 [hongdongdong] Rename HA_ZK_ENGINE_REF_ID to HA_ENGINE_REF_ID
aa2b4260 [hongdongdong] [KYUUBI #2644][WIP] Add etcd discovery client for HA
Authored-by: hongdongdong <hongdongdong@cmss.chinamobile.com>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
Handles configuring the JUL -> SLF4J bridge, otherwise, the components which use JUL like Jersey will be not controlled by log4j2 configurations
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3037 from pan3793/jul.
Closes#3037
fda05ff6 [Cheng Pan] update license
31f579fd [Cheng Pan] nit
7da0f9fb [Cheng Pan] Fix dependencyList
3179e6bd [Cheng Pan] Handles configuring the JUL -> SLF4J bridge
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
close#3007
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3015 from lsm1/features/Bump_scopt.
Closes#3007
876659ba [senmiaoliu] fix style
e1a42033 [LSM] Merge pull request #1 from cxzl25/PR_3015_UT
9a34eedb [sychen] fix UT
46e1dff9 [senmiaoliu] uodate dependencyList
8481b14b [senmiaoliu] Bump scopt from 4.0.1 to 4.1.0
Lead-authored-by: senmiaoliu <senmiaoliu@trip.com>
Co-authored-by: sychen <sychen@ctrip.com>
Co-authored-by: LSM <senmiaoliu@trip.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
### _Why are the changes needed?_
close#3010
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
This patch had conflicts when merged, resolved by
Committer: Cheng Pan <chengpan@apache.org>
Closes#3014 from lsm1/features/bump_Jetty.
Closes#3010
ae18e0f6 [senmiaoliu] update dependencyList
b32cd8c3 [senmiaoliu] Bump Jetty from 9.4.41.v20210516 to 9.4.48.v20220622
Authored-by: senmiaoliu <senmiaoliu@trip.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
close#3011
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3016 from lsm1/features/bump_swagger.
Closes#3011
df0a18e4 [senmiaoliu] update dependencyList
ac07032a [senmiaoliu] Bump swagger from 2.1.11 to 2.2.1
6d8a1b8e [senmiaoliu] Bump swagger from 2.1.11 to 2.2.1
Authored-by: senmiaoliu <senmiaoliu@trip.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
fix#3009
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3013 from jiaoqingbo/kyuubi-3009.
Closes#3009
6f47ec6d [jiaoqingbo] [KYUUBI #3009] Bump Jersey from 2.35 to 2.36
Authored-by: jiaoqingbo <1178404354@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
https://github.com/apache/incubator-kyuubi/issues/2904
This reverts commit e250c4ae35.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2915 from pan3793/revert.
Closes#2915Closes#2000
9c789eb1 [Cheng Pan] Revert "[KYUUBI #2000][DEPS] Bump Hadoop 3.3.2"
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
This PR moves engine integration tests except to spark engine from `kyuubi-server` module to dedicated IT modules e.g. `kyuubi-hive-it`, then we can spilt different engine test into jobs to speed up CI tes
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
before
<img width="836" alt="image" src="https://user-images.githubusercontent.com/26535726/173265839-5d0f1ce6-e133-4d60-9a95-c95aa16f817f.png">
after
<img width="825" alt="image" src="https://user-images.githubusercontent.com/26535726/173265903-fb01e910-8f78-4091-91da-dca25be4357d.png">
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2827 from pan3793/hive-it.
Closes#2827
df3798f1 [Cheng Pan] Remove Java 11 test for Hive
e1645338 [Cheng Pan] nit
66524226 [Cheng Pan] nit
3aef2cef [Cheng Pan] review
500c9cff [Cheng Pan] MVN_OPT
3b4ad142 [Cheng Pan] nit
daa6e0ab [Cheng Pan] review
3383d4c1 [Cheng Pan] [BUILD][TEST] Decouple integration tests from kyuubi-server
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Fix#2879
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2883 from zhouyifan279/2879.
Closes#2883
bbd6c2e7 [zhouyifan279] [Bug] java.lang.NoClassDefFoundError: org/apache/hadoop/hive/common/ValidWriteIdList in HiveDelegationTokenProvider#initialize
d5046f36 [zhouyifan279] [Bug] java.lang.NoClassDefFoundError: org/apache/hadoop/hive/common/ValidWriteIdList in HiveDelegationTokenProvider#initialize
Authored-by: zhouyifan279 <zhouyifan279@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
1. reformat code should include all of the modules in the project.
2. stylecheck should include all of the modules in the project.
3. there is a bug in the stage `Upload style check report`.
Since the `grep` returns code 1 when the pattern is not found, the upload report does not work before this pr.
```
> echo "a" > /tmp/a; grep abc /tmp/a; echo $?
1
```
Before this PR
```
Run for log in `find * -name "scalastyle-output.xml"`; do echo "=========$log========="; grep "error" $log; done
=========dev/kyuubi-tpcds/target/scalastyle-output.xml=========
Error: Process completed with exit code 1.
```
After this PR
```
Run for log in `find * -name "scalastyle-output.xml"`;
<?xml version="1.0" encoding="UTF-8"?>
<checkstyle version="5.0">
<file name="/home/runner/work/incubator-kyuubi/incubator-kyuubi/extensions/spark/kyuubi-extension-spark-3-3/src/main/scala/org/apache/kyuubi/sql/KyuubiSparkSQLExtension.scala">
<error column="0" line="21" source="org.scalastyle.scalariform.ImportOrderChecker" severity="error" message="There should at least one a single empty line separating groups 3rdParty and kyuubi."/>
</file>
</checkstyle>
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2780 from cfmcgrady/refine-stylecheck.
Closes#2780
a9c4a71e [Fu Chen] update reformat
f5d4d4e4 [Fu Chen] address comment
0665e7e5 [Fu Chen] address
91d123ee [Fu Chen] fix
f5d6414f [Fu Chen] style.yml
c56fa081 [Fu Chen] dev/reformat
Authored-by: Fu Chen <cfmcgrady@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
Close#2715Close#2374
Introduce jdbc based session state store to share the batch state information across kyuubi instances.
This pr is based on docs: https://docs.google.com/document/d/1_BMekMT88P2vCHCl4ZacWEeWslpIv4pxsJH32PAnWSI and refactor it during implementation.
In this PR, I introduce `StateStore` interface and implement the default `JDBCStateStore`.
The data structures:
1. BatchState - the batch state info for multiple-HA, including batch operation state and application information
2. BatchMeta - the meta data used for batch recovery, including sessionConf, batchRequest information
For JDBCStateStore:
- support to specify data base type:
- 1. DERBY - it will use derby jdbc driver by default
- 2. MYSQL - it will use mysql jdbc by default
- 3. CUSTOM - the user need specify the jdbc driver
- support to init schema
By default, the jdbc url is a derby in-memory database url for single kyuubi instance.
To support multiple HA in production env, we should config the production jdbc url to share batch info across kyuubi instances.
The datasource is managed by HiKariCP.
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2717 from turboFei/kyuubi_2715_hikaricp_v2.
Closes#2717
367b944a3 [Fei Wang] comments
9a7963dbd [Fei Wang] post session type in event
b1c130853 [Fei Wang] move session type to session package
66def0cb6 [Fei Wang] add JDBCStateStoreConf
839b72bf6 [Fei Wang] refactor
644cbcd28 [Fei Wang] filter by session type
58f1c7854 [Fei Wang] refactor
1c0035208 [Fei Wang] add session type
34a087ee5 [Fei Wang] refactor
fba38e473 [Fei Wang] refactor
fcbde25a0 [Fei Wang] refactor
332a67537 [Fei Wang] comments
135ba9b6d [Fei Wang] Revert "revert class utils change"
0e045a483 [Fei Wang] Revert "Revert "move""
05048416e [Fei Wang] Revert "test again"
01a50c35b [Fei Wang] test again
b89159d13 [Fei Wang] Revert "move"
3764a9051 [Fei Wang] move
eef17822c [Fei Wang] revert
4166db261 [Fei Wang] adjust engine idle timeout
5fc95a824 [Fei Wang] revert class utils change
23f88d63d [Fei Wang] refactor
a66cb811d [Fei Wang] refactor
89fa0edb4 [Fei Wang] fix
e5228fefd [Fei Wang] refactor
581908d86 [Fei Wang] exclude sql
3446aaa06 [Fei Wang] UPPER
ab980c677 [Fei Wang] [KYUUBI #2715] Introduce session state store for batch session multiple HA
Authored-by: Fei Wang <fwang12@ebay.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
### _Why are the changes needed?_
To close#2643
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2723 from lightning-L/kyuubi-2643.
Closes#2643
b7c409bcf [Tianlin Liao] unify junit dependency
ae2caca33 [Tianlin Liao] implements AutoClosable for KyuubiRestClient; rename server in rest builder to spnego builder
19e078a32 [Tianlin Liao] reuse ObjectMapper
ef98f4c2a [Tianlin Liao] minor improvements
15d958acf [Tianlin Liao] unify httpclient version parameter
37032a6a9 [Tianlin Liao] fix dependency
6c97745aa [Tianlin Liao] support NONE and NOSASL basic auth type
aa0a1c011 [Tianlin Liao] add optional tag for hadoop-client-api/hadoop-client-runtime
275d1d042 [Tianlin Liao] add integration test in kyuubi-server
b407d0229 [Tianlin Liao] apiBasePath is not necessary for the client builder
db11f10eb [Tianlin Liao] add test case for http response code is not 200
361343fe0 [Tianlin Liao] add builder arguments validation
a2684de86 [Tianlin Liao] remove gson dependency; fix get log api typo
ed3498938 [Tianlin Liao] Define KyuubiRestException and remove kyuubi-common dependency
b46f0c4d3 [Tianlin Liao] add AuthScheme config in Builder
61796d782 [Tianlin Liao] fix sun.security does not exist/not visible error
de9bb26e2 [Tianlin Liao] [KYUUBI #2643] [SUB-TASK][KPIP-4] Implement kyuubi rest sdk for batch job operation
Authored-by: Tianlin Liao <tiliao@ebay.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
### _Why are the changes needed?_
fix#2736
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2739 from jiaoqingbo/kyuubi-2736.
Closes#2736
00c8c618 [jiaoqingbo] fix ut failed
56a7b4ef [jiaoqingbo] fix failed
00184fc2 [jiaoqingbo] fix failed
11647f21 [jiaoqingbo] [KYUUBI #2736] Upgrade Jackson 2.13.3
Authored-by: jiaoqingbo <1178404354@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
As there are some issues with the TPC-DS column name reported in https://github.com/trinodb/tpcds/pull/2, we need a workaround before the upstream patch release is available.
This PR also introduces the option `useTableSchema_2_6`(default is `true`), user can set `false` to make the column name compatible with old TPC-DS queries.
Close#2679
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2702 from pan3793/tpcds-fix-colname.
Closes#2702
82503711 [Cheng Pan] Rename conf
fb007ed7 [Cheng Pan] tpcds 3.2
5ebf8691 [Cheng Pan] Fix TPC-DS columns name
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
To close#2674
Rewrite the rest DTO classes in java, which is the preparation for kyuubi rest sdk #2643
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2668 from turboFei/client_do.
Closes#2668
e2bf779c [Fei Wang] revert by reformat
a23f65f6 [Fei Wang] address comments
6648ef90 [Fei Wang] regen the equals & hashCode without accepct subclasses as parameter to equals()
840866c9 [Fei Wang] add equals & hashCode for dto classes
1cc9afaa [Fei Wang] return empty collections
2d49db50 [Fei Wang] return empty batch info map by defaults
1bb399c0 [Fei Wang] only reserve slf4j-api
c6621ed8 [Fei Wang] fix ut
e15ea9f0 [Fei Wang] refactor
ef6dc16c [Fei Wang] add dependes into assembly
ffacf68d [Fei Wang] rewrite the dto classes in java
Authored-by: Fei Wang <fwang12@ebay.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
### Why are the changes needed?
upgrade hive to 3.1.3 in the kyuubi-hive-jdbc module
### change details
file|comment
--- | ---
kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/HttpBasicAuthInterceptor.java | add customCookies as a construct parameter
kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/HttpKerberosRequestInterceptor.java | replace assumeSubject field with `Subject loggedInSubject` ; add customCookies as a construct parameter
kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/HttpRequestInterceptorBase.java | add customCookies as a construct parameter; add customCookies initialization codes
kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/HttpTokenAuthInterceptor.java | add customCookies as a construct parameter
kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/JdbcColumn.java | support TIMESTAMP_WITH_TIMEZONE & UNION_TYPE type
kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/KyuubiBaseResultSet.java | support TIMESTAMPLOCALTZ_TYPE
kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/KyuubiConnection.java | add loggedInSubject,wmPool & wmApp fields and their initialization codes; add getAllUrls function; use Utils.getCanonicalHostName to extract host from connParams; clear hive conf in embedded mode; use SessionUtils.getTokenStrForm to get token string; remove isZkDynamicDiscoveryMode
kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/KyuubiDatabaseMetaData.java | support getSQLKeywords; add Object class type parameter to KyuubiMetaDataResultSet
kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/KyuubiMetaDataResultSet.java | optimize initialization logic of data & columnTypes field; add normalizeColumnNames function
kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/KyuubiStatement.java | split closeClientOperation into 2 functions(closeStatementIfNeeded & closeClientOperation) & add reInitState function to replace initFlags & ad error messages to exceptions in CANCELED_STATE
kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/Utils.java | add dummyAuthorityString, SERVICE_DISCOVERY_MODE_ZOOKEEPER_HA, ZOOKEEPER_ACTIVE_PASSIVE_HA_DEFAULT_NAMESPACE, WM_POOL, HTTP_COOKIE_PREFIX, APPLICATION fields; replace authorityList field with suppliedURLAuthority in JdbcConnectionParams;add a construct function to JdbcConnectionParams; replace getAuthorityList function with getSuppliedURLAuthority; split parseURL into 2 different functions(parseURL, extractURLComponents); add configureConnParamsFromZooKeeper, handleAllDeprecations, getAuthorityFromJdbcURL functions; remove configureConnParams;remove joinStringArray;add getCanonicalHostName function
kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/ZooKeeperHiveClientHelper.java | add more functions
kyuubi-server/src/main/resources/META-INF/services/org.apache.hadoop.security.token.TokenIdentifier | change package name of DelegationTokenIdentifier
kyuubi-server/src/main/scala/org/apache/kyuubi/credentials/HiveDelegationTokenProvider.scala | change package name of DelegationTokenIdentifier
kyuubi-server/src/test/scala/org/apache/kyuubi/credentials/HiveDelegationTokenProviderSuite.scala | add configs, METASTORE_CONNECTION_POOLING_TYPE = none,METASTORE_AUTO_CREATE_ALL = true, METASTORE_SCHEMA_VERIFICATION = false; add construct parameter clientConf to HadoopThriftAuthBridgeWithServerContextClassLoader
kyuubi-common/src/test/scala/org/apache/kyuubi/operation/SparkMetadataTests.scala | remove getSQLKeywords from unsupported list
externals/kyuubi-hive-sql-engine/pom.xml | add HikariCP to test scope
pom.xml | update hive version to 3.1.3
kyuubi-hive-jdbc-shaded/pom.xml | add hive-standalone-metastore dependency; add hive-llap-client dependency
kyuubi-hive-jdbc/pom.xml | add hive-standalone-metastore dependency;add hive-llap-client dependency
kyuubi-server/pom.xml | add hive-standalone-metastore dependency
dev/dependencyList | add hive-standalone-metastore
kyuubi-hive-jdbc-shaded/src/main/resources/META-INF/LICENSE | update content because of hive-standalone-metastore & hive-llap-client
kyuubi-hive-jdbc-shaded/src/main/resources/META-INF/NOTICE | update content because of hive-standalone-metastore & hive-llap-client
### How was this patch tested?
use existing UTs to test this patch
Closes#2560 from gabrywu/jdbc-3.1.
Closes#2560
9c5e3eda [gabry.wu] Merge branch 'master' into jdbc-3.1
c23cefc5 [gabrywu] update NOTICE rollback unnecessary change
b9394144 [gabrywu] change hikaricp.version position to satisfy alphabetical order
59ebed37 [gabrywu] add hive-storage-api explicitly
d22dc585 [gabrywu] rollback to minimize diff of this PR
33c13f0f [gabrywu] update LICENSE, NOTICE, LICENSE add hive-llap-client to kyuubi-hive-jdbc-shaded module
d1ce2c6c [gabrywu] manage hikaricp.version under dependencyManagement
1f4b7c72 [gabrywu] Merge remote-tracking branch 'origin/jdbc-3.1' into jdbc-3.1
fd309054 [gabrywu] manage hikaricp.version under dependencyManagement
39aa7ae6 [gabrywu] manage hikaricp.version under dependencyManagement
85dc983a [gabrywu] manage hikaricp.version under dependencyManagement
545578c5 [gabrywu] update dependencyList file
f479f642 [gabrywu] remove optional flag of hive-standalone-metastore in kyuubi-server
5ee8f89b [gabrywu] 1. add hive-storage-api 2. enable METASTORE_AUTO_CREATE_ALL 3. disable METASTORE_SCHEMA_VERIFICATION
f70b0354 [gabrywu] add HikariCP to test scope
c93ab167 [gabrywu] set METASTORE_CONNECTION_POOLING_TYPE to none
feb53d6e [gabrywu] support getSQLKeywords func
4601b9b9 [gabrywu] exclude package.jdo
4f4a19bf [gabrywu] upgrade hive version to 3.3.1 in dependencyList
0b88a77a [gabrywu] upgrade hive to 3.1.3
Lead-authored-by: gabrywu <gabrywu@apache.org>
Co-authored-by: gabry.wu <gabrywu@apache.org>
Signed-off-by: Fei Wang <fwang12@ebay.com>
### _Why are the changes needed?_
This PR proposes change the Kyuubi TPC-DS generator to pure Java implementation instead of the original C binary.
The new pure Java TPC-DS generator is under Apache License, and in fact, I don't know the original C binary License, so we exclude them from release in the past.
Since the change removes the License issue of Kyuubi TPC-DS module, we can bundle the TPC-DS tool in the future release.
And after migration, I haven't see "error=26, Text file busy" described in #2439 any more.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
1. Use old C binary based TPC-DS generator generate 1GB data under database `tpcds_s1`
2. Use new pure Java base TPC-DS generator generate 1GB data under database `new_tpcds_sf1`
3. Compare results of `select count(*)`, and `select sum(hash(*))`
```
spark-sql> select count(*) from tpcds_s1.inventory;
11745000
Time taken: 0.161 seconds, Fetched 1 row(s)
spark-sql> select count(*) from new_tpcds_sf1.inventory;
11745000
Time taken: 0.141 seconds, Fetched 1 row(s)
spark-sql> select sum(hash(*)) from tpcds_s1.inventory;
-556768665838
Time taken: 0.252 seconds, Fetched 1 row(s)
spark-sql> select sum(hash(*)) from new_tpcds_sf1.inventory;
-556768665838
Time taken: 0.232 seconds, Fetched 1 row(s)
```
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2449 from pan3793/tpcds.
Closes#2439
a270bcba [Cheng Pan] Remove the exclusion in source release
7c8d3271 [Cheng Pan] [KYUUBI #2439] Using Pure Java TPC-DS generator
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
Separate events to a submodule - kyuubi-event
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2392 from zhaomin1423/event-module.
Closes#2323
7efa923f [Min Zhao] fix unit test
07126f43 [Min Zhao] [KYUUBI #2323] Separate events to a submodule - kyuubi-event
cccbdf11 [Min Zhao] [KYUUBI #2323] Separate events to a submodule - kyuubi-event
194aca82 [Min Zhao] add shade and log4j to test/resources
f75891a1 [Min Zhao] [KYUUBI #2323] Separate events to a submodule - kyuubi-event
357395fa [Min Zhao] [KYUUBI #2323] Separate events to a submodule - kyuubi-event
b6dfd186 [Min Zhao] [KYUUBI #2323] Separate events to a submodule - kyuubi-event
fc876688 [Min Zhao] [KYUUBI #2323] Separate events to a submodule - kyuubi-event
1b86d41b [Min Zhao] seperate event
Authored-by: Min Zhao <zhaomin1423@163.com>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
Bump hive-service-rpc 3.1.3 version
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2318 from deadwind4/2317-hive-rpc.
Closes#2317
93cf3b38 [Ada Wang] [KYUUBI #2317][BUILD] Bump hive-service-rpc 3.1.3 version
Authored-by: Ada Wang <wang4luning@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Move `dev/kyuubi-spark-extension*` to `extension/spark/`
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2291 from ulysses-you/unify-spark-extension.
Closes#2292
4e6f4ca6 [ulysses-you] fix
c2cc018d [ulysses-you] unify
2375af0d [ulysses-you] unify
Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
