Commit Graph

204 Commits

Author SHA1 Message Date
liangbowen
892a44dbae
[KYUUBI #4174] Bump guava from 30.1 to 31.1
### _Why are the changes needed?_

- bump `com.google.guava:guava` from 30.1 to 31.1 (released in Mar 2022, https://github.com/google/guava/releases/tag/v31.1)

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4174 from bowenliang123/guava-31.1.

Closes #4174

4376cd8a6 [liangbowen] update dependencyList
09b7f876e [liangbowen] bump guava from 30.1 to 31.1

Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Kent Yao <yao@apache.org>
2023-01-16 18:40:22 +08:00
liangbowen
5978bcf02e
[KYUUBI #4132] Bump commons-lang3 to 3.12.0 and commons-io to 2.11.0
### _Why are the changes needed?_

Upgrade all outdated Apache Commons series dependencies released in 2020 to the latest versions released in 2021.
- bump `commons-io` from 2.8.0 to 2.11.0 (release notes: https://commons.apache.org/proper/commons-io/changes-report.html#a2.11.0)
- bump `commons-lang3` from 3.10 to 3.12.0 (release notes: https://commons.apache.org/proper/commons-lang/changes-report.html#a3.12.0)

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4132 from bowenliang123/bump-apache-commons.

Closes #4132

2b8c7150 [liangbowen] update dependencyList
385ecc11 [liangbowen] bump commons-lang3 from 3.10 to 3.12.0
a452b611 [liangbowen] bump commons-io from 2.8.0 to 2.11.0

Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2023-01-11 16:06:35 +08:00
liangbowen
7459fac3b6 [KYUUBI #4082] Bump Jersey from 2.36 to 2.38
### _Why are the changes needed?_

Release notes for Jersey:
- https://github.com/eclipse-ee4j/jersey/releases/tag/2.38
- https://github.com/eclipse-ee4j/jersey/releases/tag/2.37

Changes in this PR:
- `jakarta.activation-api` is now removed from `dependencyList`
- `jakarta.activation-api` added to `kyuubi-kubernetes-it` module's dependency in test scope for satisfying  `hadoop-client-minicluster`

Facts:
- Before : with jersey 2.36,
	- `kyuubi-kubernetes-it` relies on `hadoop-client-minicluster` in test scope
	- `hadoop-client-minicluster` uses `DataSource` and other classes in package `javax.activation` without depending on the depencency
	- `javax.activation` package is actually provied by `jakarta.activation-api` included by `jersey-media-json-jackson`

- Investigation: with jersey 2.38
	- `jakarta.activation-api` was removed in Jersey 2.38 compared to 2.36 (https://github.com/eclipse-ee4j/jersey/compare/2.36...2.38)
	- from the blame history, it's clear that `jakarta.activation-api` was explictly excluded from `jersey-media-json-jackson`
	(https://github.com/eclipse-ee4j/jersey/blob/2.38/media/json-jackson/pom.xml), which in order to apply changes from `maven-enforcer-plugin` in (7e7d339281 (diff-f47eb7c28b888c017334e057492d4d2eab5be3e0e02bea1ff10765542bf9b9c8R101))

- Solution:
	- added `jakarta.activation-api` to `kyuubi-kubernetes-it` module's dependency in test scope statisfying `hadoop-client-minicluster`
	- confirm `jakarta.activation-api` is no longer directly or indirectly dependent in maven runtime scope

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4082 from bowenliang123/jersey-2.38.

Closes #4082

88c7cd80 [liangbowen] move jakarta.activation-api dependency closer to its dependant hadoop-client-minicluster
ce6df817 [liangbowen] add jakarta.activation-api dependency in test scope to kubernetes-integration-tests_2.1 module
8422dc2f [Bowen Liang] Merge branch 'master' into jersey-2.38
606cad0a [liangbowen] Bump Jersey from 2.36 to 2.38

Lead-authored-by: liangbowen <liangbowen@gf.com.cn>
Co-authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2023-01-05 01:48:36 +00:00
liangbowen
bfc56b2cf5
[KYUUBI #4079] Bump Jetty from 9.4.48.v20220622 to 9.4.50.v20221201
### _Why are the changes needed?_

- https://github.com/eclipse/jetty.project/releases/tag/jetty-9.4.50.v20221201
- https://github.com/eclipse/jetty.project/releases/tag/jetty-9.4.49.v20220914

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4079 from bowenliang123/jetty-9.4.50.

Closes #4079

e60883c1 [liangbowen] Bump Jetty from 9.4.48.v20220622 to 9.4.50.v20221201
6763d56f [liangbowen] Bump Jetty from 9.4.48.v20220622 to 9.4.50.v20221201

Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2023-01-04 18:01:55 +08:00
Tianlin Liao
8e8d8be8d2
[KYUUBI #4020] remove incubating from kyuubi source code
### _Why are the changes needed?_

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4069 from lightning-L/kyuubi-4020.

Closes #4020

97406ca0 [Tianlin Liao] [KYUUBI #4020] remove incubating from kyuubi source code

Authored-by: Tianlin Liao <tiliao@ebay.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
2023-01-04 09:43:20 +08:00
Yikf
ad1e9a422b
[KYUUBI #4039] Bump antlr4 from 4.8 to 4.9.3
### _Why are the changes needed?_

This PR aims to bump [antlr4](https://github.com/antlr/antlr4) from 4.8 to 4.9.3.

Version 4.8 was released on Jan 21, 2020, It's been nearly three years, and there have been a lot of bug fixes during that time, see more at [change list](https://github.com/antlr/antlr4/compare/4.8...4.9.3).

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4039 from Yikf/bump-antlr4.

Closes #4039

24aed3a7 [Yikf] Bump Antlr4 from 4.8 to 4.9.3

Authored-by: Yikf <yikaifei1@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-12-29 15:53:54 +08:00
Yikf
7b5af4ae09
[KYUUBI #3926] Introduce antlr4 to parse query statement
### _Why are the changes needed?_

Close https://github.com/apache/incubator-kyuubi/issues/3926, we intend to introduce the parser module based on antlr4 in the Apache kyuubi server. Through this module, we can achieve:

1. Expand kyuubi capabilities, for example, implement `SHOW KYUUBI_SESSIONS` through SQL.
2. Sql translation for mysql front-end protocol.
3. Sql translation for trino front-end protocol.

This issue is the first step of the parser module, which introduces the parser module and implements `SHOW KYUUBI_SESSION`, and then improves the Apache kyuubi parser module based on this initial pr.

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3944 from Yikf/parser-init.

Closes #3926

4463b298 [Yikf] [KYUUBI #3926] Introduce antlr4 to parse query statement

Authored-by: Yikf <yikaifei1@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-12-15 17:07:05 +08:00
odone
b5590b829d
[KYUUBI #3977] [KYUUBI #3905] Add basic TrinoResource
close #3905

### _Why are the changes needed?_

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3977 from iodone/kyuubi-3905.

Closes #3977

Closes #3905

4e4a22b1 [odone] trino server init and run

Authored-by: odone <odone.zhang@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
2022-12-15 09:27:06 +08:00
liangbowen
5e6c56abf4
[KYUUBI #3911] Bump Jackson from 2.13.4 to 2.14.1
### _Why are the changes needed?_

to close #3911 .

Bump Jackson from 2.13.4 to 2.14.1.

- From 2.14, Java8 is required
- v2.14.1 fixed heap consumption increased from 2.13.x to 2.14.0

Relase notes for 2.14 and 2.14.1
https://github.com/FasterXML/jackson/wiki/Jackson-Release-2.14
https://github.com/FasterXML/jackson/wiki/Jackson-Release-2.14.1

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3912 from bowenliang123/3911-jackson-214.

Closes #3911

5832471e9 [liangbowen] update dependencyList
669249117 [liangbowen] Bump Jackson from 2.13.4 to 2.14.1

Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Kent Yao <yao@apache.org>
2022-12-06 18:07:06 +08:00
liangbowen
2ac10f91d5
[KYUUBI #3842] [Improvement] Support maven pom.xml code style check with spotless plugin
### _Why are the changes needed?_

Introduce code style check support for Maven's pom.xml with sortPom in spotless maven plugin.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3843 from bowenliang123/spotless-pom.

Closes #3842

3c654597 [liangbowen] apply to pom.xml
fd1536f7 [liangbowen] set expandEmptyElements to true
e498423f [liangbowen] apply spotless:apply to all pom.xml
e46bcfec [liangbowen] add pom style check support in spotless

Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-11-23 22:08:00 +08:00
Xuedong Luan
6bf4e492bf
[KYUUBI #3834] Upgrade SLF4J from 1.7.35 to 1.7.36
### _Why are the changes needed?_

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3834 from leoluan2009/slf4j.

Closes #3834

e17b08ef [Xuedong Luan] Upgrade SLF4J from 1.7.35 to 1.7.36

Authored-by: Xuedong Luan <luanxuedong2009@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-11-22 21:42:42 +08:00
liangbowen
00f52a2cb2
[KYUUBI #3828] [PySpark] Support Python style check with spotless in CI style workflow and reformat tool
### _Why are the changes needed?_

to close #3828.

 Python code style checking support.
1. reuse Spotless maven plugin for Python style check
2. add style check to CI style workflow
3. add python linting support to `dev/reformat`. checks whether `black` installed in PATH.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3823 from bowenliang123/spotless-python.

Closes #3828

4a4de885 [liangbowen] simplify empty tags
0bb9ec7c [liangbowen] simplify empty tag in pom
9dd39531 [liangbowen] lint python code with black via spotless
f85020fa [liangbowen] typo
4c93bce0 [liangbowen] install python 3.9 first
23fc4b96 [liangbowen] ci install black version from added `spotless.python.black.version` property
73f746b0 [Bowen Liang] Update dev/reformat
46667a00 [liangbowen] update style.yml
9c20b434 [liangbowen] update style.yml
21017e5e [liangbowen] update style.yml
8272c0bc [liangbowen] add python style to style checking for CI
e102726c [liangbowen] add profile spotless in dev/reformat if black found in path
062e9bf2 [liangbowen] add python scan for spotless. add new profile `spotless-python` for python file path.

Lead-authored-by: liangbowen <liangbowen@gf.com.cn>
Co-authored-by: Bowen Liang <bowen.liang.123@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-11-22 19:44:02 +08:00
yuqi
c537557fcf
[KYUUBI #3813] [BUG] Caused by: java.lang.ClassNotFoundException: com.google.common.util.concurrent.internal.InternalFutureFailureAccess
### _Why are the changes needed?_

**Fix the following bug:**
```
2022-11-16 13:08:45.540 ERROR org.apache.kyuubi.server.KyuubiTBinaryFrontendService: Error opening session:
org.apache.kyuubi.KyuubiSQLException: Error opening session for test_user client ip 127.0.0.1, due to com/google/common/util/concurrent/internal/InternalFutureFailureAccess
	at org.apache.kyuubi.KyuubiSQLException$.apply(KyuubiSQLException.scala:69) ~[kyuubi-common_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	at org.apache.kyuubi.session.KyuubiSessionManager.openSession(KyuubiSessionManager.scala:102) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	at org.apache.kyuubi.service.AbstractBackendService.openSession(AbstractBackendService.scala:45) ~[kyuubi-common_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	at org.apache.kyuubi.server.KyuubiServer$$anon$1.org$apache$kyuubi$server$BackendServiceMetric$$super$openSession(KyuubiServer.scala:111) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	at org.apache.kyuubi.server.BackendServiceMetric.$anonfun$openSession$1(BackendServiceMetric.scala:37) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	at org.apache.kyuubi.metrics.MetricsSystem$.timerTracing(MetricsSystem.scala:111) ~[kyuubi-metrics_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	at org.apache.kyuubi.server.BackendServiceMetric.openSession(BackendServiceMetric.scala:37) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	at org.apache.kyuubi.server.BackendServiceMetric.openSession$(BackendServiceMetric.scala:30) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	at org.apache.kyuubi.server.KyuubiServer$$anon$1.openSession(KyuubiServer.scala:111) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	at org.apache.kyuubi.service.TFrontendService.getSessionHandle(TFrontendService.scala:181) ~[kyuubi-common_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	at org.apache.kyuubi.server.KyuubiTBinaryFrontendService.OpenSession(KyuubiTBinaryFrontendService.scala:81) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1497) ~[hive-service-rpc-3.1.3.jar:3.1.3]
	at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1482) ~[hive-service-rpc-3.1.3.jar:3.1.3]
	at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) ~[libthrift-0.9.3.jar:0.9.3]
	at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) ~[libthrift-0.9.3.jar:0.9.3]
	at org.apache.kyuubi.service.authentication.TSetIpAddressProcessor.process(TSetIpAddressProcessor.scala:36) ~[kyuubi-common_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) ~[libthrift-0.9.3.jar:0.9.3]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_222]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_222]
	at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_222]
Caused by: java.lang.NoClassDefFoundError: com/google/common/util/concurrent/internal/InternalFutureFailureAccess
	at java.lang.ClassLoader.defineClass1(Native Method) ~[?:1.8.0_222]
	at java.lang.ClassLoader.defineClass(ClassLoader.java:763) ~[?:1.8.0_222]
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) ~[?:1.8.0_222]
	at java.net.URLClassLoader.defineClass(URLClassLoader.java:468) ~[?:1.8.0_222]
	at java.net.URLClassLoader.access$100(URLClassLoader.java:74) ~[?:1.8.0_222]
	at java.net.URLClassLoader$1.run(URLClassLoader.java:369) ~[?:1.8.0_222]
	at java.net.URLClassLoader$1.run(URLClassLoader.java:363) ~[?:1.8.0_222]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_222]
	at java.net.URLClassLoader.findClass(URLClassLoader.java:362) ~[?:1.8.0_222]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_222]
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[?:1.8.0_222]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_222]
	at java.lang.ClassLoader.defineClass1(Native Method) ~[?:1.8.0_222]
	at java.lang.ClassLoader.defineClass(ClassLoader.java:763) ~[?:1.8.0_222]
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) ~[?:1.8.0_222]
	at java.net.URLClassLoader.defineClass(URLClassLoader.java:468) ~[?:1.8.0_222]
	at java.net.URLClassLoader.access$100(URLClassLoader.java:74) ~[?:1.8.0_222]
	at java.net.URLClassLoader$1.run(URLClassLoader.java:369) ~[?:1.8.0_222]
	at java.net.URLClassLoader$1.run(URLClassLoader.java:363) ~[?:1.8.0_222]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_222]
	at java.net.URLClassLoader.findClass(URLClassLoader.java:362) ~[?:1.8.0_222]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_222]
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[?:1.8.0_222]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_222]
	at java.lang.ClassLoader.defineClass1(Native Method) ~[?:1.8.0_222]
	at java.lang.ClassLoader.defineClass(ClassLoader.java:763) ~[?:1.8.0_222]
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) ~[?:1.8.0_222]
	at java.net.URLClassLoader.defineClass(URLClassLoader.java:468) ~[?:1.8.0_222]
	at java.net.URLClassLoader.access$100(URLClassLoader.java:74) ~[?:1.8.0_222]
	at java.net.URLClassLoader$1.run(URLClassLoader.java:369) ~[?:1.8.0_222]
	at java.net.URLClassLoader$1.run(URLClassLoader.java:363) ~[?:1.8.0_222]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_222]
	at java.net.URLClassLoader.findClass(URLClassLoader.java:362) ~[?:1.8.0_222]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_222]
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[?:1.8.0_222]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_222]
	at com.google.common.cache.LocalCache$LoadingValueReference.<init>(LocalCache.java:3472) ~[guava-30.1-jre.jar:?]
	at com.google.common.cache.LocalCache$LoadingValueReference.<init>(LocalCache.java:3476) ~[guava-30.1-jre.jar:?]
	at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2134) ~[guava-30.1-jre.jar:?]
	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045) ~[guava-30.1-jre.jar:?]
	at com.google.common.cache.LocalCache.get(LocalCache.java:3951) ~[guava-30.1-jre.jar:?]
	at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974) ~[guava-30.1-jre.jar:?]
	at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4935) ~[guava-30.1-jre.jar:?]
	at org.apache.kyuubi.session.FileSessionConfAdvisor.getConfOverlay(FileSessionConfAdvisor.scala:41) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	at org.apache.kyuubi.session.KyuubiSessionImpl.<init>(KyuubiSessionImpl.scala:55) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	at org.apache.kyuubi.session.KyuubiSessionManager.createSession(KyuubiSessionManager.scala:83) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	at org.apache.kyuubi.session.SessionManager.openSession(SessionManager.scala:98) ~[kyuubi-common_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	at org.apache.kyuubi.session.KyuubiSessionManager.openSession(KyuubiSessionManager.scala:95) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	... 18 more
Caused by: java.lang.ClassNotFoundException: com.google.common.util.concurrent.internal.InternalFutureFailureAccess
	at java.net.URLClassLoader.findClass(URLClassLoader.java:382) ~[?:1.8.0_222]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_222]
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[?:1.8.0_222]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_222]
	at java.lang.ClassLoader.defineClass1(Native Method) ~[?:1.8.0_222]
	at java.lang.ClassLoader.defineClass(ClassLoader.java:763) ~[?:1.8.0_222]
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) ~[?:1.8.0_222]
	at java.net.URLClassLoader.defineClass(URLClassLoader.java:468) ~[?:1.8.0_222]
	at java.net.URLClassLoader.access$100(URLClassLoader.java:74) ~[?:1.8.0_222]
	at java.net.URLClassLoader$1.run(URLClassLoader.java:369) ~[?:1.8.0_222]
	at java.net.URLClassLoader$1.run(URLClassLoader.java:363) ~[?:1.8.0_222]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_222]
	at java.net.URLClassLoader.findClass(URLClassLoader.java:362) ~[?:1.8.0_222]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_222]
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[?:1.8.0_222]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_222]
	at java.lang.ClassLoader.defineClass1(Native Method) ~[?:1.8.0_222]
	at java.lang.ClassLoader.defineClass(ClassLoader.java:763) ~[?:1.8.0_222]
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) ~[?:1.8.0_222]
	at java.net.URLClassLoader.defineClass(URLClassLoader.java:468) ~[?:1.8.0_222]
	at java.net.URLClassLoader.access$100(URLClassLoader.java:74) ~[?:1.8.0_222]
	at java.net.URLClassLoader$1.run(URLClassLoader.java:369) ~[?:1.8.0_222]
	at java.net.URLClassLoader$1.run(URLClassLoader.java:363) ~[?:1.8.0_222]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_222]
	at java.net.URLClassLoader.findClass(URLClassLoader.java:362) ~[?:1.8.0_222]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_222]
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[?:1.8.0_222]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_222]
	at java.lang.ClassLoader.defineClass1(Native Method) ~[?:1.8.0_222]
	at java.lang.ClassLoader.defineClass(ClassLoader.java:763) ~[?:1.8.0_222]
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) ~[?:1.8.0_222]
	at java.net.URLClassLoader.defineClass(URLClassLoader.java:468) ~[?:1.8.0_222]
	at java.net.URLClassLoader.access$100(URLClassLoader.java:74) ~[?:1.8.0_222]
	at java.net.URLClassLoader$1.run(URLClassLoader.java:369) ~[?:1.8.0_222]
	at java.net.URLClassLoader$1.run(URLClassLoader.java:363) ~[?:1.8.0_222]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_222]
	at java.net.URLClassLoader.findClass(URLClassLoader.java:362) ~[?:1.8.0_222]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_222]
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[?:1.8.0_222]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_222]
	at com.google.common.cache.LocalCache$LoadingValueReference.<init>(LocalCache.java:3472) ~[guava-30.1-jre.jar:?]
	at com.google.common.cache.LocalCache$LoadingValueReference.<init>(LocalCache.java:3476) ~[guava-30.1-jre.jar:?]
	at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2134) ~[guava-30.1-jre.jar:?]
	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045) ~[guava-30.1-jre.jar:?]
	at com.google.common.cache.LocalCache.get(LocalCache.java:3951) ~[guava-30.1-jre.jar:?]
	at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974) ~[guava-30.1-jre.jar:?]
	at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4935) ~[guava-30.1-jre.jar:?]
	at org.apache.kyuubi.session.FileSessionConfAdvisor.getConfOverlay(FileSessionConfAdvisor.scala:41) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	at org.apache.kyuubi.session.KyuubiSessionImpl.<init>(KyuubiSessionImpl.scala:55) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	at org.apache.kyuubi.session.KyuubiSessionManager.createSession(KyuubiSessionManager.scala:83) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	at org.apache.kyuubi.session.SessionManager.openSession(SessionManager.scala:98) ~[kyuubi-common_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	at org.apache.kyuubi.session.KyuubiSessionManager.openSession(KyuubiSessionManager.scala:95) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	... 18 more
```

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [x] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3813 from ychris78/yuqi_1116.

Closes #3813

7fbfdff8 [yuqi] update dev/dependencyList and LICENSE-binary
f664907d [yuqi] fix bug: Caused by: java.lang.ClassNotFoundException: com.google.common.util.concurrent.internal.InternalFutureFailureAccess

Authored-by: yuqi <1450636878@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-11-16 17:24:23 +08:00
Cheng Pan
be82aca237
[KYUUBI #3798] Bump protobuf-java from 3.21.1 to 3.21.7
Bumps [protobuf-java](https://github.com/protocolbuffers/protobuf) from 3.21.1 to 3.21.7.
<details>
<summary>Commits</summary>
<ul>
<li><a href="54489e95e0"><code>54489e9</code></a> Updating changelog</li>
<li><a href="5fc03e1647"><code>5fc03e1</code></a> Updating version.json and repo version numbers to: 21.7</li>
<li><a href="a3888f5331"><code>a3888f5</code></a> Clean up TextFormat parser (<a href="https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/10673">#10673</a>)</li>
<li><a href="3b5301c114"><code>3b5301c</code></a> Refactoring Java parsing (21.x) (<a href="https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/10665">#10665</a>)</li>
<li><a href="bea6726c73"><code>bea6726</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/10579">#10579</a> from protocolbuffers/21.x-202209142140</li>
<li><a href="b1924e1de9"><code>b1924e1</code></a> Update version.json to: 21.7-dev</li>
<li><a href="929e13d1a9"><code>929e13d</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/10572">#10572</a> from deannagarcia/21.x</li>
<li><a href="de7597e577"><code>de7597e</code></a> Update python/release.sh to handle delay between twine upload and pip install...</li>
<li><a href="480bd3b970"><code>480bd3b</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/10557">#10557</a> from protocolbuffers/21.x-202209132118</li>
<li><a href="aa8c73d845"><code>aa8c73d</code></a> Updating changelog</li>
<li>Additional commits viewable in <a href="https://github.com/protocolbuffers/protobuf/compare/v3.21.1...v3.21.7">compare view</a></li>
</ul>
</details>
<br />

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=com.google.protobuf:protobuf-java&package-manager=maven&previous-version=3.21.1&new-version=3.21.7)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `dependabot rebase` will rebase this PR
- `dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `dependabot merge` will merge this PR after your CI passes on it
- `dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `dependabot cancel merge` will cancel a previously requested merge and block automerging
- `dependabot reopen` will reopen this PR if it is closed
- `dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
- `dependabot use these labels` will set the current labels as the default for future PRs for this repo and language
- `dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language
- `dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language
- `dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/apache/incubator-kyuubi/network/alerts).

</details>

Closes #3798 from dependabot[bot]/dependabot/maven/com.google.protobuf-protobuf-java-3.21.7.

Closes #3798

ab92cde0 [Cheng Pan] deplist
5f535b97 [dependabot[bot]] Bump protobuf-java from 3.21.1 to 3.21.7

Lead-authored-by: Cheng Pan <chengpan@apache.org>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-11-11 11:58:57 +08:00
Cheng Pan
de0a92529a
[KYUUBI #3796] Bump Netty from 4.1.73.Final to 4.1.84.Final
### _Why are the changes needed?_

Bump Netty to latest version.

https://github.com/netty/netty/pull/12146 explained that netty-tcnative-classes should be optional, otherwise will cause https://github.com/netty/netty/issues/12101

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3796 from pan3793/netty.

Closes #3796

2b299233 [Cheng Pan] license
447a8d08 [Cheng Pan] nit
62abe481 [Cheng Pan] nit
506f76bf [Cheng Pan] Bump Netty from 4.1.73.Final to 4.1.84.Final

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-11-10 16:48:09 +08:00
Fu Chen
bea8e4f1cb [KYUUBI #3710] Delete the swagger UI related resource files from the source folder
### _Why are the changes needed?_

This pr aims to reduce the source file size.

This PR removes the swagger UI resource file in the folder `kyuubi-server/src/main/resources/org/apache/kyuubi/ui/swagger` which was introduced by [PR-2061](https://github.com/apache/incubator-kyuubi/pull/2061) , and now we rely on Maven dependency `org.webjars::swagger-ui::4.9.1` to package these resource files, we also add a new REST endpoint `/swagger-static/` for downloading these resource files

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3710 from cfmcgrady/reduce-src-size.

Closes #3710

c83bd313 [Fu Chen] update comment
441129b5 [Fu Chen] update LICENSE-binary
14eac59b [Fu Chen] update dev/dependencyList
7971c388 [Fu Chen] bump swagger-ui to  4.9.1
455aa206 [Fu Chen] reduce source file size

Authored-by: Fu Chen <cfmcgrady@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-10-29 10:40:07 +00:00
Bowen Liang
fa0dda1c11
[KYUUBI #3602] Bump log4j from 2.18.0 to 2.19.0
### _Why are the changes needed?_

to close #3602 .

Bump log4j from 2.18.0 to 2.19.0.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3603 from bowenliang123/3602-log4j2-218.

Closes #3602

fef6865c [Bowen Liang] Bump log4j from 2.18.0 to 2.19.0

Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-10-10 19:53:16 +08:00
Yikf
98ff122183
[KYUUBI #3573] Bump scala from 2.12.16 to 2.12.17
### _Why are the changes needed?_

Close https://github.com/apache/incubator-kyuubi/issues/3573.

This PR aims to upgrade scala to 2.12.17

1. [Scala 2.12.17](https://www.scala-lang.org/news/2.12.17) is available currently, This version has notable changes affecting compatibility, for more details see [release note](https://github.com/scala/scala/releases/tag/v2.12.17).
2. Spark was [upgraded](https://github.com/apache/spark/pull/37892) to this version half a month ago.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3574 from Yikf/bump-scala.

Closes #3573

51521d86 [Yikf] bump scala from 2.12.16 to 2.12.17

Authored-by: Yikf <yikaifei1@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-09-29 10:16:20 +08:00
zwangsheng
17289b13b5
[KYUUBI #3539] [FEATURE][TPCDS] Add white list help run the specified queries
### _Why are the changes needed?_

Add `white-list` help run the specified queries.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

  `--white-list q4,q15,q19,q25,q42,q49,q60,q66,q68,q72,q76,q78,q79,q89,q97`

![popo_2022-09-22  15-01-05](https://user-images.githubusercontent.com/52876270/191679906-76639468-a67d-408f-8b83-a18b34ef80e7.jpg)

Closes #3539 from zwangsheng/feature/tpcds_white_list.

Closes #3539

74657957 [zwangsheng] fix
26ade916 [Binjie Yang] Update dev/kyuubi-tpcds/src/main/scala/org/apache/kyuubi/tpcds/benchmark/RunBenchmark.scala
11a5b5c9 [Binjie Yang] Update dev/kyuubi-tpcds/src/main/scala/org/apache/kyuubi/tpcds/benchmark/RunBenchmark.scala
780410a3 [zwangsheng] fix
19018932 [zwangsheng] naming
1a4d1a25 [zwangsheng] naming
e1bb0069 [zwangsheng] fix
1be2bcb9 [zwangsheng] white

Lead-authored-by: zwangsheng <2213335496@qq.com>
Co-authored-by: Binjie Yang <52876270+zwangsheng@users.noreply.github.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-09-27 20:08:31 +08:00
zwangsheng
73612b7750
[KYUUBI #3522][FOLLOWUP] Fix benchmark results-dir
### _Why are the changes needed?_

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3523 from zwangsheng/bugfix/bechmark_fix_result_dir.

Closes #3522

4f957960 [zwangsheng] rebase
17de184f [zwangsheng] fix
39361cfc [zwangsheng] fix

Authored-by: zwangsheng <2213335496@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-09-20 11:29:10 +08:00
zwangsheng
9b67a0230f
[KYUUBI #3522] [BUG][TPCDS] Fix benchmark --results-dir
### _Why are the changes needed?_

`r` or `results-dir` should point to results-dir

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3522 from zwangsheng/bugfix/bechmark_fix_result_dir.

Closes #3522

39361cfc [zwangsheng] fix

Authored-by: zwangsheng <2213335496@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-09-20 10:41:17 +08:00
yikf
e87baa7505
[KYUUBI #3443] Bump Scala from 2.12.15 to 2.12.16
### _Why are the changes needed?_

Close https://github.com/apache/incubator-kyuubi/issues/3443, This pr aims to bump scala version from 2.12.15 to 2.12.16

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3467 from Yikf/bump-scala.

Closes #3443

63b6b381 [yikf] Bump Scala from 2.12.15 to 2.12.16

Authored-by: yikf <yikaifei1@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-09-11 17:25:32 +08:00
yongqian
0ae017402d
[KYUUBI #3460] Bump Jackson from 2.13.3 to 2.13.4
### _Why are the changes needed?_

close #3460

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3466 from QianyongY/features/kyuubi-3460.

Closes #3460

d3509a14 [yongqian] fix failed
52020fa2 [yongqian] [KYUUBI apache#3460] Bump Jackson from 2.13.3 to 2.13.4

Authored-by: yongqian <yongqian@trip.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-09-11 14:17:16 +08:00
Cheng Pan
08f6f4e13d
Revert "[KYUUBI #2041] Bump Zookeeper from 3.4.14 to 3.6.3"
This reverts commit 812ea2bdfe.
2022-09-09 16:48:36 +08:00
Cheng Pan
812ea2bdfe
[KYUUBI #2041] Bump Zookeeper from 3.4.14 to 3.6.3
### _Why are the changes needed?_

#2040 adds the test to verify the compatibility for Zookeeper Server 3.4, I will rebase this PR after #2040 get merged.

Close #1941

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #2041 from pan3793/zk.

Closes #2041

d6722f11 [Cheng Pan] license
eb7101a0 [Cheng Pan] fix
d9ce7c29 [Cheng Pan] Bump Zookeeper 3.6.3

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-09-09 12:31:55 +08:00
yikf
3adcebd557
[KYUUBI #3378][SUBTASK] Improve hive-connector module tests
### _Why are the changes needed?_

Fix https://github.com/apache/incubator-kyuubi/issues/3378

This pr aims to improve hive-connector module tests and make CI perform tests for the hive connector

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3379 from Yikf/hive-connector-test.

Closes #3378

72ad050c [yikf] CI test for hive-connector

Authored-by: yikf <yikaifei1@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-09-03 04:44:01 +08:00
odone
9716548380
[KYUUBI #2282] Add lineage records for sql statement execution in Kyuubi engine logs
### _Why are the changes needed?_

Lineage information:
```
col0 -> (table.a, table.b)
col1 -> (table.c, table.a)
```

SQL lineage logger JSON format example.
**SQL:**
```
select a as col0, b as col1 from test_table0
```
**Lineage:**
```
{
   "inputTables": ["default.test_table0"],
   "outputTables": [],
   "columnLineage": [{
      "column": "col0",
      "originalColumns": ["default.test_table0.a"]
   }, {
      "column": "col1",
      "originalColumns": ["default.test_table0.b"]
   }]
}
```

Currently supported column lineage for spark `Command` and `Query` `TreeNode`:

### Query
- `Select`

### Command
- `CreateDataSourceTableAsSelectCommand`
- `CreateHiveTableAsSelectCommand`
- `OptimizedCreateHiveTableAsSelectCommand`
- `CreateTableAsSelect`
- `ReplaceTableAsSelect`
- `InsertIntoDataSourceCommand`
- `InsertIntoHadoopFsRelationCommand`
- `InsertIntoDataSourceDirCommand`
- `InsertIntoHiveDirCommand`
- `InsertIntoHiveTable`

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3185 from iodone/kyuubi-2282.

Closes #2282

002c6d61 [odone] delete spark-sql-engine test for lineage
e1728a79 [odone] update lineage entity schema
de2a3e9a [odone] change kyuubi-spark-listener module to kyuubi-spark-lineage module
9258125e [odone] optimize lineage output
834669ed [odone] delete engine lineage parse
d9c7a3dc [odone] add spark listener to support lineage
4bae8c2f [odone] update for code cleaning
32b3392b [odone] update for review
fe09e478 [odone] add some test

Authored-by: odone <odone.zhang@gmail.com>
Signed-off-by: Kent Yao <yao@apache.org>
2022-08-24 14:11:35 +08:00
yikf
c01be3ae3d
[KYUUBI #3298][FOLLOWUP] Fix CI failure
### _Why are the changes needed?_

This pr is a follow-up, aims to fix CI failure due to https://github.com/apache/incubator-kyuubi/pull/3299#discussion_r953265471

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3319 from Yikf/master.

Closes #3298

d0cba33f [yikf] [KYUUBI #3298][FOLLOWUP] Fix CI failure

Authored-by: yikf <yikaifei1@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-08-24 11:54:36 +08:00
yikf
a54daf27c2
[KYUUBI #3298] Unify the approach of get classLoader in Kyuubi
### _Why are the changes needed?_

fix https://github.com/apache/incubator-kyuubi/issues/3298

Curently, a serval snippets use `getClass.getClassLoader` to get the classLoader in Apache Kyuubi, this pr aims to unify this behavior in `Utils.getContextOrKyuubiClassLoader`

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3299 from yikf/unify-calssloader.

Closes #3298

ae081ad9 [yikf] Unify the approach of get classLoader in Kyuubi

Authored-by: yikf <yikaifei1@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-08-23 13:50:10 +08:00
SteNicholas
77b036f3a8
[KYUUBI #3264] [RELEASE] Bump 1.7.0-SNAPSHOT
### _Why are the changes needed?_

Preparing v1.7.0-SNAPSHOT with branch-1.6 cut

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3264 from SteNicholas/prepare-1.7.0-snapshot.

Closes #3264

374d56bf [SteNicholas] preparing v1.7.0-SNAPSHOT with branch-1.6 cut

Authored-by: SteNicholas <programgeek@163.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-08-18 11:23:54 +08:00
Cheng Pan
782e5fb956
[KYUUBI #3162] Bump Hadoop 3.3.4
### _Why are the changes needed?_

Release notes: https://dist.apache.org/repos/dist/release/hadoop/common/hadoop-3.3.4/RELEASENOTES.md

Specifically, [HADOOP-18332](https://issues.apache.org/jira/browse/HADOOP-18332) fixed https://github.com/apache/incubator-kyuubi/issues/2904

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3162 from pan3793/hadoop-3.3.4.

Closes #3162

e6bfe650 [Cheng Pan] Remove staging repo
9c7f1d16 [Cheng Pan] Exclude snappy-java
7e0297b8 [Cheng Pan] Update dependencyList
bded82ba [Cheng Pan] Bump Hadoop 3.3.4-RC1
c46f8ec6 [Cheng Pan] Bump Hadoop 3.3.4-RC0

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-08-09 10:48:22 +08:00
Cheng Pan
a46d6550c7 [KYUUBI #3189] [BUILD] Bump jetcd 0.7.3 and pin Netty dependencies
### _Why are the changes needed?_

This PR proposes to upgrade the latest version of jetcd. As https://github.com/etcd-io/jetcd/pull/1074 has been included, https://github.com/apache/incubator-kyuubi/pull/3148 is not required then.

This PR also pin Netty dependecies' version.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3189 from pan3793/etcd.

Closes #3189

1b2317af [Cheng Pan] [BUILD] Bump jetcd 0.7.3 and pin Netty dependencies

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: hongdongdong <hongdongdong@cmss.chinamobile.com>
2022-08-08 14:02:51 +08:00
ParisaTork
3618002b96
[KYUUBI #3145] Bump log4j from 2.17.2 to 2.18.0
### _Why are the changes needed?_

Bumping log4j from 2.17.2 to 2.18.0 will reduce the risk of CVEs.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request (Ran ```./build/mvn clean test``` - see screenshots below for more info)

Closes #3187 from ParisaTork/bump-log4j.

Closes #3145

c9f9e4a8 [ParisaTork] Update dependency list
c0d88f39 [ParisaTork] Bump log4j from 2.17.2 to 2.18.0

Authored-by: ParisaTork <47482049+ParisaTork@users.noreply.github.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-08-07 22:02:49 +08:00
ParisaTork
b87ee9831c
[KYUUBI #3135] Bump gRPC from 1.47.0 to 1.48.0
### _Why are the changes needed?_

Bumping gRPC from 1.47.0 to 1.48.0 will reduce the risk of CVEs.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3188 from ParisaTork/bump-gRPC.

Closes #3135

491afa5d [ParisaTork] Bump gRPC from 1.47.0 to 1.48.0

Authored-by: ParisaTork <47482049+ParisaTork@users.noreply.github.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-08-07 14:58:45 +08:00
xsun4
b0685f9b48
[KYUUBI #3008] Bump prometheus from 0.14.1 to 0.16.0
### _Why are the changes needed?_
upgrade prometheus from 0.14.1 to 0.16.0

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

This patch had conflicts when merged, resolved by
Committer: Cheng Pan <chengpan@apache.org>

Closes #3097 from xinsun6/kyuubi_3008.

Closes #3097

97092c86 [xsun4] [KYUUBI apache#3008]Bump prometheus from 0.14.1 to 0.16.0

Authored-by: xsun4 <xsun4@ebay.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-07-20 10:14:44 +08:00
hongdongdong
32970ce685
[KYUUBI #2644] Add etcd discovery client for HA
### _Why are the changes needed?_

Add etcd discovery client for HA

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #2767 from hddong/etcd-support.

Closes #2644

6fa37578 [hongdongdong] Rebase and remove from rat-excludes
fc23a2b4 [hongdongdong] upgrade grpc-1.47.0
61c77d27 [hongdongdong] Fix tests
28abd13e [hongdongdong] fix comments
9e81a6a3 [hongdongdong] Rename HA_ZK_ENGINE_REF_ID to HA_ENGINE_REF_ID
aa2b4260 [hongdongdong] [KYUUBI #2644][WIP] Add etcd discovery client for HA

Authored-by: hongdongdong <hongdongdong@cmss.chinamobile.com>
Signed-off-by: Kent Yao <yao@apache.org>
2022-07-12 11:04:35 +08:00
guanhua.lgh
c1158accca
[KYUUBI #2974][FEATURE] EOL Support for Spark 3.0
### _Why are the changes needed?_
This PR is to drop support for spark 3.0 from compiling/building and test.  [#2974](https://github.com/apache/incubator-kyuubi/issues/2974)
Following the discussion on dev list https://lists.apache.org/thread/3mw8zz9r0tzg2dpj6717xy59610bhkp4

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [X] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3035 from huage1994/2974.

Closes #2974

add45cfe [guanhua.lgh] [FEATURE] EOL Support for Spark 3.0

Authored-by: guanhua.lgh <guanhua.lgh@alibaba-inc.com>
Signed-off-by: Kent Yao <yao@apache.org>
2022-07-12 11:01:50 +08:00
Cheng Pan
c5d29260a3
[KYUUBI #3037] Handles configuring the JUL -> SLF4J bridge
### _Why are the changes needed?_

Handles configuring the JUL -> SLF4J bridge, otherwise, the components which use JUL like Jersey will be not controlled by log4j2 configurations

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3037 from pan3793/jul.

Closes #3037

fda05ff6 [Cheng Pan] update license
31f579fd [Cheng Pan] nit
7da0f9fb [Cheng Pan] Fix dependencyList
3179e6bd [Cheng Pan] Handles configuring the JUL -> SLF4J bridge

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-07-11 17:54:39 +08:00
senmiaoliu
c652bba4da [KYUUBI #3007] Bump scopt from 4.0.1 to 4.1.0
### _Why are the changes needed?_

close #3007

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3015 from lsm1/features/Bump_scopt.

Closes #3007

876659ba [senmiaoliu] fix style
e1a42033 [LSM] Merge pull request #1 from cxzl25/PR_3015_UT
9a34eedb [sychen] fix UT
46e1dff9 [senmiaoliu] uodate dependencyList
8481b14b [senmiaoliu] Bump scopt from 4.0.1 to 4.1.0

Lead-authored-by: senmiaoliu <senmiaoliu@trip.com>
Co-authored-by: sychen <sychen@ctrip.com>
Co-authored-by: LSM <senmiaoliu@trip.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
2022-07-07 18:14:49 +08:00
senmiaoliu
1f59a592d8
[KYUUBI #3010] Bump Jetty from 9.4.41.v20210516 to 9.4.48.v20220622
### _Why are the changes needed?_

close #3010

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

This patch had conflicts when merged, resolved by
Committer: Cheng Pan <chengpan@apache.org>

Closes #3014 from lsm1/features/bump_Jetty.

Closes #3010

ae18e0f6 [senmiaoliu] update dependencyList
b32cd8c3 [senmiaoliu] Bump Jetty from 9.4.41.v20210516 to 9.4.48.v20220622

Authored-by: senmiaoliu <senmiaoliu@trip.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-07-05 20:14:06 +08:00
senmiaoliu
dc6e764f50
[KYUUBI #3011] Bump swagger from 2.1.11 to 2.2.1
### _Why are the changes needed?_

close #3011

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3016 from lsm1/features/bump_swagger.

Closes #3011

df0a18e4 [senmiaoliu] update dependencyList
ac07032a [senmiaoliu] Bump swagger from 2.1.11 to 2.2.1
6d8a1b8e [senmiaoliu] Bump swagger from 2.1.11 to 2.2.1

Authored-by: senmiaoliu <senmiaoliu@trip.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-07-05 20:10:17 +08:00
jiaoqingbo
a2431d0c37
[KYUUBI #3009] Bump Jersey from 2.35 to 2.36
### _Why are the changes needed?_

fix #3009

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3013 from jiaoqingbo/kyuubi-3009.

Closes #3009

6f47ec6d [jiaoqingbo] [KYUUBI #3009] Bump Jersey from 2.35 to 2.36

Authored-by: jiaoqingbo <1178404354@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-07-05 20:02:24 +08:00
Cheng Pan
5eb83b4c7c
[KYUUBI #2999] Fix Kyuubi Hive Beeline dependencies
### _Why are the changes needed?_

Before https://github.com/apache/incubator-kyuubi/pull/1715, `kyuubi-hive-beeline` and `kyuubi-hive-jdbc` managed dependencies in-self, after https://github.com/apache/incubator-kyuubi/pull/1715 and https://github.com/apache/incubator-kyuubi/issues/1716, `kyuubi-hive-beeline` uses some transitive dependencies of `kyuubi-hive-jdbc`, the recent dependencies change of `kyuubi-hive-jdbc` breaks `kyuubi-hive-beeline`.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #2999 from pan3793/beeline.

Closes #2999

b92b8a17 [Cheng Pan] Fix Kyuubi Hive Beeline dependencies

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-07-04 14:19:39 +08:00
zwangsheng
3bc299d01e
[KYUUBI #2868] [K8S] Add KubernetesApplicationOperation
### _Why are the changes needed?_

close #2868
Add Kubernetes Application Operation.
Help Tag Spark Driver with Engine Ref Id.

1) Find taged Engine:
`KyuubiApplicationManager => KubernetesApplicationOperation =(doesn't found taged pod)=> KubernetesApplicationOperation#JpsApplicationOperation`

2) `SparkBatchProcessBuilder` add logic to help tag Kubernetes Engine Tag.

3) Add Unit Test help to check `KubernetesApplicationOperation` find & kill  Spark Engine On Kubernetes with both client mode and cluster mode.

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #2869 from zwangsheng/feature/k8s-application-operation.

Closes #2868

04c97210 [zwangsheng] remove
fc151603 [zwangsheng] fix kyuubi on kubernetes integration test
29a3da52 [zwangsheng] fix
f43f0989 [zwangsheng] fix
2f2c9581 [zwangsheng] test
209dd65d [zwangsheng] fix jps
e12c681c [zwangsheng] fix tag
80971f98 [zwangsheng] fix
0760b63d [zwangsheng] fix
087eadb1 [zwangsheng] fix
d4989b09 [zwangsheng] try
93d83f81 [zwangsheng] add test
19378da4 [zwangsheng] fix style
63007261 [zwangsheng] fix test
fd4bf714 [zwangsheng] fix
66fc902b [zwangsheng] Using jps for client
2a1b797f [zwangsheng] fix setting
4827c459 [zwangsheng] fix
9ee90da6 [zwangsheng] decoupling spark
6b5831da [zwangsheng] useless driver label
f9ca898c [zwangsheng] fix
2add72fe [zwangsheng] fix setting
aaa081f8 [zwangsheng] fix style & dependency list
e102ac73 [zwangsheng] add kubernetesApplicationOperation

Authored-by: zwangsheng <2213335496@qq.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
2022-07-01 12:15:12 +08:00
Fei Wang
6f308c434c [KYUUBI #2964] [SUB-TASK][KPIP-4] Refine the batch response and render
### _Why are the changes needed?_

This is make some enhancements for batch response:
- combine batch name and batch app name
- flatten the batchInfo map to `appId`, `appUrl`, `appState` and`appDiagnostic`

Using https://github.com/JakeWharton/flip-tables for table render, which support that the column value could be a multiple-line string.
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [x] Add screenshots for manual tests if appropriate

Batch List:
![image](https://user-images.githubusercontent.com/6757692/176608602-89082758-4170-4337-ae64-b0dc75eaba77.png)

Batch Report:
![image](https://user-images.githubusercontent.com/6757692/176608636-448edf13-3147-4fc7-9df5-0192841f4201.png)

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #2964 from turboFei/refine_batch_resp.

Closes #2964

deaedcbc [Fei Wang] add into LICENSE-binary
1f9691e3 [Fei Wang] [SUB-TASK][KPIP-4] Refine the batch response and render

Authored-by: Fei Wang <fwang12@ebay.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
2022-06-30 18:27:45 +08:00
Cheng Pan
3435e2ae49
[KYUUBI #2915] Revert "[KYUUBI #2000][DEPS] Bump Hadoop 3.3.2"
### _Why are the changes needed?_

https://github.com/apache/incubator-kyuubi/issues/2904
This reverts commit e250c4ae35.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #2915 from pan3793/revert.

Closes #2915

Closes #2000

9c789eb1 [Cheng Pan] Revert "[KYUUBI #2000][DEPS] Bump Hadoop 3.3.2"

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-06-21 02:18:09 +08:00
Cheng Pan
2fd4e3a858
[KYUUBI #2827] [BUILD][TEST] Decouple integration tests from kyuubi-server
### _Why are the changes needed?_

This PR moves engine integration tests except to spark engine from `kyuubi-server` module to dedicated IT modules e.g. `kyuubi-hive-it`, then we can spilt different engine test into jobs to speed up CI tes

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [x] Add screenshots for manual tests if appropriate

before
<img width="836" alt="image" src="https://user-images.githubusercontent.com/26535726/173265839-5d0f1ce6-e133-4d60-9a95-c95aa16f817f.png">

after
<img width="825" alt="image" src="https://user-images.githubusercontent.com/26535726/173265903-fb01e910-8f78-4091-91da-dca25be4357d.png">

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #2827 from pan3793/hive-it.

Closes #2827

df3798f1 [Cheng Pan] Remove Java 11 test for Hive
e1645338 [Cheng Pan] nit
66524226 [Cheng Pan] nit
3aef2cef [Cheng Pan] review
500c9cff [Cheng Pan] MVN_OPT
3b4ad142 [Cheng Pan] nit
daa6e0ab [Cheng Pan] review
3383d4c1 [Cheng Pan] [BUILD][TEST] Decouple integration tests from kyuubi-server

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-06-18 00:01:34 +08:00
zhouyifan279
bdceaaf1f6
[KYUUBI #2883] [Bug] java.lang.NoClassDefFoundError: org/apache/hadoop/hive/common/ValidWriteIdList in HiveDelegationTokenProvider#initialize
### _Why are the changes needed?_
Fix #2879

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #2883 from zhouyifan279/2879.

Closes #2883

bbd6c2e7 [zhouyifan279] [Bug] java.lang.NoClassDefFoundError: org/apache/hadoop/hive/common/ValidWriteIdList in HiveDelegationTokenProvider#initialize
d5046f36 [zhouyifan279] [Bug] java.lang.NoClassDefFoundError: org/apache/hadoop/hive/common/ValidWriteIdList in HiveDelegationTokenProvider#initialize

Authored-by: zhouyifan279 <zhouyifan279@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-06-14 23:30:49 +08:00
Fu Chen
6cd2ad9ef7
[KYUUBI #2780] Refine stylecheck
### _Why are the changes needed?_

1. reformat code should include all of the modules in the project.
2. stylecheck should include all of the modules in the project.
3. there is a bug in the stage `Upload style check report`.

Since the `grep` returns code 1 when the pattern is not found, the upload report does not work before this pr.

```
> echo "a" > /tmp/a; grep abc /tmp/a; echo $?
1
```

Before this PR
```
Run for log in `find * -name "scalastyle-output.xml"`;  do echo "=========$log========="; grep "error" $log; done
=========dev/kyuubi-tpcds/target/scalastyle-output.xml=========
Error: Process completed with exit code 1.
```

After this PR
```
Run for log in `find * -name "scalastyle-output.xml"`;
<?xml version="1.0" encoding="UTF-8"?>
<checkstyle version="5.0">
 <file name="/home/runner/work/incubator-kyuubi/incubator-kyuubi/extensions/spark/kyuubi-extension-spark-3-3/src/main/scala/org/apache/kyuubi/sql/KyuubiSparkSQLExtension.scala">
  <error column="0" line="21" source="org.scalastyle.scalariform.ImportOrderChecker" severity="error" message="There should at least one a single empty line separating groups 3rdParty and kyuubi."/>
 </file>
</checkstyle>
```

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #2780 from cfmcgrady/refine-stylecheck.

Closes #2780

a9c4a71e [Fu Chen] update reformat
f5d4d4e4 [Fu Chen] address comment
0665e7e5 [Fu Chen] address
91d123ee [Fu Chen] fix
f5d6414f [Fu Chen] style.yml
c56fa081 [Fu Chen] dev/reformat

Authored-by: Fu Chen <cfmcgrady@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
2022-06-01 16:49:01 +08:00
Fei Wang
73c6b1b14a [KYUUBI #2717] [SUB-TASK][KPIP-4] Introduce jdbc session state store for batch session multiple HA
### _Why are the changes needed?_

Close #2715
Close #2374
Introduce jdbc based session state store to share the batch state information across kyuubi instances.

This pr is based on docs: https://docs.google.com/document/d/1_BMekMT88P2vCHCl4ZacWEeWslpIv4pxsJH32PAnWSI and  refactor it during implementation.

In this PR, I introduce `StateStore` interface and implement the default `JDBCStateStore`.

The data structures:
1. BatchState - the batch state info for multiple-HA, including batch operation state and application information
2. BatchMeta - the meta data used for batch recovery, including sessionConf, batchRequest information

For JDBCStateStore:
- support to specify data base type:
  - 1. DERBY - it will use derby jdbc driver by default
  - 2. MYSQL - it will use mysql jdbc by default
  - 3. CUSTOM - the user need specify the jdbc driver
- support to init schema

By default, the jdbc url is a derby in-memory database url for single kyuubi instance.

To support multiple HA in production env, we should config the production jdbc url to share batch info across kyuubi instances.

The datasource is managed by HiKariCP.

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #2717 from turboFei/kyuubi_2715_hikaricp_v2.

Closes #2717

367b944a3 [Fei Wang] comments
9a7963dbd [Fei Wang] post session type in event
b1c130853 [Fei Wang] move session type to session package
66def0cb6 [Fei Wang] add JDBCStateStoreConf
839b72bf6 [Fei Wang] refactor
644cbcd28 [Fei Wang] filter by session type
58f1c7854 [Fei Wang] refactor
1c0035208 [Fei Wang] add session type
34a087ee5 [Fei Wang] refactor
fba38e473 [Fei Wang] refactor
fcbde25a0 [Fei Wang] refactor
332a67537 [Fei Wang] comments
135ba9b6d [Fei Wang] Revert "revert class utils change"
0e045a483 [Fei Wang] Revert "Revert "move""
05048416e [Fei Wang] Revert "test again"
01a50c35b [Fei Wang] test again
b89159d13 [Fei Wang] Revert "move"
3764a9051 [Fei Wang] move
eef17822c [Fei Wang] revert
4166db261 [Fei Wang] adjust engine idle timeout
5fc95a824 [Fei Wang] revert class utils change
23f88d63d [Fei Wang] refactor
a66cb811d [Fei Wang] refactor
89fa0edb4 [Fei Wang] fix
e5228fefd [Fei Wang] refactor
581908d86 [Fei Wang] exclude sql
3446aaa06 [Fei Wang] UPPER
ab980c677 [Fei Wang] [KYUUBI #2715] Introduce session state store for batch session multiple HA

Authored-by: Fei Wang <fwang12@ebay.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
2022-05-31 15:06:39 +08:00