Commit Graph

226 Commits

Author SHA1 Message Date
Fu Chen
ad97b03579
[KYUUBI #4827] Bump Apache Arrow from 11.0.0 to 12.0.0
### _Why are the changes needed?_

https://arrow.apache.org/release/12.0.0.html

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4827 from cfmcgrady/arrow-12.0.0.

Closes #4827

324a8ddbf [Fu Chen] bump arrow from 11.0.0 to 12.0.0

Authored-by: Fu Chen <cfmcgrady@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2023-05-11 22:35:52 +08:00
Fu Chen
4c37a882d2 [KYUUBI #4824] Bump Jackson from 2.14.2 to 2.15.0
### _Why are the changes needed?_

spark bump jackson from 2.14.2 to 2.15.0 in https://github.com/apache/spark/pull/40933

to fix

https://github.com/apache/kyuubi/actions/runs/4943800010/jobs/8838642303

```
Caused by: org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 1.0 failed 1 times, most recent failure: Lost task 0.0 in stage 1.0 (TID 1) (localhost executor driver): java.lang.NoClassDefFoundError: com/fasterxml/jackson/core/StreamReadConstraints
	at org.apache.spark.sql.catalyst.json.JSONOptions.buildJsonFactory(JSONOptions.scala:195)
	at org.apache.spark.sql.catalyst.json.JsonInferSchema.$anonfun$infer$1(JsonInferSchema.scala:83)
	at org.apache.spark.rdd.RDD.$anonfun$mapPartitions$2(RDD.scala:855)
	at org.apache.spark.rdd.RDD.$anonfun$mapPartitions$2$adapted(RDD.scala:855)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:364)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:328)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:92)
	at org.apache.spark.TaskContext.runTaskWithListeners(TaskContext.scala:161)
	at org.apache.spark.scheduler.Task.run(Task.scala:139)
	at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:554)
	at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1514)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:557)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.ClassNotFoundException: com.fasterxml.jackson.core.StreamReadConstraints
	at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
	... 16 more
```

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4824 from cfmcgrady/jackson-2.15.0.

Closes #4824

7a8a3de89 [Fu Chen] update dev/dependencyList
2d01b4b9d [Fu Chen] bump jackson

Authored-by: Fu Chen <cfmcgrady@gmail.com>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
2023-05-11 21:05:09 +08:00
liangbowen
4e0562df8f [KYUUBI #4808] Bump Jetty from 9.4.50 to 9.4.51
### _Why are the changes needed?_

- to fix 2 Dependabot alerts
  1. https://github.com/apache/kyuubi/security/dependabot/19
  2. https://github.com/apache/kyuubi/security/dependabot/18

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4808 from bowenliang123/jetty-9.4.51.

Closes #4808

84789292b [liangbowen] update dependencyList
d1ebdc68a [liangbowen] Revert "update dependencyList"
e4e26b7b5 [liangbowen] update dependencyList
6cfb37dbb [liangbowen] bump jetty to 9.4.51

Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: bowenliang <bowenliang@apache.org>
2023-05-09 08:44:00 +08:00
liangbowen
d73ec64b37 [KYUUBI #4733] Introduce Kafka event logger for server events
### _Why are the changes needed?_

- introduce new event logger type `KAFKA`
- send server events to the Kafka topic with initializing and closing Kafka producer properly with server's lifecyle
- use Kafka 3.4.0 as the client version, and tested with Kakfa servers of 2.8.x and 3.4.x

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4733 from bowenliang123/kafka-logger.

Closes #4733

b5220d234 [liangbowen] introduce kafka server event logger

Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
2023-05-08 22:45:52 +08:00
liangbowen
0b72a6151c [KYUUBI #4772] Bump Jersey from 2.39 to 2.39.1
### _Why are the changes needed?_

- Jersey 2.39.1 release note: https://github.com/eclipse-ee4j/jersey/releases/tag/2.39.1
    - https://github.com/eclipse-ee4j/jersey/pull/5282

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4772 from bowenliang123/jersey-2.39.1.

Closes #4772

356753565 [liangbowen] update dependencyList
e3d3f11bd [liangbowen] bump jersey from 2.39 to 2.39.1

Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
2023-04-25 22:22:16 +08:00
Cheng Pan
a9d3e11926
[KYUUBI #4652] Upgrade Hadoop from 3.3.4 to 3.3.5
### _Why are the changes needed?_

Upgrade Hadoop from 3.3.4 to 3.3.5, close #4652

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4689 from huapan123456/master.

Closes #4652

810af9848 [Cheng Pan] fix
380786e9f [Cheng Pan] nit
7c5f18ef9 [Cheng Pan] nit
2d308943b [Cheng Pan] nit
76869d822 [Cheng Pan] handle HDFS-16591
36142c787 [huapan] fix(KYUUBI #4652): the position of JaasConfiguration has changed on hadoop-clint-api-3.3.5
1b5de6dfe [huapan] feat(KYUUBI #4652): upgrade hadoop-version from 3.3.4 to 3.3.5
f9d34c716 [huapan] feat(KYUUBI #4652): upgrade hadoop-client-api from 3.3.4 to 3.3.5

Lead-authored-by: Cheng Pan <chengpan@apache.org>
Co-authored-by: huapan <huapan@52tt.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2023-04-13 13:06:29 +08:00
yehere
ca93a4e5c2
[KYUUBI #4325] Support replace preparedStatement for Trino-jdbc
### _Why are the changes needed?_

close #4325

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4417 from yehere/kyuubi-4325.

Closes #4325

7b2864b53 [yehere] [KYUUBI #4325] Support replace preparedStatement for Trino-jdbc,update the dependency file, run './build/dependency.sh --replace'
749b1c15c [yehere] [KYUUBI #4325] Support replace preparedStatement for Trino-jdbc,Code optimization
33ea9ba2b [yehere] [KYUUBI #4325] Support replace preparedStatement for Trino-jdbc,Code optimization
568418a21 [yehere] [KYUUBI #4325] Support replace preparedStatement for Trino-jdbc,Add test case for DEALLOCATE PREPARE
358a8e3b8 [yehere] [KYUUBI #4325] Support replace preparedStatement for Trino-jdbc,Support DEALLOCATE PREPARE
7d4a32402 [yehere] [KYUUBI #4325] Support replace preparedStatement for Trino-jdbc
91392add6 [yehere] [KYUUBI #4325] Support replace preparedStatement for Trino-jdbc
63bf8c462 [yehere] [KYUUBI #4325] Support replace preparedStatement for Trino-jdbc
f5b7fb786 [yehere] [KYUUBI #4325] Support replace preparedStatement for Trino-jdbc
b0476a79d [yehere] [KYUUBI #4325] Support replace preparedStatement for Trino-jdbc
1a8f147a0 [yehere] [KYUUBI #4325] Support replace preparedStatement for Trino-jdbc

Authored-by: yehere <867171931@qq.com>
Signed-off-by: ulyssesyou <ulyssesyou@apache.org>
2023-03-30 20:37:55 +08:00
zwangsheng
1bc05e5e45
[KYUUBI #4453][Improvement][K8S] Bump Kubernetes Client Version to 6.4.1
### _Why are the changes needed?_

Close #4453

kubernetes client [compare](https://github.com/fabric8io/kubernetes-client/compare/v5.12.1...v6.4.1)

version | K8s 1.26.0 | K8s 1.25.3 | K8s 1.24.7 | K8s 1.23.13 | K8s 1.22.1 | K8s 1.21.1 | K8s 1.20.2 | K8s 1.19.1 | K8s 1.18.0 | K8s 1.17.0 | K8s 1.16.0 | K8s 1.15.3 | K8s 1.14.2 | K8s 1.12.0 | K8s 1.11.0 | K8s 1.10.0 | K8s 1.9.0
--|-- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | --
kubernetes-client 6.4.1 |   | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | -
kubernetes-client 5.12.1 |   |   |   | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | -

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

- [x] [Run ci test]

Closes #4456 from zwangsheng/bump/kubernetes-client-6.4.1.

Closes #4453

39039f0f9 [zwangsheng] [KYUUBI #4453] Remove dup dependence
4a5b27ce8 [zwangsheng] [KYUUBI #4453] IT test with same logic
95a292f99 [zwangsheng] [KYUUBI #4453] Fix Dependency
e4bf0107e [zwangsheng] [KYUUBI #4453] Init Bump

Authored-by: zwangsheng <2213335496@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2023-03-07 14:02:26 +08:00
liangbowen
cc4ec5a5d5 [KYUUBI #4448] Bump Netty from 4.1.87.Final to 4.1.89.Final
### _Why are the changes needed?_

- Netty 4.1.89.Final: https://netty.io/news/2023/02/13/4-1-89-Final.html
- Netty 4.1.88.Final: https://netty.io/news/2023/02/12/4-1-88-Final.html

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4448 from bowenliang123/netty-4.1.89.

Closes #4448

8854bb3ea [liangbowen] bump netty to 4.1.89.Final

Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
2023-03-06 13:05:46 +08:00
liangbowen
489d8a3c66 [KYUUBI #4447] Bump log4j from 2.19.0 to 2.20.0
### _Why are the changes needed?_

- log4j 2.20.0 release notes: https://logging.apache.org/log4j/2.x/release-notes/2.20.0.html

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4447 from bowenliang123/log4j-2.20.0.

Closes #4447

c9bc9aa95 [liangbowen] bump log4j from 2.19.0 to 2.20.0

Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
2023-03-06 11:20:16 +08:00
liangbowen
9d33b849fb
[KYUUBI #4357] Bump Jersey from 2.38 to 2.39
### _Why are the changes needed?_

to close #4357.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4366 from bowenliang123/jersey-2.39.

Closes #4357

bd214e8d [liangbowen] bump jersey from 2.38 to 2.39

Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2023-02-19 16:33:26 +08:00
liangbowen
bb3e06a035
[KYUUBI #4312] [DOCS] Include **/README.md in markdown style check
### _Why are the changes needed?_

- Include `**/README.md` markdown files in spotless style check

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4312 from bowenliang123/reformat-readme.

Closes #4312

1fda1bdeb [Bowen Liang] Merge branch 'master' into reformat-readme
2ca8b4c81 [liangbowen] merge master
876f52a4c [liangbowen] include `**/README.md` in spotless style check

Lead-authored-by: liangbowen <liangbowen@gf.com.cn>
Co-authored-by: Bowen Liang <bowenliang@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2023-02-14 02:23:32 +08:00
odone
41f08059f0 [KYUUBI #3935] Support use Trino client to submit SQL
### _Why are the changes needed?_

Close #3935

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4232 from iodone/kyuubi-3935.

Closes #3935

936ea1f8 [odone] address
e7bd01a1 [odone] support trino client connect kyuubi trino server
9ea8b6af [odone] [WIP] trion request/response implementation

Authored-by: odone <odone.zhang@gmail.com>
Signed-off-by: ulyssesyou <ulyssesyou@apache.org>
2023-02-13 19:28:14 +08:00
Cheng Pan
4e226ac3cc
Bump 1.8.0-SNAPSHOT 2023-02-10 15:25:49 +08:00
runzhliu
673e8e1ce7 [KYUUBI #4286] Bump Jackson from 2.14.1 to 2.14.2
### _Why are the changes needed?_

- Jackson 2.14.2 release note: https://github.com/FasterXML/jackson/wiki/Jackson-Release-2.14.2

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4286 from runzhliu/patch-4.

Closes #4286

6902e2ec [runzhliu] Update dependencyList
3cde2004 [runzhliu] Merge branch 'apache:master' into patch-4
8ac6eccd [runzhliu] bump the jackson maven dependecy version to 2.14.2

Authored-by: runzhliu <runzhliu@163.com>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
2023-02-10 07:39:22 +08:00
Cheng Pan
eb1b11cd17 [KYUUBI #4152] Enhance LDAP authentication
### _Why are the changes needed?_

This PR proposes to enhance the LDAP support, which mainly referring the code introduced in HIVE-14713.

Currently, Kyuubi has very limited LDAP support, and the implementation is from the early Hive codebase. Hive enhanced the LDAP support in later versions, considering the Hive ecosystem is quite mature, I think we'd better to porting this functionality and keep the same behavior w/ Hive first, and we can improve it if meet certain requirements/issues in the future.

Basically, this PR introduces the following configurations

```
kyuubi.authentication.ldap.url (since 1.0.0)
kyuubi.authentication.ldap.domain (since 1.0.0)
kyuubi.authentication.ldap.guidKey (since 1.2.0)
kyuubi.authentication.ldap.base.dn (since 1.0.0 deprecated)
kyuubi.authentication.ldap.baseDN
kyuubi.authentication.ldap.groupMembershipKey
kyuubi.authentication.ldap.userMembershipKey
kyuubi.authentication.ldap.groupClassKey
kyuubi.authentication.ldap.groupDNPattern
kyuubi.authentication.ldap.userDNPattern
kyuubi.authentication.ldap.groupFilter
kyuubi.authentication.ldap.userFilter
kyuubi.authentication.ldap.customLDAPQuery
kyuubi.authentication.ldap.binddn
kyuubi.authentication.ldap.bindpw
```

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

This PR ports all LDAP-related UT&IT from Hive codebase

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4152 from pan3793/ldap.

Closes #4152

d251c959 [Cheng Pan] nit
6d14f44b [Cheng Pan] nit
6b3d116c [Cheng Pan] nit
ab47d822 [Cheng Pan] nit
a56e8702 [Cheng Pan] nit
4624619a [Cheng Pan] nit
b82c0c05 [Cheng Pan] LDAP test password uses alphanumeric
86a01cca [Cheng Pan] Enhance LDAP authentication

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2023-02-03 05:48:02 +00:00
fwang12
e70a2b791a [KYUUBI #4230] Bump httpclient version to 4.5.14 and httpcore version to 4.4.16
### _Why are the changes needed?_

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4230 from turboFei/upgrade_http_version.

Closes #4230

10fe8a34 [fwang12] Bump httpclient and httpcore version

Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
2023-02-02 22:33:48 +08:00
liangbowen
37cf4cb29e [KYUUBI #4221] REST client for creating batch with uploading resource file
### _Why are the changes needed?_

- REST client support for creating batch with uploading resource file (https://kyuubi.readthedocs.io/en/master/client/rest/rest_api.html#post-batches-with-uploading-resource) introduced in #4144

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4221 from bowenliang123/restclient-batch-upload.

Closes #4221

9ec83766 [liangbowen] add MultiPart class to avoid exposing httpclient classes to RestClient interface
83051843 [liangbowen] fix content type
056edb84 [liangbowen] nit
74ad5642 [liangbowen] style
68dcdb38 [liangbowen] update dependencyList
a274d392 [liangbowen] client for creating batch with uploading resource file

Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
2023-02-02 17:46:14 +08:00
liangbowen
b5b4cd1407
[KYUUBI #4207] Bump snakeyaml from 1.31 to 1.33
### _Why are the changes needed?_

- Bump snakeyaml from 1.31 to 1.33 reducing 2 direct CVE vulnerabilities, see (https://bitbucket.org/snakeyaml/snakeyaml/wiki/Changes)

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4207 from bowenliang123/snakeyaml-1.33.

Closes #4207

080024ce5 [liangbowen] bump snakeyaml from 1.31 to 1.33

Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2023-01-29 17:13:46 +08:00
liangbowen
5357a0211b [KYUUBI #4201] [Improvement] Remove Gson as direct dependency by using Jackson instead in Authz tests
### _Why are the changes needed?_

- use `Jackson` instead of `Gson` in Authz tests, where it is the only place `Gson` is used in project
- remove `Gson` as the dependency
- Notice: Gson project is in maintenance mode, (see https://github.com/google/gson/pull/2085)
- Gson is still on `dependencyList`, as depended in kyuubi-ha
<img width="640" alt="image" src="https://user-images.githubusercontent.com/1935105/213956635-ce079638-15c3-4de2-9f46-a93dfe3db199.png">

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4201 from bowenliang123/remove-gson.

Closes #4201

96df2513 [liangbowen] update dependencyList
81303d6e [liangbowen] imports
a73a1f98 [liangbowen] use jackson instead of gson in authz testing and remove gson as dependency

Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
2023-01-23 16:16:23 +08:00
liangbowen
609071d110 [KYUUBI #4106] Introduce resource file uploading in batch creation via REST API
### _Why are the changes needed?_

to close #4106 .

1. add `POST /batches`API in `BatchesResource` of REST API, which consumes `multipart/form-data` media type to support uploading `resourcefile`. And a `batchRequest` form data part in the JSON format string of `BatchRequest`, as in the required request body of `POST /batches`
2. the uploaded `resourceFile` is saved to a temp local file which will be cleaned up after job execution at the end of  `submitAndMonitorBatchJob`
3. the local temp copy of `resourceFile` will be used as `resource` in `BatchJobSubmission`, eg. as <application-jar> for spark-submit

Todos in follow-up:
1. add a related description in Rest API doc
4. add `multipart` media type support to `RestClient` and implement `createBatchWithUploadingResource` in `BatchRestApi`

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4144 from bowenliang123/rest-batch-upload.

Closes #4106

f6723a02 [Bowen Liang] Merge branch 'master' into rest-batch-upload
0dd67245 [liangbowen] correct dependencyList
6365a0cd [liangbowen] introducing resource file upload support in batch creation

Lead-authored-by: liangbowen <liangbowen@gf.com.cn>
Co-authored-by: Bowen Liang <bowenliang@apache.org>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
2023-01-21 08:37:46 +08:00
liangbowen
dd04f818bc [KYUUBI #4190] Bump Netty from 4.1.84 to 4.1.87
### _Why are the changes needed?_

- Bump Netty from `4.1.84.Final` to `4.1.87.Final` (release note: https://netty.io/news/2023/01/12/4-1-87-Final.html)
- with 2 CVE ( including 1 in high risk level) fixed in 4.1.86.Final for 4.1.85 and before, `CVE-2022-41915` and `CVE-2022-41881`
 (https://netty.io/news/2022/12/12/4-1-86-Final.html)
- exclude `netty-handler-ssl-ocsp` which is released with `netty-all` since `4.1.86.Final`, as no SSL ocsp related feature used in kyuubi server

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4190 from bowenliang123/netty-4.1.87.

Closes #4190

97198aa9 [liangbowen] exclude netty-handler-ssl-ocsp from netty-all
44c3fab2 [liangbowen] update dependencyList
26a9ca7f [liangbowen] bump netty from 4.1.84 to 4.1.87

Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
2023-01-20 13:16:21 +08:00
liangbowen
892a44dbae
[KYUUBI #4174] Bump guava from 30.1 to 31.1
### _Why are the changes needed?_

- bump `com.google.guava:guava` from 30.1 to 31.1 (released in Mar 2022, https://github.com/google/guava/releases/tag/v31.1)

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4174 from bowenliang123/guava-31.1.

Closes #4174

4376cd8a6 [liangbowen] update dependencyList
09b7f876e [liangbowen] bump guava from 30.1 to 31.1

Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Kent Yao <yao@apache.org>
2023-01-16 18:40:22 +08:00
liangbowen
5978bcf02e
[KYUUBI #4132] Bump commons-lang3 to 3.12.0 and commons-io to 2.11.0
### _Why are the changes needed?_

Upgrade all outdated Apache Commons series dependencies released in 2020 to the latest versions released in 2021.
- bump `commons-io` from 2.8.0 to 2.11.0 (release notes: https://commons.apache.org/proper/commons-io/changes-report.html#a2.11.0)
- bump `commons-lang3` from 3.10 to 3.12.0 (release notes: https://commons.apache.org/proper/commons-lang/changes-report.html#a3.12.0)

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4132 from bowenliang123/bump-apache-commons.

Closes #4132

2b8c7150 [liangbowen] update dependencyList
385ecc11 [liangbowen] bump commons-lang3 from 3.10 to 3.12.0
a452b611 [liangbowen] bump commons-io from 2.8.0 to 2.11.0

Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2023-01-11 16:06:35 +08:00
liangbowen
7459fac3b6 [KYUUBI #4082] Bump Jersey from 2.36 to 2.38
### _Why are the changes needed?_

Release notes for Jersey:
- https://github.com/eclipse-ee4j/jersey/releases/tag/2.38
- https://github.com/eclipse-ee4j/jersey/releases/tag/2.37

Changes in this PR:
- `jakarta.activation-api` is now removed from `dependencyList`
- `jakarta.activation-api` added to `kyuubi-kubernetes-it` module's dependency in test scope for satisfying  `hadoop-client-minicluster`

Facts:
- Before : with jersey 2.36,
	- `kyuubi-kubernetes-it` relies on `hadoop-client-minicluster` in test scope
	- `hadoop-client-minicluster` uses `DataSource` and other classes in package `javax.activation` without depending on the depencency
	- `javax.activation` package is actually provied by `jakarta.activation-api` included by `jersey-media-json-jackson`

- Investigation: with jersey 2.38
	- `jakarta.activation-api` was removed in Jersey 2.38 compared to 2.36 (https://github.com/eclipse-ee4j/jersey/compare/2.36...2.38)
	- from the blame history, it's clear that `jakarta.activation-api` was explictly excluded from `jersey-media-json-jackson`
	(https://github.com/eclipse-ee4j/jersey/blob/2.38/media/json-jackson/pom.xml), which in order to apply changes from `maven-enforcer-plugin` in (7e7d339281 (diff-f47eb7c28b888c017334e057492d4d2eab5be3e0e02bea1ff10765542bf9b9c8R101))

- Solution:
	- added `jakarta.activation-api` to `kyuubi-kubernetes-it` module's dependency in test scope statisfying `hadoop-client-minicluster`
	- confirm `jakarta.activation-api` is no longer directly or indirectly dependent in maven runtime scope

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4082 from bowenliang123/jersey-2.38.

Closes #4082

88c7cd80 [liangbowen] move jakarta.activation-api dependency closer to its dependant hadoop-client-minicluster
ce6df817 [liangbowen] add jakarta.activation-api dependency in test scope to kubernetes-integration-tests_2.1 module
8422dc2f [Bowen Liang] Merge branch 'master' into jersey-2.38
606cad0a [liangbowen] Bump Jersey from 2.36 to 2.38

Lead-authored-by: liangbowen <liangbowen@gf.com.cn>
Co-authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2023-01-05 01:48:36 +00:00
liangbowen
bfc56b2cf5
[KYUUBI #4079] Bump Jetty from 9.4.48.v20220622 to 9.4.50.v20221201
### _Why are the changes needed?_

- https://github.com/eclipse/jetty.project/releases/tag/jetty-9.4.50.v20221201
- https://github.com/eclipse/jetty.project/releases/tag/jetty-9.4.49.v20220914

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4079 from bowenliang123/jetty-9.4.50.

Closes #4079

e60883c1 [liangbowen] Bump Jetty from 9.4.48.v20220622 to 9.4.50.v20221201
6763d56f [liangbowen] Bump Jetty from 9.4.48.v20220622 to 9.4.50.v20221201

Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2023-01-04 18:01:55 +08:00
Tianlin Liao
8e8d8be8d2
[KYUUBI #4020] remove incubating from kyuubi source code
### _Why are the changes needed?_

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4069 from lightning-L/kyuubi-4020.

Closes #4020

97406ca0 [Tianlin Liao] [KYUUBI #4020] remove incubating from kyuubi source code

Authored-by: Tianlin Liao <tiliao@ebay.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
2023-01-04 09:43:20 +08:00
Yikf
ad1e9a422b
[KYUUBI #4039] Bump antlr4 from 4.8 to 4.9.3
### _Why are the changes needed?_

This PR aims to bump [antlr4](https://github.com/antlr/antlr4) from 4.8 to 4.9.3.

Version 4.8 was released on Jan 21, 2020, It's been nearly three years, and there have been a lot of bug fixes during that time, see more at [change list](https://github.com/antlr/antlr4/compare/4.8...4.9.3).

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4039 from Yikf/bump-antlr4.

Closes #4039

24aed3a7 [Yikf] Bump Antlr4 from 4.8 to 4.9.3

Authored-by: Yikf <yikaifei1@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-12-29 15:53:54 +08:00
Yikf
7b5af4ae09
[KYUUBI #3926] Introduce antlr4 to parse query statement
### _Why are the changes needed?_

Close https://github.com/apache/incubator-kyuubi/issues/3926, we intend to introduce the parser module based on antlr4 in the Apache kyuubi server. Through this module, we can achieve:

1. Expand kyuubi capabilities, for example, implement `SHOW KYUUBI_SESSIONS` through SQL.
2. Sql translation for mysql front-end protocol.
3. Sql translation for trino front-end protocol.

This issue is the first step of the parser module, which introduces the parser module and implements `SHOW KYUUBI_SESSION`, and then improves the Apache kyuubi parser module based on this initial pr.

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3944 from Yikf/parser-init.

Closes #3926

4463b298 [Yikf] [KYUUBI #3926] Introduce antlr4 to parse query statement

Authored-by: Yikf <yikaifei1@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-12-15 17:07:05 +08:00
odone
b5590b829d
[KYUUBI #3977] [KYUUBI #3905] Add basic TrinoResource
close #3905

### _Why are the changes needed?_

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3977 from iodone/kyuubi-3905.

Closes #3977

Closes #3905

4e4a22b1 [odone] trino server init and run

Authored-by: odone <odone.zhang@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
2022-12-15 09:27:06 +08:00
liangbowen
5e6c56abf4
[KYUUBI #3911] Bump Jackson from 2.13.4 to 2.14.1
### _Why are the changes needed?_

to close #3911 .

Bump Jackson from 2.13.4 to 2.14.1.

- From 2.14, Java8 is required
- v2.14.1 fixed heap consumption increased from 2.13.x to 2.14.0

Relase notes for 2.14 and 2.14.1
https://github.com/FasterXML/jackson/wiki/Jackson-Release-2.14
https://github.com/FasterXML/jackson/wiki/Jackson-Release-2.14.1

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3912 from bowenliang123/3911-jackson-214.

Closes #3911

5832471e9 [liangbowen] update dependencyList
669249117 [liangbowen] Bump Jackson from 2.13.4 to 2.14.1

Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Kent Yao <yao@apache.org>
2022-12-06 18:07:06 +08:00
liangbowen
2ac10f91d5
[KYUUBI #3842] [Improvement] Support maven pom.xml code style check with spotless plugin
### _Why are the changes needed?_

Introduce code style check support for Maven's pom.xml with sortPom in spotless maven plugin.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3843 from bowenliang123/spotless-pom.

Closes #3842

3c654597 [liangbowen] apply to pom.xml
fd1536f7 [liangbowen] set expandEmptyElements to true
e498423f [liangbowen] apply spotless:apply to all pom.xml
e46bcfec [liangbowen] add pom style check support in spotless

Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-11-23 22:08:00 +08:00
Xuedong Luan
6bf4e492bf
[KYUUBI #3834] Upgrade SLF4J from 1.7.35 to 1.7.36
### _Why are the changes needed?_

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3834 from leoluan2009/slf4j.

Closes #3834

e17b08ef [Xuedong Luan] Upgrade SLF4J from 1.7.35 to 1.7.36

Authored-by: Xuedong Luan <luanxuedong2009@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-11-22 21:42:42 +08:00
liangbowen
00f52a2cb2
[KYUUBI #3828] [PySpark] Support Python style check with spotless in CI style workflow and reformat tool
### _Why are the changes needed?_

to close #3828.

 Python code style checking support.
1. reuse Spotless maven plugin for Python style check
2. add style check to CI style workflow
3. add python linting support to `dev/reformat`. checks whether `black` installed in PATH.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3823 from bowenliang123/spotless-python.

Closes #3828

4a4de885 [liangbowen] simplify empty tags
0bb9ec7c [liangbowen] simplify empty tag in pom
9dd39531 [liangbowen] lint python code with black via spotless
f85020fa [liangbowen] typo
4c93bce0 [liangbowen] install python 3.9 first
23fc4b96 [liangbowen] ci install black version from added `spotless.python.black.version` property
73f746b0 [Bowen Liang] Update dev/reformat
46667a00 [liangbowen] update style.yml
9c20b434 [liangbowen] update style.yml
21017e5e [liangbowen] update style.yml
8272c0bc [liangbowen] add python style to style checking for CI
e102726c [liangbowen] add profile spotless in dev/reformat if black found in path
062e9bf2 [liangbowen] add python scan for spotless. add new profile `spotless-python` for python file path.

Lead-authored-by: liangbowen <liangbowen@gf.com.cn>
Co-authored-by: Bowen Liang <bowen.liang.123@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-11-22 19:44:02 +08:00
yuqi
c537557fcf
[KYUUBI #3813] [BUG] Caused by: java.lang.ClassNotFoundException: com.google.common.util.concurrent.internal.InternalFutureFailureAccess
### _Why are the changes needed?_

**Fix the following bug:**
```
2022-11-16 13:08:45.540 ERROR org.apache.kyuubi.server.KyuubiTBinaryFrontendService: Error opening session:
org.apache.kyuubi.KyuubiSQLException: Error opening session for test_user client ip 127.0.0.1, due to com/google/common/util/concurrent/internal/InternalFutureFailureAccess
	at org.apache.kyuubi.KyuubiSQLException$.apply(KyuubiSQLException.scala:69) ~[kyuubi-common_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	at org.apache.kyuubi.session.KyuubiSessionManager.openSession(KyuubiSessionManager.scala:102) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	at org.apache.kyuubi.service.AbstractBackendService.openSession(AbstractBackendService.scala:45) ~[kyuubi-common_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	at org.apache.kyuubi.server.KyuubiServer$$anon$1.org$apache$kyuubi$server$BackendServiceMetric$$super$openSession(KyuubiServer.scala:111) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	at org.apache.kyuubi.server.BackendServiceMetric.$anonfun$openSession$1(BackendServiceMetric.scala:37) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	at org.apache.kyuubi.metrics.MetricsSystem$.timerTracing(MetricsSystem.scala:111) ~[kyuubi-metrics_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	at org.apache.kyuubi.server.BackendServiceMetric.openSession(BackendServiceMetric.scala:37) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	at org.apache.kyuubi.server.BackendServiceMetric.openSession$(BackendServiceMetric.scala:30) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	at org.apache.kyuubi.server.KyuubiServer$$anon$1.openSession(KyuubiServer.scala:111) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	at org.apache.kyuubi.service.TFrontendService.getSessionHandle(TFrontendService.scala:181) ~[kyuubi-common_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	at org.apache.kyuubi.server.KyuubiTBinaryFrontendService.OpenSession(KyuubiTBinaryFrontendService.scala:81) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1497) ~[hive-service-rpc-3.1.3.jar:3.1.3]
	at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1482) ~[hive-service-rpc-3.1.3.jar:3.1.3]
	at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) ~[libthrift-0.9.3.jar:0.9.3]
	at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) ~[libthrift-0.9.3.jar:0.9.3]
	at org.apache.kyuubi.service.authentication.TSetIpAddressProcessor.process(TSetIpAddressProcessor.scala:36) ~[kyuubi-common_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) ~[libthrift-0.9.3.jar:0.9.3]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_222]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_222]
	at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_222]
Caused by: java.lang.NoClassDefFoundError: com/google/common/util/concurrent/internal/InternalFutureFailureAccess
	at java.lang.ClassLoader.defineClass1(Native Method) ~[?:1.8.0_222]
	at java.lang.ClassLoader.defineClass(ClassLoader.java:763) ~[?:1.8.0_222]
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) ~[?:1.8.0_222]
	at java.net.URLClassLoader.defineClass(URLClassLoader.java:468) ~[?:1.8.0_222]
	at java.net.URLClassLoader.access$100(URLClassLoader.java:74) ~[?:1.8.0_222]
	at java.net.URLClassLoader$1.run(URLClassLoader.java:369) ~[?:1.8.0_222]
	at java.net.URLClassLoader$1.run(URLClassLoader.java:363) ~[?:1.8.0_222]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_222]
	at java.net.URLClassLoader.findClass(URLClassLoader.java:362) ~[?:1.8.0_222]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_222]
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[?:1.8.0_222]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_222]
	at java.lang.ClassLoader.defineClass1(Native Method) ~[?:1.8.0_222]
	at java.lang.ClassLoader.defineClass(ClassLoader.java:763) ~[?:1.8.0_222]
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) ~[?:1.8.0_222]
	at java.net.URLClassLoader.defineClass(URLClassLoader.java:468) ~[?:1.8.0_222]
	at java.net.URLClassLoader.access$100(URLClassLoader.java:74) ~[?:1.8.0_222]
	at java.net.URLClassLoader$1.run(URLClassLoader.java:369) ~[?:1.8.0_222]
	at java.net.URLClassLoader$1.run(URLClassLoader.java:363) ~[?:1.8.0_222]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_222]
	at java.net.URLClassLoader.findClass(URLClassLoader.java:362) ~[?:1.8.0_222]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_222]
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[?:1.8.0_222]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_222]
	at java.lang.ClassLoader.defineClass1(Native Method) ~[?:1.8.0_222]
	at java.lang.ClassLoader.defineClass(ClassLoader.java:763) ~[?:1.8.0_222]
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) ~[?:1.8.0_222]
	at java.net.URLClassLoader.defineClass(URLClassLoader.java:468) ~[?:1.8.0_222]
	at java.net.URLClassLoader.access$100(URLClassLoader.java:74) ~[?:1.8.0_222]
	at java.net.URLClassLoader$1.run(URLClassLoader.java:369) ~[?:1.8.0_222]
	at java.net.URLClassLoader$1.run(URLClassLoader.java:363) ~[?:1.8.0_222]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_222]
	at java.net.URLClassLoader.findClass(URLClassLoader.java:362) ~[?:1.8.0_222]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_222]
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[?:1.8.0_222]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_222]
	at com.google.common.cache.LocalCache$LoadingValueReference.<init>(LocalCache.java:3472) ~[guava-30.1-jre.jar:?]
	at com.google.common.cache.LocalCache$LoadingValueReference.<init>(LocalCache.java:3476) ~[guava-30.1-jre.jar:?]
	at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2134) ~[guava-30.1-jre.jar:?]
	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045) ~[guava-30.1-jre.jar:?]
	at com.google.common.cache.LocalCache.get(LocalCache.java:3951) ~[guava-30.1-jre.jar:?]
	at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974) ~[guava-30.1-jre.jar:?]
	at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4935) ~[guava-30.1-jre.jar:?]
	at org.apache.kyuubi.session.FileSessionConfAdvisor.getConfOverlay(FileSessionConfAdvisor.scala:41) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	at org.apache.kyuubi.session.KyuubiSessionImpl.<init>(KyuubiSessionImpl.scala:55) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	at org.apache.kyuubi.session.KyuubiSessionManager.createSession(KyuubiSessionManager.scala:83) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	at org.apache.kyuubi.session.SessionManager.openSession(SessionManager.scala:98) ~[kyuubi-common_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	at org.apache.kyuubi.session.KyuubiSessionManager.openSession(KyuubiSessionManager.scala:95) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	... 18 more
Caused by: java.lang.ClassNotFoundException: com.google.common.util.concurrent.internal.InternalFutureFailureAccess
	at java.net.URLClassLoader.findClass(URLClassLoader.java:382) ~[?:1.8.0_222]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_222]
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[?:1.8.0_222]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_222]
	at java.lang.ClassLoader.defineClass1(Native Method) ~[?:1.8.0_222]
	at java.lang.ClassLoader.defineClass(ClassLoader.java:763) ~[?:1.8.0_222]
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) ~[?:1.8.0_222]
	at java.net.URLClassLoader.defineClass(URLClassLoader.java:468) ~[?:1.8.0_222]
	at java.net.URLClassLoader.access$100(URLClassLoader.java:74) ~[?:1.8.0_222]
	at java.net.URLClassLoader$1.run(URLClassLoader.java:369) ~[?:1.8.0_222]
	at java.net.URLClassLoader$1.run(URLClassLoader.java:363) ~[?:1.8.0_222]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_222]
	at java.net.URLClassLoader.findClass(URLClassLoader.java:362) ~[?:1.8.0_222]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_222]
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[?:1.8.0_222]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_222]
	at java.lang.ClassLoader.defineClass1(Native Method) ~[?:1.8.0_222]
	at java.lang.ClassLoader.defineClass(ClassLoader.java:763) ~[?:1.8.0_222]
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) ~[?:1.8.0_222]
	at java.net.URLClassLoader.defineClass(URLClassLoader.java:468) ~[?:1.8.0_222]
	at java.net.URLClassLoader.access$100(URLClassLoader.java:74) ~[?:1.8.0_222]
	at java.net.URLClassLoader$1.run(URLClassLoader.java:369) ~[?:1.8.0_222]
	at java.net.URLClassLoader$1.run(URLClassLoader.java:363) ~[?:1.8.0_222]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_222]
	at java.net.URLClassLoader.findClass(URLClassLoader.java:362) ~[?:1.8.0_222]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_222]
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[?:1.8.0_222]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_222]
	at java.lang.ClassLoader.defineClass1(Native Method) ~[?:1.8.0_222]
	at java.lang.ClassLoader.defineClass(ClassLoader.java:763) ~[?:1.8.0_222]
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) ~[?:1.8.0_222]
	at java.net.URLClassLoader.defineClass(URLClassLoader.java:468) ~[?:1.8.0_222]
	at java.net.URLClassLoader.access$100(URLClassLoader.java:74) ~[?:1.8.0_222]
	at java.net.URLClassLoader$1.run(URLClassLoader.java:369) ~[?:1.8.0_222]
	at java.net.URLClassLoader$1.run(URLClassLoader.java:363) ~[?:1.8.0_222]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_222]
	at java.net.URLClassLoader.findClass(URLClassLoader.java:362) ~[?:1.8.0_222]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_222]
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[?:1.8.0_222]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_222]
	at com.google.common.cache.LocalCache$LoadingValueReference.<init>(LocalCache.java:3472) ~[guava-30.1-jre.jar:?]
	at com.google.common.cache.LocalCache$LoadingValueReference.<init>(LocalCache.java:3476) ~[guava-30.1-jre.jar:?]
	at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2134) ~[guava-30.1-jre.jar:?]
	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045) ~[guava-30.1-jre.jar:?]
	at com.google.common.cache.LocalCache.get(LocalCache.java:3951) ~[guava-30.1-jre.jar:?]
	at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974) ~[guava-30.1-jre.jar:?]
	at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4935) ~[guava-30.1-jre.jar:?]
	at org.apache.kyuubi.session.FileSessionConfAdvisor.getConfOverlay(FileSessionConfAdvisor.scala:41) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	at org.apache.kyuubi.session.KyuubiSessionImpl.<init>(KyuubiSessionImpl.scala:55) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	at org.apache.kyuubi.session.KyuubiSessionManager.createSession(KyuubiSessionManager.scala:83) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	at org.apache.kyuubi.session.SessionManager.openSession(SessionManager.scala:98) ~[kyuubi-common_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	at org.apache.kyuubi.session.KyuubiSessionManager.openSession(KyuubiSessionManager.scala:95) ~[kyuubi-server_2.12-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
	... 18 more
```

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [x] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3813 from ychris78/yuqi_1116.

Closes #3813

7fbfdff8 [yuqi] update dev/dependencyList and LICENSE-binary
f664907d [yuqi] fix bug: Caused by: java.lang.ClassNotFoundException: com.google.common.util.concurrent.internal.InternalFutureFailureAccess

Authored-by: yuqi <1450636878@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-11-16 17:24:23 +08:00
Cheng Pan
be82aca237
[KYUUBI #3798] Bump protobuf-java from 3.21.1 to 3.21.7
Bumps [protobuf-java](https://github.com/protocolbuffers/protobuf) from 3.21.1 to 3.21.7.
<details>
<summary>Commits</summary>
<ul>
<li><a href="54489e95e0"><code>54489e9</code></a> Updating changelog</li>
<li><a href="5fc03e1647"><code>5fc03e1</code></a> Updating version.json and repo version numbers to: 21.7</li>
<li><a href="a3888f5331"><code>a3888f5</code></a> Clean up TextFormat parser (<a href="https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/10673">#10673</a>)</li>
<li><a href="3b5301c114"><code>3b5301c</code></a> Refactoring Java parsing (21.x) (<a href="https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/10665">#10665</a>)</li>
<li><a href="bea6726c73"><code>bea6726</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/10579">#10579</a> from protocolbuffers/21.x-202209142140</li>
<li><a href="b1924e1de9"><code>b1924e1</code></a> Update version.json to: 21.7-dev</li>
<li><a href="929e13d1a9"><code>929e13d</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/10572">#10572</a> from deannagarcia/21.x</li>
<li><a href="de7597e577"><code>de7597e</code></a> Update python/release.sh to handle delay between twine upload and pip install...</li>
<li><a href="480bd3b970"><code>480bd3b</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/10557">#10557</a> from protocolbuffers/21.x-202209132118</li>
<li><a href="aa8c73d845"><code>aa8c73d</code></a> Updating changelog</li>
<li>Additional commits viewable in <a href="https://github.com/protocolbuffers/protobuf/compare/v3.21.1...v3.21.7">compare view</a></li>
</ul>
</details>
<br />

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=com.google.protobuf:protobuf-java&package-manager=maven&previous-version=3.21.1&new-version=3.21.7)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `dependabot rebase` will rebase this PR
- `dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `dependabot merge` will merge this PR after your CI passes on it
- `dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `dependabot cancel merge` will cancel a previously requested merge and block automerging
- `dependabot reopen` will reopen this PR if it is closed
- `dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
- `dependabot use these labels` will set the current labels as the default for future PRs for this repo and language
- `dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language
- `dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language
- `dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/apache/incubator-kyuubi/network/alerts).

</details>

Closes #3798 from dependabot[bot]/dependabot/maven/com.google.protobuf-protobuf-java-3.21.7.

Closes #3798

ab92cde0 [Cheng Pan] deplist
5f535b97 [dependabot[bot]] Bump protobuf-java from 3.21.1 to 3.21.7

Lead-authored-by: Cheng Pan <chengpan@apache.org>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-11-11 11:58:57 +08:00
Cheng Pan
de0a92529a
[KYUUBI #3796] Bump Netty from 4.1.73.Final to 4.1.84.Final
### _Why are the changes needed?_

Bump Netty to latest version.

https://github.com/netty/netty/pull/12146 explained that netty-tcnative-classes should be optional, otherwise will cause https://github.com/netty/netty/issues/12101

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3796 from pan3793/netty.

Closes #3796

2b299233 [Cheng Pan] license
447a8d08 [Cheng Pan] nit
62abe481 [Cheng Pan] nit
506f76bf [Cheng Pan] Bump Netty from 4.1.73.Final to 4.1.84.Final

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-11-10 16:48:09 +08:00
Fu Chen
bea8e4f1cb [KYUUBI #3710] Delete the swagger UI related resource files from the source folder
### _Why are the changes needed?_

This pr aims to reduce the source file size.

This PR removes the swagger UI resource file in the folder `kyuubi-server/src/main/resources/org/apache/kyuubi/ui/swagger` which was introduced by [PR-2061](https://github.com/apache/incubator-kyuubi/pull/2061) , and now we rely on Maven dependency `org.webjars::swagger-ui::4.9.1` to package these resource files, we also add a new REST endpoint `/swagger-static/` for downloading these resource files

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3710 from cfmcgrady/reduce-src-size.

Closes #3710

c83bd313 [Fu Chen] update comment
441129b5 [Fu Chen] update LICENSE-binary
14eac59b [Fu Chen] update dev/dependencyList
7971c388 [Fu Chen] bump swagger-ui to  4.9.1
455aa206 [Fu Chen] reduce source file size

Authored-by: Fu Chen <cfmcgrady@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-10-29 10:40:07 +00:00
Bowen Liang
fa0dda1c11
[KYUUBI #3602] Bump log4j from 2.18.0 to 2.19.0
### _Why are the changes needed?_

to close #3602 .

Bump log4j from 2.18.0 to 2.19.0.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3603 from bowenliang123/3602-log4j2-218.

Closes #3602

fef6865c [Bowen Liang] Bump log4j from 2.18.0 to 2.19.0

Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-10-10 19:53:16 +08:00
Yikf
98ff122183
[KYUUBI #3573] Bump scala from 2.12.16 to 2.12.17
### _Why are the changes needed?_

Close https://github.com/apache/incubator-kyuubi/issues/3573.

This PR aims to upgrade scala to 2.12.17

1. [Scala 2.12.17](https://www.scala-lang.org/news/2.12.17) is available currently, This version has notable changes affecting compatibility, for more details see [release note](https://github.com/scala/scala/releases/tag/v2.12.17).
2. Spark was [upgraded](https://github.com/apache/spark/pull/37892) to this version half a month ago.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3574 from Yikf/bump-scala.

Closes #3573

51521d86 [Yikf] bump scala from 2.12.16 to 2.12.17

Authored-by: Yikf <yikaifei1@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-09-29 10:16:20 +08:00
zwangsheng
17289b13b5
[KYUUBI #3539] [FEATURE][TPCDS] Add white list help run the specified queries
### _Why are the changes needed?_

Add `white-list` help run the specified queries.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

  `--white-list q4,q15,q19,q25,q42,q49,q60,q66,q68,q72,q76,q78,q79,q89,q97`

![popo_2022-09-22  15-01-05](https://user-images.githubusercontent.com/52876270/191679906-76639468-a67d-408f-8b83-a18b34ef80e7.jpg)

Closes #3539 from zwangsheng/feature/tpcds_white_list.

Closes #3539

74657957 [zwangsheng] fix
26ade916 [Binjie Yang] Update dev/kyuubi-tpcds/src/main/scala/org/apache/kyuubi/tpcds/benchmark/RunBenchmark.scala
11a5b5c9 [Binjie Yang] Update dev/kyuubi-tpcds/src/main/scala/org/apache/kyuubi/tpcds/benchmark/RunBenchmark.scala
780410a3 [zwangsheng] fix
19018932 [zwangsheng] naming
1a4d1a25 [zwangsheng] naming
e1bb0069 [zwangsheng] fix
1be2bcb9 [zwangsheng] white

Lead-authored-by: zwangsheng <2213335496@qq.com>
Co-authored-by: Binjie Yang <52876270+zwangsheng@users.noreply.github.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-09-27 20:08:31 +08:00
zwangsheng
73612b7750
[KYUUBI #3522][FOLLOWUP] Fix benchmark results-dir
### _Why are the changes needed?_

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3523 from zwangsheng/bugfix/bechmark_fix_result_dir.

Closes #3522

4f957960 [zwangsheng] rebase
17de184f [zwangsheng] fix
39361cfc [zwangsheng] fix

Authored-by: zwangsheng <2213335496@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-09-20 11:29:10 +08:00
zwangsheng
9b67a0230f
[KYUUBI #3522] [BUG][TPCDS] Fix benchmark --results-dir
### _Why are the changes needed?_

`r` or `results-dir` should point to results-dir

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3522 from zwangsheng/bugfix/bechmark_fix_result_dir.

Closes #3522

39361cfc [zwangsheng] fix

Authored-by: zwangsheng <2213335496@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-09-20 10:41:17 +08:00
yikf
e87baa7505
[KYUUBI #3443] Bump Scala from 2.12.15 to 2.12.16
### _Why are the changes needed?_

Close https://github.com/apache/incubator-kyuubi/issues/3443, This pr aims to bump scala version from 2.12.15 to 2.12.16

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3467 from Yikf/bump-scala.

Closes #3443

63b6b381 [yikf] Bump Scala from 2.12.15 to 2.12.16

Authored-by: yikf <yikaifei1@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-09-11 17:25:32 +08:00
yongqian
0ae017402d
[KYUUBI #3460] Bump Jackson from 2.13.3 to 2.13.4
### _Why are the changes needed?_

close #3460

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3466 from QianyongY/features/kyuubi-3460.

Closes #3460

d3509a14 [yongqian] fix failed
52020fa2 [yongqian] [KYUUBI apache#3460] Bump Jackson from 2.13.3 to 2.13.4

Authored-by: yongqian <yongqian@trip.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-09-11 14:17:16 +08:00
Cheng Pan
08f6f4e13d
Revert "[KYUUBI #2041] Bump Zookeeper from 3.4.14 to 3.6.3"
This reverts commit 812ea2bdfe.
2022-09-09 16:48:36 +08:00
Cheng Pan
812ea2bdfe
[KYUUBI #2041] Bump Zookeeper from 3.4.14 to 3.6.3
### _Why are the changes needed?_

#2040 adds the test to verify the compatibility for Zookeeper Server 3.4, I will rebase this PR after #2040 get merged.

Close #1941

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #2041 from pan3793/zk.

Closes #2041

d6722f11 [Cheng Pan] license
eb7101a0 [Cheng Pan] fix
d9ce7c29 [Cheng Pan] Bump Zookeeper 3.6.3

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-09-09 12:31:55 +08:00
yikf
3adcebd557
[KYUUBI #3378][SUBTASK] Improve hive-connector module tests
### _Why are the changes needed?_

Fix https://github.com/apache/incubator-kyuubi/issues/3378

This pr aims to improve hive-connector module tests and make CI perform tests for the hive connector

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3379 from Yikf/hive-connector-test.

Closes #3378

72ad050c [yikf] CI test for hive-connector

Authored-by: yikf <yikaifei1@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-09-03 04:44:01 +08:00
odone
9716548380
[KYUUBI #2282] Add lineage records for sql statement execution in Kyuubi engine logs
### _Why are the changes needed?_

Lineage information:
```
col0 -> (table.a, table.b)
col1 -> (table.c, table.a)
```

SQL lineage logger JSON format example.
**SQL:**
```
select a as col0, b as col1 from test_table0
```
**Lineage:**
```
{
   "inputTables": ["default.test_table0"],
   "outputTables": [],
   "columnLineage": [{
      "column": "col0",
      "originalColumns": ["default.test_table0.a"]
   }, {
      "column": "col1",
      "originalColumns": ["default.test_table0.b"]
   }]
}
```

Currently supported column lineage for spark `Command` and `Query` `TreeNode`:

### Query
- `Select`

### Command
- `CreateDataSourceTableAsSelectCommand`
- `CreateHiveTableAsSelectCommand`
- `OptimizedCreateHiveTableAsSelectCommand`
- `CreateTableAsSelect`
- `ReplaceTableAsSelect`
- `InsertIntoDataSourceCommand`
- `InsertIntoHadoopFsRelationCommand`
- `InsertIntoDataSourceDirCommand`
- `InsertIntoHiveDirCommand`
- `InsertIntoHiveTable`

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3185 from iodone/kyuubi-2282.

Closes #2282

002c6d61 [odone] delete spark-sql-engine test for lineage
e1728a79 [odone] update lineage entity schema
de2a3e9a [odone] change kyuubi-spark-listener module to kyuubi-spark-lineage module
9258125e [odone] optimize lineage output
834669ed [odone] delete engine lineage parse
d9c7a3dc [odone] add spark listener to support lineage
4bae8c2f [odone] update for code cleaning
32b3392b [odone] update for review
fe09e478 [odone] add some test

Authored-by: odone <odone.zhang@gmail.com>
Signed-off-by: Kent Yao <yao@apache.org>
2022-08-24 14:11:35 +08:00
yikf
c01be3ae3d
[KYUUBI #3298][FOLLOWUP] Fix CI failure
### _Why are the changes needed?_

This pr is a follow-up, aims to fix CI failure due to https://github.com/apache/incubator-kyuubi/pull/3299#discussion_r953265471

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3319 from Yikf/master.

Closes #3298

d0cba33f [yikf] [KYUUBI #3298][FOLLOWUP] Fix CI failure

Authored-by: yikf <yikaifei1@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-08-24 11:54:36 +08:00