Commit Graph

4318 Commits

Author SHA1 Message Date
Cheng Pan
4b506f4cb9
[KYUUBI #6820] Explicitly disable attach-scaladocs for pure Java modules
# 🔍 Description

```
export JAVA_HOME=/path/of/openjdk-17
build/mvn clean install -DskipTests -Dmaven.scaladoc.skip=false
```

```
[INFO] --- scala-maven-plugin:4.9.2:doc-jar (attach-scaladocs)  kyuubi-server-plugin ---
[INFO] compiler plugin: BasicArtifact(com.github.ghik,silencer-plugin_2.12.20,1.7.19,null)
error: fatal error: object scala in compiler mirror not found.
```

## Types of changes 🔖

- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

Successfully run the build command
```
export JAVA_HOME=/path/of/openjdk-17
build/mvn clean install -DskipTests -Dmaven.scaladoc.skip=false
```

---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6820 from pan3793/scaladoc.

Closes #6820

f5cee3429 [Cheng Pan] Explicitly disable attach-scaladocs for pure Java modules

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2024-11-22 17:12:00 +08:00
Bowen Liang
c391d169fe
[KYUUBI #6813] [BUILD] Replace java.version 1.8 with 8
# 🔍 Description
## Issue References 🔗

This pull request fixes #

## Describe Your Solution 🔧

- fix the compilation error on Zulu JDK8 (Zulu 8.82.0.21-cA-macos-aarch64 with M4 Max chip on MacOS 15.1), when running `build/mvn clean install -pl :kyuubi-util -DskipTests -am` command to build a pure Java module,  by replacing version code sytle from 1.8 to 8.

![image](https://github.com/user-attachments/assets/d4d473b5-8db7-49cf-b7d5-6cc23541496c)

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6813 from bowenliang123/java8.

Closes #6813

1a1e3f3a7 [Bowen Liang] set java.version to 8

Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2024-11-19 17:22:59 +08:00
Bowen Liang
903e333ec0
[KYUUBI #6807] [BUILD] Bump maven download plugin from 1.8.1 to 1.11.3
# 🔍 Description
## Issue References 🔗

## Describe Your Solution 🔧

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6807 from pionCham/bump-maven-plugin-version.

Closes #6807

6582a194c [Bowen Liang] bump to 1.11.3
1505449a6 [chengpeiming] Bump maven plugin download version to 1.11.2

Lead-authored-by: Bowen Liang <bowenliang@apache.org>
Co-authored-by: chengpeiming <chengpeiming@gf.com.cn>
Signed-off-by: Kent Yao <yao@apache.org>
2024-11-19 15:00:39 +08:00
Bowen Liang
27e7254930 [KYUUBI #6773] Bump commons-codec from 1.15 to 1.17.1
# 🔍 Description
## Issue References 🔗

This pull request fixes #

## Describe Your Solution 🔧

- bump commons-codec from 1.15 (Seq 2020) to 1.17.1 (Jul 2024): https://commons.apache.org/proper/commons-codec/changes-report.html

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6773 from bowenliang123/commons-codec-1.17.

Closes #6773

347ef0fe9 [Bowen Liang] update

Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
2024-11-15 11:20:04 +08:00
Bowen Liang
e03207baa2
[KYUUBI #6765] [CI] speed up the script for freeing disk space by batching removals
# 🔍 Description
## Issue References 🔗

This pull request fixes #

## Describe Your Solution 🔧

- batching removals to avoid repeatedly reading and updating the apt-get databases
- 50% speed up the execution time for freeing disk space from 2 minutes or more , to about 1 minutues.

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6765 from bowenliang123/speedup-freedisk.

Closes #6765

dd475c243 [Bowen Liang] update (+1 squashed commit) Squashed commits: [688793f47] update

Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Kent Yao <yao@apache.org>
2024-11-14 18:31:39 +08:00
dnskr
2453efe145
[KYUUBI #6806] [K8S][HELM] Headless service renders ports for enabled protocols only
# 🔍 Description
## Issue References 🔗

Headless service exposes ports for all protocols (enabled and disabled).

## Describe Your Solution 🔧

Add condition to render ports for enabled protocols only.

## Types of changes 🔖

- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

> Note: `thrift-binary` and `rest` protocols are enabled by default

#### Behavior Without This Pull Request ⚰️
```shell
helm template kyuubi charts/kyuubi -s templates/kyuubi-headless-service.yaml
---
# Source: kyuubi/templates/kyuubi-headless-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: kyuubi-headless
  labels:
    helm.sh/chart: kyuubi-0.1.0
    app.kubernetes.io/name: kyuubi
    app.kubernetes.io/instance: kyuubi
    app.kubernetes.io/version: "1.9.2"
    app.kubernetes.io/managed-by: Helm
spec:
  type: ClusterIP
  clusterIP: None
  ports:
    - name: mysql
      port: 3309
      targetPort: 3309
    - name: rest
      port: 10099
      targetPort: 10099
    - name: thrift-binary
      port: 10009
      targetPort: 10009
    - name: thrift-http
      port: 10010
      targetPort: 10010
    - name: prometheus
      port: 10019
      targetPort: prometheus
  selector:
    app.kubernetes.io/name: kyuubi
    app.kubernetes.io/instance: kyuubi
```

#### Behavior With This Pull Request 🎉
```shell
helm template kyuubi charts/kyuubi -s templates/kyuubi-headless-service.yaml
---
# Source: kyuubi/templates/kyuubi-headless-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: kyuubi-headless
  labels:
    helm.sh/chart: kyuubi-0.1.0
    app.kubernetes.io/name: kyuubi
    app.kubernetes.io/instance: kyuubi
    app.kubernetes.io/version: "1.9.2"
    app.kubernetes.io/managed-by: Helm
spec:
  type: ClusterIP
  clusterIP: None
  ports:
    - name: rest
      port: 10099
      targetPort: 10099
    - name: thrift-binary
      port: 10009
      targetPort: 10009
    - name: prometheus
      port: 10019
      targetPort: prometheus
  selector:
    app.kubernetes.io/name: kyuubi
    app.kubernetes.io/instance: kyuubi
```

---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

Closes #6806 from dnskr/headless-service-renders-enabled-protocol-ports.

Closes #6806

9d3f07484 [dnskr] [K8S][HELM] Headless service renders ports for enabled protocols only

Authored-by: dnskr <dnskrv88@gmail.com>
Signed-off-by: Kent Yao <yao@apache.org>
2024-11-14 18:26:22 +08:00
chengpeiming
a4eaacd850
[KYUUBI #6804] Bump Iceberg from 1.6.1 to 1.7.0
# 🔍 Description
## Issue References 🔗

Apache Iceberg 1.7.0 release https://github.com/apache/iceberg/releases/tag/apache-iceberg-1.7.0

## Describe Your Solution 🔧
- Bump Apache Iceberg to 1.7.0
- As Apache Iceberg 1.7.0 drops support for Java 8 and building with Java 11, keep it in 1.6.x for Java 8

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6804 from pionCham/bump-iceberg-version.

Closes #6804

0896ac768 [Bowen Liang] keep iceberg 1.6.1 in playground
eba16ae6c [chengpeiming] Specify the iceberg version in java-8 profile
3b160ddd6 [chengpeiming] Bump iceberg version

Lead-authored-by: chengpeiming <chengpeiming@gf.com.cn>
Co-authored-by: Bowen Liang <bowenliang@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
2024-11-14 18:25:09 +08:00
Bowen Liang
d22ad7fd22
[KYUUBI #6774] Bump log4j from 2.20.0 to 2.24.1
# 🔍 Description
## Issue References 🔗

This pull request fixes #

## Describe Your Solution 🔧

- bump log4j from 2.20.0(Feb 2023) to 2.24.1 (Sep 2024) : https://logging.apache.org/log4j/2.x/release-notes.html#release-notes-2-24-1

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6774 from bowenliang123/log4j-2.24.

Closes #6774

98a89ef8a [Bowen Liang] bump log4j from 2.20.0 to 2.24.1

Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Kent Yao <yao@apache.org>
2024-11-14 18:18:17 +08:00
chengpeiming
ad53eefe4b [KYUUBI #6808] [BUILD]Replace systemProperties with systemPropertyVariables in maven-surefire-plugin
# 🔍 Description
## Issue References 🔗

## Describe Your Solution 🔧
Fix the warning of `maven-surefire-plugin` that show `Parameter 'systemProperties' is deprecated. Use systemPropertyVariables instead.` in the build.

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6808 from pionCham/use-systemPropertyVariables.

Closes #6808

668bfe252 [chengpeiming] Replace  systemProperties with systemPropertyVariables in maven-surefire-plugin
0e74b4725 [chengpeiming] Revert "Replace systemProperties with systemPropertyVariables"
797384a5b [chengpeiming] Replace systemProperties with systemPropertyVariables

Authored-by: chengpeiming <chengpeiming@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
2024-11-13 22:41:23 +08:00
dnskr
c986c47082 [KYUUBI #6805] [K8S][HELM] Update default Kyuubi version to 1.10.0
# 🔍 Description
## Issue References 🔗
Default Kyuubi version in the chart is not up to date with the latest release version.

## Describe Your Solution 🔧

Upgrade default version to the latest 1.10.0 to be up to date with the release version.

## Types of changes 🔖
- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

Check the chart deploys Kyuubi 1.10.0

Install the chart:
```shell
helm install kyuubi charts/kyuubi
```
Check Kyuubi version:
```shell
kubectl exec kyuubi-0 -- cat /opt/kyuubi/RELEASE
Kyuubi 1.10.0 (git revision bfcd1f1) built for
Java 1.8.0_432
Scala 2.12
Flink 1.20.0
Spark 3.5.2
Kyuubi Hadoop 3.3.6
Hive 3.1.3
Build flags:
```

---

# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

Closes #6805 from dnskr/helm-default-kyuubi-version-1.10.0.

Closes #6805

ee36f5788 [dnskr] [K8S][HELM] Update default Kyuubi version to 1.10.0

Authored-by: dnskr <dnskrv88@gmail.com>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
2024-11-13 14:23:10 +08:00
Bowen Liang
dddb03722d [KYUUBI #6778] [BUILD] allow frontend plugin not to inherits Maven's http proxy config
# 🔍 Description
## Issue References 🔗

## Describe Your Solution 🔧

- previously, `pnpm install` and `npm install` are forced to inherits HTTP proxy configs from Maven by the frontend maven plugin (https://github.com/eirslett/frontend-maven-plugin?tab=readme-ov-file#proxy-settings)
- adding a parameter `maven.plugin.frontend.inheritsProxyConfigFromMave` to control whether it inherits the proxy configs with false as default value

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6778 from bowenliang123/frontend-build-proxy.

Closes #6778

c8eb2e4f1 [Bowen Liang] inheritsProxyConfigFromMaven

Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
2024-11-12 15:20:45 +08:00
Bowen Liang
c8b89221b6 [KYUUBI #6786] Skip repeated checks on convert function in TColumnGenerator
# 🔍 Description
## Issue References 🔗

This pull request fixes #

## Describe Your Solution 🔧

- `TColumnGenerator` is used for generating column-based data. This PR skips repeated checks on checking nullability of the convert function in generating single column.

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6786 from bowenliang123/skip-converfunc-check.

Closes #6786

f76f6864a [Bowen Liang] nit
930dfef3a [Bowen Liang] fix
9712274e3 [Bowen Liang] comment
9db16ef6b [Bowen Liang] nit
977d21533 [Bowen Liang] skip repeated checking on convert function

Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
2024-11-12 14:54:59 +08:00
wforget
b4838b40e6 [KYUUBI #6790] Fix engine cannot exit when gracefully stopped
# 🔍 Description
## Issue References 🔗

Fix engine cannot exit when gracefully stopped

This pull request fixes #6790

## Describe Your Solution 🔧

## Types of changes 🔖

- [X] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [X] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6792 from wForget/KYUUBI-6790.

Closes #6790

efe7a2ffa [wforget] [KYUUBI-6790] Fix engine cannot exit when gracefully stopped

Authored-by: wforget <643348094@qq.com>
Signed-off-by: Wang, Fei <fwang12@ebay.com>
2024-11-04 20:05:14 -08:00
senmiaoliu
c9d9433f74 [KYUUBI #6787] Improve the compatibility of queryTimeout in more version clients
# 🔍 Description
## Issue References 🔗

This pull request fixes #2112

## Describe Your Solution 🔧

Similar to #2113, the query-timeout-thread should verify the Thrift protocol version. For protocol versions <= HIVE_CLI_SERVICE_PROTOCOL_V8, it should convert TIMEDOUT_STATE to CANCELED.

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6787 from lsm1/branch-timer-checker-set-cancel.

Closes #6787

9fbe1ac97 [senmiaoliu] add isHive21OrLower method
0c77c6f6f [senmiaoliu] time checker set cancel state

Authored-by: senmiaoliu <senmiaoliu@trip.com>
Signed-off-by: senmiaoliu <senmiaoliu@trip.com>
2024-11-04 19:12:48 +08:00
Bowen Liang
d3520ddbce [KYUUBI #6769] [RELEASE] Bump 1.11.0-SNAPSHOT
# 🔍 Description
## Issue References 🔗

This pull request fixes #

## Describe Your Solution 🔧

Preparing v1.11.0-SNAPSHOT after branch-1.10 cut

```shell
build/mvn versions:set -DgenerateBackupPoms=false -DnewVersion="1.11.0-SNAPSHOT"
(cd kyuubi-server/web-ui && npm version "1.11.0-SNAPSHOT")
```

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6769 from bowenliang123/bump-1.11.

Closes #6769

6db219d28 [Bowen Liang] get latest_branch by sorting version in branch name
465276204 [Bowen Liang] update package.json
81f2865e5 [Bowen Liang] bump

Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
2024-10-23 17:10:56 +08:00
Bowen Liang
8862767827 [KYUUBI #6034] Kyuubi Server HA&ZK get server from serverHosts support more strategy
# 🔍 Description
## Issue References 🔗

This pull request fixes #6034

## Describe Your Solution 🔧
Currently, use beeline to connect kyuubiServer with HA mode, the strategy only support random, this will lead to a high load on the machine. So i make this pr to support choose strategy.
[description]
First, we need know, beeline connect kyuubiServer dependency on kyuubi-hive-jdbc, it is isolated from the kyuubi cluster, so the code only support random choose serverHost from zk node /${namespace}. Because kyuubi-hive-jdbc is a stateless module, only run once, cannot store var about get serverHost from zk node.
[Solution]
This pr, we could implement a interface named ChooseServerStrategy to choose serverHost. I implement two strategy
1. poll: it will create a zk node named ${namespace}-counter, when a beeline client want connect kyuubiServer, the node will increment 1, use this value to take the remainder from serverHosts, like counter % serverHost.size, so we could get a order serverHost
2. random: random get serverHost from serverHosts
3. User Definied Class: implemented the ChooseServerStrategy, then put the jar to beeline-jars, it can use your strategy to choose serverHost

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪
Test the Strategy in my test Cluster
#### Behavior Without This Pull Request ⚰️
![image](https://github.com/apache/kyuubi/assets/51512358/d65b14c1-1b02-4436-8843-27b2e55d27ce)
![image](https://github.com/apache/kyuubi/assets/51512358/0524a30c-c2c3-464e-8453-84f3f1a74fb1)
![image](https://github.com/apache/kyuubi/assets/51512358/12feb93e-b743-4a43-821d-454f3c1af336)

#### Behavior With This Pull Request 🎉

[Use Case]
1. poll: `bin/beeline -u 'jdbc:hive2://xxx:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=kyuubi;zooKeeperStrategy=poll?spark.yarn.queue=root.kylin;spark.app.name=testspark;spark.shuffle.useOldFetchProtocol=true' -n mfw_hadoop --verbose=true --showNestedErrs=true`
2. random: `bin/beeline -u 'jdbc:hive2://xxx:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=kyuubi;zooKeeperStrategy=random?spark.yarn.queue=root.kylin;spark.app.name=testspark;spark.shuffle.useOldFetchProtocol=true' -n mfw_hadoop --verbose=true --showNestedErrs=true` or `bin/beeline -u 'jdbc:hive2://xxx:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=kyuubi?spark.yarn.queue=root.kylin;spark.app.name=testspark;spark.shuffle.useOldFetchProtocol=true' -n mfw_hadoop --verbose=true --showNestedErrs=true`
3. YourStrategy: `bin/beeline -u 'jdbc:hive2://xxx:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=kyuubi;zooKeeperStrategy=xxx.xxx.xxx.XxxChooseServerStrategy?spark.yarn.queue=root.kylin;spark.app.name=testspark;spark.shuffle.useOldFetchProtocol=true' -n mfw_hadoop --verbose=true --showNestedErrs=true`

[Result: The Cluster have two Server (221,233)]
1. poll:
1.1. zkNode: counterValue
![image](https://github.com/apache/kyuubi/assets/51512358/5cbd15f9-bba4-4b23-bbfb-d61ed46f931f)

1.2. result:
![image](https://github.com/apache/kyuubi/assets/51512358/5a867167-8b06-49ed-aa44-b70726f3ae97)
![image](https://github.com/apache/kyuubi/assets/51512358/404b05e8-c828-458c-a9c4-97a323bf6ce7)
![image](https://github.com/apache/kyuubi/assets/51512358/3182e92b-6976-4931-a899-5e0d89cd2ac2)
![image](https://github.com/apache/kyuubi/assets/51512358/a55450ff-49cf-4b4a-9b90-91dd02982aa5)

2. random:
![image](https://github.com/apache/kyuubi/assets/51512358/d65b14c1-1b02-4436-8843-27b2e55d27ce)
![image](https://github.com/apache/kyuubi/assets/51512358/0524a30c-c2c3-464e-8453-84f3f1a74fb1)
![image](https://github.com/apache/kyuubi/assets/51512358/12feb93e-b743-4a43-821d-454f3c1af336)

3. YourStrategy(the test case only get the first serverHost):
![image](https://github.com/apache/kyuubi/assets/51512358/2e6395c2-6496-4516-9cf6-90abc921de7f)
![image](https://github.com/apache/kyuubi/assets/51512358/72975513-48d2-4f41-8a95-95cde0302c5b)
![image](https://github.com/apache/kyuubi/assets/51512358/487951fd-de45-4e1c-861a-94e0e5564e37)

#### Related Unit Tests

There is no Unit Tests.
---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6213 from davidyuan1223/ha_zk_support_more_strategy.

Closes #6034

961d3e989 [Bowen Liang] rename ServerStrategyFactory to ServerSelectStrategyFactory
353f94059 [Bowen Liang] repeat
8822ad471 [Bowen Liang] repeat
619339402 [Bowen Liang] nit
e94f9e909 [Bowen Liang] nit
40f427ae5 [Bowen Liang] rename StrategyFactory to StrategyFactoryServerStrategyFactory
7668f99cc [Bowen Liang] test name
e194ea62f [Bowen Liang] remove ZooKeeperHiveClientException from method signature of chooseServer
265965e5d [Bowen Liang] polling
b39c56700 [Bowen Liang] style
1ab79b494 [Bowen Liang] strategyName
8f8ca28f2 [Bowen Liang] nit
228bf1091 [Bowen Liang] rename parameter zooKeeperStrategy to serverSelectStrategy
125c82358 [Bowen Liang] rename ChooseServerStrategy to ServerSelectStrategy
b4aeb3dbd [Bowen Liang] repeat testing on pollingChooseStrategy
465548005 [davidyuan] update
09a84f1f9 [david yuan] remove the distirbuted lock
93f4a2699 [davidyuan] remove reset
7b0c1b811 [davidyuan] fix var not valid and counter getAndIncrement
c95382a23 [davidyuan] fix var not valid and counter getAndIncrement
9ed2cac85 [david yuan] remove test comment
8eddd7682 [davidyuan] Add Strategy Unit Test Case and fix the polling strategy counter begin with 0
73952f878 [davidyuan] Kyuubi Server HA&ZK get server from serverHosts support more strategy
97b959776 [davidyuan] Kyuubi Server HA&ZK get server from serverHosts support more strategy
ee5a9ad68 [davidyuan] Kyuubi Server HA&ZK get server from serverHosts support more strategy
6a0445357 [davidyuan] Kyuubi Server HA&ZK get server from serverHosts support more strategy
1892f148d [davidyuan] add common method to get session level config
7c0c6058d [yuanfuyuan] fix_4186

Lead-authored-by: Bowen Liang <liangbowen@gf.com.cn>
Co-authored-by: davidyuan <yuanfuyuan@mafengwo.com>
Co-authored-by: davidyuan <davidyuan1223@gmail.com>
Co-authored-by: david yuan <davidyuan1223@gmail.com>
Co-authored-by: yuanfuyuan <1406957364@qq.com>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
2024-10-23 16:57:09 +08:00
Bowen Liang
e446724488 [KYUUBI #6775] Bump Github actions action/cache and actions/dependency-review-action v4
# 🔍 Description
## Issue References 🔗

This pull request fixes #

## Describe Your Solution 🔧

- as titled

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6775 from bowenliang123/gha-cache-v4.

Closes #6775

de7b9fdf0 [Bowen Liang] actions/dependency-review-action@v4
c9e41923a [Bowen Liang] actions/cache@v4

Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
2024-10-23 14:32:16 +08:00
senmiaoliu
f876600c4a [KYUUBI #6772] Fix ProcessBuilder to properly handle Java opts as a list
# 🔍 Description
## Issue References 🔗

## Describe Your Solution 🔧

This PR addresses an issue in the ProcessBuilder class where Java options passed as a single string (e.g., "-Dxxx -Dxxx") do not take effect. The command list must separate these options into individual elements to ensure they are recognized correctly by the Java runtime.

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6772 from lsm1/branch-fix-processBuilder.

Closes #6772

fb6d53234 [senmiaoliu] fix process builder java opts

Authored-by: senmiaoliu <senmiaoliu@trip.com>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
2024-10-23 13:28:58 +08:00
Bowen Liang
1d668ea096 [KYUUBI #6768] [CI] skip explicitly setting check-latest to false
# 🔍 Description
## Issue References 🔗

This pull request fixes #

## Describe Your Solution 🔧

- skip explicitly set the parameter `check-latest` of Github action `actions/setup-java` to false, which is its default value.
- docs: https://github.com/actions/setup-java?tab=readme-ov-file#check-latest

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6768 from bowenliang123/action-java-latest-false.

Closes #6768

a0ecd4688 [Bowen Liang] update

Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
2024-10-22 16:09:35 +08:00
Bowen Liang
e0a80f2650 [KYUUBI #6762] [BUILD] cleanup dockerfile warnings
# 🔍 Description
## Issue References 🔗

This pull request fixes #

## Describe Your Solution 🔧

- fix the dockerfile warning reported by `GitHub Actions
/ Kyuubi Server On Kubernetes Integration Test` :
  - Legacy key/value format with whitespace separator should not be used
LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format
More info: https://docs.docker.com/go/dockerfile/rule/legacy-key-value-format/

  - The 'as' keyword should match the case of the 'from' keyword
FromAsCasing: 'as' and 'FROM' keywords' casing do not match
More info: https://docs.docker.com/go/dockerfile/rule/from-as-casing/

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6762 from bowenliang123/docker-warning.

Closes #6762

d9d7b7465 [Bowen Liang] cleanup dockerfile warning

Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
2024-10-22 13:38:02 +08:00
wforget
7cb6af1fe0 [KYUUBI #6763] [CI] Specify JDK 11 for spark engine on kubernetes IT
# 🔍 Description
## Issue References 🔗

`Spark Engine On Kubernetes Integration Test` always fails after #6739

The default java version of Ubuntu 24.04 is OpenJDK 21: https://discourse.ubuntu.com/t/ubuntu-24-04-lts-noble-numbat-release-notes/39890#openjdk , this is a version higher than that supported by spark 3.5.X.

## Describe Your Solution 🔧

Specify JDK 11 for spark engine on kubernetes it.

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

test ga: https://github.com/wForget/kyuubi/actions/runs/11441231714/job/31828818049

---

# Checklist 📝

- [X] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6763 from wForget/minor.

Closes #6763

aee39339f [wforget] Specify JDK 11 for spark engine on kubernetes it

Authored-by: wforget <643348094@qq.com>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
2024-10-22 13:32:01 +08:00
wankunde
04f443792b [KYUUBI #6754][AUTHZ] Improve the performance of Ranger access requests deduplication
# 🔍 Description
## Issue References 🔗

This pull request fixes #6754

## Describe Your Solution 🔧

Right now in RuleAuthorization we use an ArrayBuffer to collect access requests, which is very slow because each new PrivilegeObject needs to be compared with all access requests.

## Types of changes 🔖

- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

Add benchmark
Before
```sh
Java HotSpot(TM) 64-Bit Server VM 17.0.12+8-LTS-286 on Mac OS X 14.6
Apple M3
Collecting files ranger access request:   Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
50000 files benchmark                            181863         189434         NaN         -0.0 -181863368958.0       1.0X
````

#### Behavior With This Pull Request 🎉

After
```sh
Java HotSpot(TM) 64-Bit Server VM 17.0.12+8-LTS-286 on Mac OS X 14.6
Apple M3
Collecting files ranger access request:   Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
50000 files benchmark                              1281           1310          33         -0.0 -1280563000.0       1.0X
```

#### Related Unit Tests

Exists UT

---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6758 from wankunde/ranger2.

Closes #6754

9d7d1964b [wankunde] [KYUUBI #6754] Improve the performance of ranger access requests
88b9c049b [wankun] Update extensions/spark/kyuubi-spark-authz/src/test/scala/org/apache/spark/sql/RuleAuthorizationBenchmark.scala
20c55fbeb [wankun] Update extensions/spark/kyuubi-spark-authz/pom.xml
f5a3b6ca5 [wankunde] [KYUUBI #6754] Improve the performance of ranger access requests
9793249de [wankunde] [KYUUBI #6754] Improve the performance of ranger access requests
d86b01f9c [wankunde] [KYUUBI #6754] Improve the performance of ranger access requests
b904b491b [wankunde] [KYUUBI #6754] Improve the performance of ranger access requests
aad08a6bb [wankunde] [KYUUBI #6754] Improve the performance of ranger access requests
1374604bc [wankunde] [KYUUBI #6754] Improve the performance of ranger access requests
01e15c149 [wankun] Update extensions/spark/kyuubi-spark-authz/pom.xml
805e8a9c0 [wankun] Update extensions/spark/kyuubi-spark-authz/pom.xml
e19817943 [wankunde] [KYUUBI #6754] Improve the performance of ranger access requests

Lead-authored-by: wankunde <wankunde@163.com>
Co-authored-by: wankun <wankun@apache.org>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
2024-10-21 21:17:51 +08:00
wforget
1e9d68b000 [KYUUBI #6368] Flink engine supports user impersonation
# 🔍 Description
## Issue References 🔗

This pull request fixes #6368

## Describe Your Solution 🔧

Support impersonation mode for flink sql engine.

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [X] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

Test in hadoop-testing env.

Connection:

```
beeline -u "jdbc:hive2://hadoop-master1.orb.local:10009/default;hive.server2.proxy.user=spark;principal=kyuubi/_HOSTTEST.ORG?kyuubi.engine.type=FLINK_SQL;flink.execution.target=yarn-application;kyuubi.engine.share.level=CONNECTION;kyuubi.engine.flink.doAs.enabled=true;"
```

sql:

```
select 1;
```

result:

![image](https://github.com/apache/kyuubi/assets/17894939/4bde3e4e-0dac-4e09-ac7c-a2c3a3607a13)

launch engine command:

```
2024-06-12 03:22:10.242 INFO KyuubiSessionManager-exec-pool: Thread-62 org.apache.kyuubi.engine.EngineRef: Launching engine:
/opt/flink-1.18.1/bin/flink run-application \
	-t yarn-application \
	-Dyarn.ship-files=/opt/flink/opt/flink-sql-client-1.18.1.jar;/opt/flink/opt/flink-sql-gateway-1.18.1.jar;/etc/hive/conf/hive-site.xml \
	-Dyarn.application.name=kyuubi_CONNECTION_FLINK_SQL_spark_6170b9aa-c690-4b50-938f-d59cca9aa2d6 \
	-Dyarn.tags=KYUUBI,6170b9aa-c690-4b50-938f-d59cca9aa2d6 \
	-Dcontainerized.master.env.FLINK_CONF_DIR=. \
	-Dcontainerized.master.env.HIVE_CONF_DIR=. \
	-Dyarn.security.appmaster.delegation.token.services=kyuubi \
	-Dsecurity.delegation.token.provider.HiveServer2.enabled=false \
	-Dsecurity.delegation.token.provider.hbase.enabled=false \
	-Dexecution.target=yarn-application \
	-Dsecurity.module.factory.classes=org.apache.flink.runtime.security.modules.JaasModuleFactory;org.apache.flink.runtime.security.modules.ZookeeperModuleFa
ctory \
	-Dsecurity.delegation.token.provider.hadoopfs.enabled=false \
	-c org.apache.kyuubi.engine.flink.FlinkSQLEngine /opt/apache-kyuubi-1.10.0-SNAPSHOT-bin/externals/engines/flink/kyuubi-flink-sql-engine_2.12-1.10.0-SNAPS
HOT.jar \
	--conf kyuubi.session.user=spark \
	--conf kyuubi.client.ipAddress=172.20.0.5 \
	--conf kyuubi.engine.credentials=SERUUwACJnRocmlmdDovL2hhZG9vcC1tYXN0ZXIxLm9yYi5sb2NhbDo5MDgzRQAFc3BhcmsEaGl2ZShreXV1YmkvaGFkb29wLW1hc3RlcjEub3JiLmxvY2Fs
QFRFU1QuT1JHigGQCneevIoBkC6EIrwWDxSg03pnAB8dA295wh+Dim7Fx4FNxhVISVZFX0RFTEVHQVRJT05fVE9LRU4ADzE3Mi4yMC4wLjU6ODAyMEEABXNwYXJrAChreXV1YmkvaGFkb29wLW1hc3RlcjEub3JiL
mxvY2FsQFRFU1QuT1JHigGQCneekIoBkC6EIpBHHBSket0SQnlXT5EIMN0U2fUKFRIVvBVIREZTX0RFTEVHQVRJT05fVE9LRU4PMTcyLjIwLjAuNTo4MDIwAA== \
	--conf kyuubi.engine.flink.doAs.enabled=true \
	--conf kyuubi.engine.hive.extra.classpath=/opt/hadoop/share/hadoop/client/*:/opt/hadoop/share/hadoop/mapreduce/* \
	--conf kyuubi.engine.share.level=CONNECTION \
	--conf kyuubi.engine.submit.time=1718162530017 \
	--conf kyuubi.engine.type=FLINK_SQL \
	--conf kyuubi.frontend.protocols=THRIFT_BINARY,REST \
	--conf kyuubi.ha.addresses=hadoop-master1.orb.local:2181 \
	--conf kyuubi.ha.engine.ref.id=6170b9aa-c690-4b50-938f-d59cca9aa2d6 \
	--conf kyuubi.ha.namespace=/kyuubi_1.10.0-SNAPSHOT_CONNECTION_FLINK_SQL/spark/6170b9aa-c690-4b50-938f-d59cca9aa2d6 \
	--conf kyuubi.server.ipAddress=172.20.0.5 \
	--conf kyuubi.session.connection.url=hadoop-master1.orb.local:10009 \
	--conf kyuubi.session.engine.startup.waitCompletion=false \
	--conf kyuubi.session.real.user=spark
```

launch engine log:

![image](https://github.com/apache/kyuubi/assets/17894939/590463a8-2858-47a2-8897-0ddfbe3ffdf6)

jobmanager job:

```
2024-06-12 03:22:26,400 INFO  org.apache.flink.runtime.security.token.DefaultDelegationTokenManager [] - Loading delegation token providers
2024-06-12 03:22:26,992 INFO  org.apache.kyuubi.engine.flink.security.token.KyuubiDelegationTokenProvider [] - Renew delegation token with engine credentials: SERUUwACJnRocmlmdDovL2hhZG9vcC1tYXN0ZXIxLm9yYi5sb2NhbDo5MDgzRQAFc3BhcmsEaGl2ZShreXV1YmkvaGFkb29wLW1hc3RlcjEub3JiLmxvY2FsQFRFU1QuT1JHigGQCneevIoBkC6EIrwWDxSg03pnAB8dA295wh+Dim7Fx4FNxhVISVZFX0RFTEVHQVRJT05fVE9LRU4ADzE3Mi4yMC4wLjU6ODAyMEEABXNwYXJrAChreXV1YmkvaGFkb29wLW1hc3RlcjEub3JiLmxvY2FsQFRFU1QuT1JHigGQCneekIoBkC6EIpBHHBSket0SQnlXT5EIMN0U2fUKFRIVvBVIREZTX0RFTEVHQVRJT05fVE9LRU4PMTcyLjIwLjAuNTo4MDIwAA==
2024-06-12 03:22:27,100 INFO  org.apache.kyuubi.engine.flink.FlinkEngineUtils              [] - Add new unknown token Kind: HIVE_DELEGATION_TOKEN, Service: , Ident: 00 05 73 70 61 72 6b 04 68 69 76 65 28 6b 79 75 75 62 69 2f 68 61 64 6f 6f 70 2d 6d 61 73 74 65 72 31 2e 6f 72 62 2e 6c 6f 63 61 6c 40 54 45 53 54 2e 4f 52 47 8a 01 90 0a 77 9e bc 8a 01 90 2e 84 22 bc 16 0f
2024-06-12 03:22:27,104 WARN  org.apache.kyuubi.engine.flink.FlinkEngineUtils              [] - Ignore token with earlier issue date: Kind: HDFS_DELEGATION_TOKEN, Service: 172.20.0.5:8020, Ident: (token for spark: HDFS_DELEGATION_TOKEN owner=spark, renewer=, realUser=kyuubi/hadoop-master1.orb.localTEST.ORG, issueDate=1718162529936, maxDate=1718767329936, sequenceNumber=71, masterKeyId=28)
2024-06-12 03:22:27,104 INFO  org.apache.kyuubi.engine.flink.FlinkEngineUtils              [] - Update delegation tokens. The number of tokens sent by the server is 2. The actual number of updated tokens is 1.
......
4-06-12 03:22:29,414 INFO  org.apache.flink.runtime.security.token.DefaultDelegationTokenManager [] - Starting tokens update task
2024-06-12 03:22:29,415 INFO  org.apache.flink.runtime.security.token.DelegationTokenReceiverRepository [] - New delegation tokens arrived, sending them to receivers
2024-06-12 03:22:29,422 INFO  org.apache.kyuubi.engine.flink.security.token.KyuubiDelegationTokenReceiver [] - Updating delegation tokens for current user
2024-06-12 03:22:29,422 INFO  org.apache.kyuubi.engine.flink.security.token.KyuubiDelegationTokenReceiver [] - Token Service: Identifier:[10, 13, 10, 9, 8, 10, 16, -78, -36, -49, -17, -5, 49, 16, 1, 16, -100, -112, -60, -127, -8, -1, -1, -1, -1, 1]
2024-06-12 03:22:29,422 INFO  org.apache.kyuubi.engine.flink.security.token.KyuubiDelegationTokenReceiver [] - Token Service: Identifier:[0, 5, 115, 112, 97, 114, 107, 4, 104, 105, 118, 101, 40, 107, 121, 117, 117, 98, 105, 47, 104, 97, 100, 111, 111, 112, 45, 109, 97, 115, 116, 101, 114, 49, 46, 111, 114, 98, 46, 108, 111, 99, 97, 108, 64, 84, 69, 83, 84, 46, 79, 82, 71, -118, 1, -112, 10, 119, -98, -68, -118, 1, -112, 46, -124, 34, -68, 22, 15]
2024-06-12 03:22:29,422 INFO  org.apache.kyuubi.engine.flink.security.token.KyuubiDelegationTokenReceiver [] - Token Service:172.20.0.5:8020 Identifier:[0, 5, 115, 112, 97, 114, 107, 0, 40, 107, 121, 117, 117, 98, 105, 47, 104, 97, 100, 111, 111, 112, 45, 109, 97, 115, 116, 101, 114, 49, 46, 111, 114, 98, 46, 108, 111, 99, 97, 108, 64, 84, 69, 83, 84, 46, 79, 82, 71, -118, 1, -112, 10, 119, -98, -112, -118, 1, -112, 46, -124, 34, -112, 71, 28]
2024-06-12 03:22:29,422 INFO  org.apache.kyuubi.engine.flink.security.token.KyuubiDelegationTokenReceiver [] - Updated delegation tokens for current user successfully

```

taskmanager log:

```
2024-06-12 03:45:06,622 INFO  org.apache.flink.runtime.taskexecutor.TaskExecutor           [] - Receive initial delegation tokens from resource manager
2024-06-12 03:45:06,627 INFO  org.apache.flink.runtime.security.token.DelegationTokenReceiverRepository [] - New delegation tokens arrived, sending them to receivers
2024-06-12 03:45:06,628 INFO  org.apache.kyuubi.engine.flink.security.token.KyuubiDelegationTokenReceiver [] - Updating delegation tokens for current user
2024-06-12 03:45:06,629 INFO  org.apache.kyuubi.engine.flink.security.token.KyuubiDelegationTokenReceiver [] - Token Service: Identifier:[10, 13, 10, 9, 8, 10, 16, -78, -36, -49, -17, -5, 49, 16, 1, 16, -100, -112, -60, -127, -8, -1, -1, -1, -1, 1]
2024-06-12 03:45:06,630 INFO  org.apache.kyuubi.engine.flink.security.token.KyuubiDelegationTokenReceiver [] - Token Service: Identifier:[0, 5, 115, 112, 97, 114, 107, 4, 104, 105, 118, 101, 40, 107, 121, 117, 117, 98, 105, 47, 104, 97, 100, 111, 111, 112, 45, 109, 97, 115, 116, 101, 114, 49, 46, 111, 114, 98, 46, 108, 111, 99, 97, 108, 64, 84, 69, 83, 84, 46, 79, 82, 71, -118, 1, -112, 10, 119, -98, -68, -118, 1, -112, 46, -124, 34, -68, 22, 15]
2024-06-12 03:45:06,630 INFO  org.apache.kyuubi.engine.flink.security.token.KyuubiDelegationTokenReceiver [] - Token Service:172.20.0.5:8020 Identifier:[0, 5, 115, 112, 97, 114, 107, 0, 40, 107, 121, 117, 117, 98, 105, 47, 104, 97, 100, 111, 111, 112, 45, 109, 97, 115, 116, 101, 114, 49, 46, 111, 114, 98, 46, 108, 111, 99, 97, 108, 64, 84, 69, 83, 84, 46, 79, 82, 71, -118, 1, -112, 10, 119, -98, -112, -118, 1, -112, 46, -124, 34, -112, 71, 28]
2024-06-12 03:45:06,636 INFO  org.apache.kyuubi.engine.flink.security.token.KyuubiDelegationTokenReceiver [] - Updated delegation tokens for current user successfully
2024-06-12 03:45:06,636 INFO  org.apache.flink.runtime.security.token.DelegationTokenReceiverRepository [] - Delegation tokens sent to receivers
```

#### Related Unit Tests

---

# Checklist 📝

- [X] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6383 from wForget/KYUUBI-6368.

Closes #6368

47df43ef0 [wforget] remove doAsEnabled
984b96c74 [wforget] update settings.md
c7f8d474e [wforget] make generateTokenFile conf to internal
8632176b1 [wforget] address comments
2ec270e8a [wforget] licenses
ed0e22f4e [wforget] separate kyuubi-flink-token-provider module
b66b855b6 [wforget] address comment
d4fc2bd1d [wforget] fix
1a3dc4643 [wforget] fix style
825e2a7a0 [wforget] address comments
a679ba1c2 [wforget] revert remove renewer
cdd499b95 [wforget] fix and comment
19caec6c0 [wforget] pass token to submit process
b2991d419 [wforget] fix
7c3bdde1b [wforget] remove security.delegation.tokens.enabled check
8987c9176 [wforget] fix
5bd8cfe7c [wforget] fix
08992642d [wforget] Implement KyuubiDelegationToken Provider/Receiver
fa16d7def [wforget] enable delegation token manager
e50db7497 [wforget] [KYUUBI #6368] Support impersonation mode for flink sql engine

Authored-by: wforget <643348094@qq.com>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
2024-10-21 17:32:39 +08:00
Bowen Liang
fb65a12936 [KYUUBI #6756] [REST] Check max file size of uploaded resource and extra resources in batch creation
# 🔍 Description
## Issue References 🔗

This pull request fixes #

## Describe Your Solution 🔧

Check the uploaded resource files when creating batch via REST API
- add config `kyuubi.batch.resource.file.max.size` for resource file's max size in bytes
- add config `kyuubi.batch.extra.resource.file.max.size` for each extra resource file's max size in bytes

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6756 from bowenliang123/resource-maxsize.

Closes #6756

5c409c425 [Bowen Liang] nit
4b16bcfc4 [Bowen Liang] nit
743920d25 [Bowen Liang] check resource file size max size

Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
2024-10-21 16:04:33 +08:00
Bowen Liang
fcf0405397 [KYUUBI #6741] [CI] update GHA actions/upload-artifact from v3 to v4
# 🔍 Description
## Issue References 🔗

This pull request fixes #

## Describe Your Solution 🔧

- as titled

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6741 from bowenliang123/gha-upload-artifact-v4.

Closes #6741

83a0ee90e [Bowen Liang] update GHA actions/upload-artifact from v3 to v4

Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
2024-10-21 15:48:19 +08:00
Bowen Liang
a3edd24188 [KYUUBI #6739] [CI] Bump runner image from Ubuntu 22.04 to 24.04
# 🔍 Description
## Issue References 🔗

This pull request fixes #

## Describe Your Solution 🔧

- The Ubuntu 24.04 image for Actions is now generally available since Sep 2024 : https://github.blog/changelog/2024-09-25-actions-new-images-and-ubuntu-latest-changes/

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6739 from bowenliang123/ubuntu-2404.

Closes #6739

2d120d1f6 [Bowen Liang] bump ci image from ubuntu 22.04 to 24.04

Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
2024-10-21 15:45:53 +08:00
Bowen Liang
e663cc35d4 [KYUUBI #6687] [BUILD] Use MAVEN_ARGS for passing Maven CLI arguments
# 🔍 Description
## Issue References 🔗

This pull request fixes #

## Describe Your Solution 🔧

- inspired by comments from #6664 : https://github.com/apache/kyuubi/pull/6664#issuecomment-2344351119

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6687 from bowenliang123/mvn-args.

Closes #6687

6a9170b60 [Cheng Pan] Update .github/workflows/master.yml
3573131d8 [liangbowen] export MAVEN_ARGS
36ced0532 [liangbowen] update
a93fa28ee [liangbowen] MAVEN_ARGS

Lead-authored-by: Bowen Liang <liangbowen@gf.com.cn>
Co-authored-by: liangbowen <liangbowen@gf.com.cn>
Co-authored-by: Cheng Pan <pan3793@gmail.com>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
2024-10-21 10:18:29 +08:00
wforget
13c84136f6 [KYUUBI #6757] Pass build args to extra spark engine build command
# 🔍 Description
## Issue References 🔗

This pull request fixes #

## Describe Your Solution 🔧

Pass build args to extra spark engine build command

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [X] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6757 from wForget/minor2.

Closes #6757

5b373d2fa [wforget] Pass build args to extra spark engine build command

Authored-by: wforget <643348094@qq.com>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
2024-10-21 10:02:54 +08:00
Bowen Liang
0feacf1776
[KYUUBI #6740] [CI] update GHA docker/build-push-action from v5 to v6 to support build summary
# 🔍 Description
## Issue References 🔗

This pull request fixes #

## Describe Your Solution 🔧

- update GHA docker/build-push-action from v5 to v6 to support build mmary
https://docs.docker.com/build/ci/github-actions/build-summary/

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6740 from bowenliang123/docker-push-v6.

Closes #6740

674cc15fc [Bowen Liang] update GHA docker/build-push-action from v5 to v6 to support build summary

Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2024-10-18 21:32:16 +08:00
wforget
e99a737e63
[KYUUBI #6755] Upgrade default flink version to 1.20.0
# 🔍 Description

Upgrade default flink version to 1.20.0

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [X] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6755 from wForget/minor.

Closes #6755

e968502a6 [wforget] comment
c0094c760 [wforget] comment
b74a0bb75 [wforget] comment
1aea2216c [wforget] Upgrade default flink version to 1.20.0

Authored-by: wforget <643348094@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2024-10-18 21:26:01 +08:00
wforget
535c4f90bc
[KYUUBI #6753] Set hadoop fs delegation token renewer to empty
# 🔍 Description
## Issue References 🔗

Allow delegation tokens to be used and renewed by yarn resourcemanager. (used in proxy user mode of flink engine, address https://github.com/apache/kyuubi/pull/6383#discussion_r1635768060)

## Describe Your Solution 🔧

Set hadoop fs delegation token renewer to empty.

## Types of changes 🔖

- [X] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [X] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6753 from wForget/renewer.

Closes #6753

f2e1f0aa1 [wforget] Set hadoop fs delegation token renewer to empty

Authored-by: wforget <643348094@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2024-10-18 10:56:33 +08:00
Bowen Liang
f8606f4c24 [KYUUBI #6752] [DOC] Bump doc build requirements
# 🔍 Description
## Issue References 🔗

This pull request fixes #

## Describe Your Solution 🔧

- build python dependencies for docs building to latest versions
- no display or behaviour changes
![image](https://github.com/user-attachments/assets/333174af-46f8-4b9d-8886-8140a9f10d59)

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6752 from bowenliang123/doc-req.

Closes #6752

ffd8782bd [Bowen Liang] update
c328c7584 [Bowen Liang] bump doc build requirements

Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
2024-10-18 10:39:02 +08:00
Wang, Fei
cefb98d27b [KYUUBI #6750] [REST] Using ForbiddenException instead of NotAllowedException
# 🔍 Description
## Issue References 🔗

Seems NotAllowedException is used for method not allowed, and currently, we use false constructor, the error message we expected would not be return to client end.

It only told:
```
{"message":"HTTP 405 Method Not Allowed"}
```
Because the message we used to build the NotAllowedException was treated as `allowed` method, not as `message`.

![image](https://github.com/user-attachments/assets/3199f20c-6148-4e6a-9183-7a0843913d8d)

## Describe Your Solution 🔧

We should use the ForbidenException instead, and then the error message we excepted can be visible in client end.

85dd5a52ef/kyuubi-server/src/main/scala/org/apache/kyuubi/server/api/api.scala (L47-L51)

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

<img width="913" alt="image" src="https://github.com/user-attachments/assets/6c4e836d-a47a-485d-85a3-fd3a35a9e425">

---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6750 from turboFei/not_allowed_exception.

Closes #6750

4dd6fc18c [Wang, Fei] Using ForbiddenException instead of NotAllowedException

Authored-by: Wang, Fei <fwang12@ebay.com>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
2024-10-18 09:20:52 +08:00
Bowen Liang
928e3d243f [KYUUBI #6731] [REST] Check all required extra resource files uploaded in creating batch request
# 🔍 Description
## Issue References 🔗

This pull request fixes #

## Describe Your Solution 🔧

- check all the required extra resource files are uploaded in POST multi-part request as expected, when creating batch with REST Batch API

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6731 from bowenliang123/extra-resource-check.

Closes #6731

116a47ea5 [Bowen Liang] update
cd4433a8c [Bowen Liang] update
4852b1569 [Bowen Liang] update
5bb2955e8 [Bowen Liang] update
1696e7328 [Bowen Liang] update
911a9c195 [Bowen Liang] update
042e42d23 [Bowen Liang] update
56dc7fb8a [Bowen Liang] update

Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
2024-10-18 08:44:13 +08:00
Bowen Liang
85dd5a52ef [KYUUBI #6742] Bump Jetty from 9.4.54 to 9.4.56
# 🔍 Description
## Issue References 🔗

This pull request fixes #

## Describe Your Solution 🔧

 - to fix CVE-2024-8184 reported in https://github.com/apache/kyuubi/security/dependabot/75

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6742 from bowenliang123/jetty-9.4.56.

Closes #6742

1e8fee555 [Bowen Liang] update
96db527d6 [Bowen Liang] bumpu jetty to 9.4.56

Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
2024-10-17 21:33:03 +08:00
Bowen Liang
ad07ae9398 [KYUUBI #6743] Bump protobuf from 3.25.4 to 3.25.5
# 🔍 Description
## Issue References 🔗

This pull request fixes #

## Describe Your Solution 🔧

- to fix CVE-2024-8184 reported in https://github.com/apache/kyuubi/security/dependabot/72

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6743 from bowenliang123/protobuf-3.25.5.

Closes #6743

a9a574041 [Bowen Liang] bump protobuf from 3.25.4 to 3.25.5

Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
2024-10-17 21:32:25 +08:00
Bowen Liang
6d4c9a6d79 [KYUUBI #6744] Bump vite from 4.5.3 to 4.5.5
# 🔍 Description
## Issue References 🔗

This pull request fixes #

## Describe Your Solution 🔧

- to fix CVE-2024-45812, CVE-2024-45811, CVE-2024-45812, CVE-2024-45811 and CVE-2024-45811 reported by dependent bot security alerts

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6744 from bowenliang123/vite-4.5.4.

Closes #6744

271db1f5c [Bowen Liang] update

Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
2024-10-17 21:31:12 +08:00
Bowen Liang
b9beec2630 [KYUUBI #6745] Bump testcontainers-scala from 0.41.0 to 0.41.4
# 🔍 Description
## Issue References 🔗

This pull request fixes #

## Describe Your Solution 🔧

Bump testcontainers-scala from 0.41.0 (released in Aug 2023) to 0.41.4 (released in Jun 2024)

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6745 from bowenliang123/testcontainers-scala-0.41.4.

Closes #6745

06551cb8d [Bowen Liang] bump testcontainers-scala from 0.41.0 to 0.41.4

Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
2024-10-17 21:30:10 +08:00
Bowen Liang
45eebda431 [KYUUBI #6746] Bump commons-lang3 from 3.13.0 to 3.17.0
# 🔍 Description
## Issue References 🔗

This pull request fixes #

## Describe Your Solution 🔧

- Bump commons-lang3 from 3.13.0 ( 2023-07-23 ) to 3.17.0 ( 2024-08-24 ), https://commons.apache.org/proper/commons-lang/changes-report.html#a3.17.0

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6746 from bowenliang123/commonslang3-3.17.

Closes #6746

d72467bfb [Bowen Liang] bump commons-lang3 from 3.13.0 to 3.17.0

Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
2024-10-17 21:29:05 +08:00
Bowen Liang
a1a5a9d27f [KYUUBI #6747] Bump Guava from 32.1.3 to 33.3.1
# 🔍 Description
## Issue References 🔗

This pull request fixes #

## Describe Your Solution 🔧

- Bump guava from 32.1.3 (Oct 11, 2023) to 33.3.1 (Sep 23, 2024 ) : https://github.com/google/guava/releases/tag/v33.3.1

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6747 from bowenliang123/guava-33.3.1.

Closes #6747

39abf1b23 [Bowen Liang] bump guava from 32.1.3 to 33.3.1

Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
2024-10-17 21:28:01 +08:00
madlnu
04468a60c9 [KYUUBI #6737] Bump vertx-grpc and vertx-core from 4.5.1 to 4.5.3
# 🔍 Description

## Describe Your Solution 🔧

Bump vertx-core from 4.5.1 to 4.5.3 to fix CVE-2024-1300 and CVE-2024-1023

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪
Build and ran locally

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6737 from Madhukar525722/vertx.

Closes #6737

7c758e082 [madlnu] Bump vertx-grpc from 4.5.1 to 4.5.3

Authored-by: madlnu <madlnu@visa.com>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
2024-10-17 15:14:53 +08:00
dependabot[bot]
c6212d0657
⬆️ Bump semver in /kyuubi-server/web-ui (#6736) 2024-10-17 05:06:51 +00:00
dependabot[bot]
be66b494d8
⬆️ Bump axios from 1.6.0 to 1.7.4 in /kyuubi-server/web-ui (#6617) 2024-10-16 06:23:34 +00:00
Bowen Liang
4f5799d2b2 [KYUUBI #6728] [DOC] update Authz plugin docs of build command with -am option
# 🔍 Description
## Issue References 🔗

This pull request fixes #

## Describe Your Solution 🔧

- as titled
- add Spark 3.4 and 3.5 to the supported Spark list

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6728 from bowenliang123/doc-authz-build-am.

Closes #6728

f8254bc5c [Bowen Liang] doc

Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
2024-10-16 13:31:14 +08:00
dependabot[bot]
d4b348e1cf
⬆️ Bump get-func-name in /kyuubi-server/web-ui (#6735) 2024-10-16 05:29:58 +00:00
Bowen Liang
0d3389c6fb [KYUUBI #6734] [DOC] add authentication example in REST API docs
# 🔍 Description
## Issue References 🔗

This pull request fixes #

## Describe Your Solution 🔧

-  add authentication example in REST API docs

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6734 from bowenliang123/rest-doc-auth.

Closes #6734

f9ac9446d [Cheng Pan] Update docs/client/rest/rest_api.md
528e55e79 [Bowen Liang] update doc
371af8806 [Bowen Liang] update doc
e64a08245 [Bowen Liang] update doc
341e7e010 [Bowen Liang] update doc

Lead-authored-by: Bowen Liang <liangbowen@gf.com.cn>
Co-authored-by: Cheng Pan <pan3793@gmail.com>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
2024-10-16 13:12:00 +08:00
dependabot[bot]
5e49c25e7c
⬆️ Bump micromatch from 4.0.5 to 4.0.8 in /kyuubi-server/web-ui (#6729) 2024-10-16 03:02:01 +00:00
Bowen Liang
daeff6bfd8 [KYUUBI #6733] Bump SQLlite from 3.42 to 3.46
# 🔍 Description
## Issue References 🔗

This pull request fixes #

## Describe Your Solution 🔧

- bump SQLlite from 3.42.0 (May 2023) to 3.46 (May 2024)
- SQLlite changelog: https://sqlite.org/changes.html

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6733 from bowenliang123/sqllite-346.

Closes #6733

a86bea232 [Bowen Liang] update

Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
2024-10-16 11:00:39 +08:00
taylor.fan
851fb5ae5c
[KYUUBI #6704] Disable periodic gc if set interval to 0
# 🔍 Description
## Issue References 🔗

This pull request fixes https://github.com/apache/kyuubi/issues/6704

## Describe Your Solution 🔧

if periodic gc is set to 0, there is no need to perform an explicit gc.

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [x] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6725 from taylor12805/master.

Closes #6704

a52ddda62 [Bowen Liang] update doc
b84a32f35 [Bowen Liang] make periodic gc thead pool lazy
2d4bd7c05 [Bowen Liang] update doc in spark style
3e04604b0 [taylor.fan] [KYUUBI #6704] disable periodic gc if set interval to 0
bf20b134b [taylor.fan] [KYUUBI #6704] disable periodic gc if set interval to 0
c2b7c3078 [taylor.fan] [KYUUBI #6704] disable periodic gc if set interval to 0
6182075fc [taylor.fan] [KYUUBI #6704] disable periodic gc if set interval to 0
52b1c078b [taylor.fan] [KYUUBI #6704] disable periodic gc if set interval to 0
ccf19cf24 [taylor.fan] [KYUUBI #6704] disable periodic gc if set interval to 0
affd67c88 [taylor.fan] [KYUUBI #6704] disable periodic gc if set interval to 0
d4ee164d1 [taylor.fan] disable periodic gc if set interval to 0

Lead-authored-by: taylor.fan <taylor.fan@vipshop.com>
Co-authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2024-10-16 10:58:17 +08:00
wforget
da2401c171 [KYUUBI #6688] [SPARK] Avoid trigger execution when getting result schema
# 🔍 Description
## Issue References 🔗

`DataFrame.isEmpty` may trigger execution again, we should avoid it.

## Describe Your Solution 🔧

## Types of changes 🔖

- [X] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [X] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6688 from wForget/planonly_schema.

Closes #6688

265f0ec26 [wforget] fix style
d71cc4aa9 [wforget] refactor resultSchema for spark operation
0c36b3d25 [wforget] Avoid trigger execution when getting result schema

Authored-by: wforget <643348094@qq.com>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
2024-10-16 10:36:45 +08:00