### Why are the changes needed?
We will split the sql by `?` when we use `KyuubiPreparedStatement`. But there exist corner case when ? exist in backticks.
For example, below sql contains `?`, but we shouldn't split it by `?`.
```sql
SELECT `(ds|hr)?+.+` FROM sales
```
More details can find at https://hive.apache.org/docs/latest/languagemanual-select_27362043/#regex-column-specification
Hive upstream fix - HIVE-29060
### How was this patch tested?
UT.
### Was this patch authored or co-authored using generative AI tooling?
NO.
Closes#7125 from ruanwenjun/dev_wenjun_fix7109.
Closes#7109
7140980fd [ruanwenjun] [KYUUBI #7109] Ignore the ? in backticks
Lead-authored-by: Wenjun Ruan <wenjun@apache.org>
Co-authored-by: ruanwenjun <zyb@wenjuns-MacBook-Pro-2.local>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### Why are the changes needed?
If `sslTrustStore` is not provided `org.apache.hadoop.conf.Configuration` class existence becomes a hard dependency.
This makes jdbc client too complex to configure: extra Hadoop jars should be provided.
`hadoopCredentialProviderAvailable` variable is useless in the previous implementation logic because it's always `true` or the code is not reachable.
<img width="898" alt="Screenshot 2025-05-09 at 13 05 12" src="https://github.com/user-attachments/assets/6d202555-38c6-40d2-accb-eb78a3d4184e" />
### How was this patch tested?
Build jar and used it to connect from DataGrip.
<img width="595" alt="Screenshot 2025-05-09 at 13 01 29" src="https://github.com/user-attachments/assets/c6e4d904-a3dd-4d3f-9bdd-8bb47ed1e834" />
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes#7058 from Khrol/master.
Closes#7051
b594757a0 [Igor Khrol] JDBC driver: allow usage without sslTrustStore
Authored-by: Igor Khrol <khroliz@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### Why are the changes needed?
It is missed in #6828733d4f0901/jdbc/src/java/org/apache/hive/jdbc/HiveBaseResultSet.java (L151-L159)
### How was this patch tested?
Minor change.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes#7038 from turboFei/fixNPE.
Closes#6828
2785e97be [Wang, Fei] Fix NPE
Authored-by: Wang, Fei <fwang12@ebay.com>
Signed-off-by: Wang, Fei <fwang12@ebay.com>
### Why are the changes needed?
Backport https://github.com/apache/hive/pull/3749
It is not possible to create SSL connection with Kerberos authentication when the server certificate is not issued to the canonical host name but to an alternative domain name.
See details about the exception and steps for reproducing in the [HIVE-26723](https://issues.apache.org/jira/browse/HIVE-26723)
Hive JDBC client validates the host name by its canonical name by default. This behaviour leads to SSLHandshakeExcpetion when trying to connect using alias name with Kerberos authentication. To solve this issue a new connection property is introduced to be able disabling canonical host name check: enableCanonicalHostnameCheck having default value true.
When the property is not given in connection string (or its value is true) then the original behaviour is applied i.e. checking canonical host name.
### How was this patch tested?
There are no new unit tests because the fix is in the HiveConnection constructor which contains lot of logic inside and also builds new SSL connections.
IMO it would have been far too much effort to mock the whole environment for creating unit tests against this tiny change. :(
There wasn't any already existing test against HiveConnection that could be extended with this new feature/bugfix. It is misleading that there is a class having name TestHiveConnection but there is no any tests that would test the class HiveConnection itself.
BTW It was tested manually: after this fix when the steps in JIRA are executed again using the new JARs then the SSL connection is created successfully, and I was able to execute queries.
### Does this PR introduce any user-facing change?
A new JDBC connection URL property has been introduced: enableCanonicalHostnameCheck to be able to turn off the canonical host name checking. Its default value is true so if it is not set the canonical host name is checked when building up the SSL connection.
To turn off the canonical host name checking just add this property to the connection string, i.e:
```
./beeline -u "jdbc:hive2://hs2.subdomain.example.com:443/default;transportMode=http;httpPath=cliservice;socketTimeout=60;ssl=true;retries=1;principal=myhiveprincipal/mydomain.example.com;enableCanonicalHostnameCheck=false;"
```
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes#7009 from turboFei/kerberos_can.
Closes#7009
40cd48814 [Wang, Fei] Backport HIVE-26723: Configurable canonical name checking.
Authored-by: Wang, Fei <fwang12@ebay.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### Why are the changes needed?
Backport https://github.com/apache/hive/pull/4823
We need to setup production tunnel because we can't connect to production environment directly:
```
sh -fN -o ServerAliveInterval=60 -o ServerAliveCountMax=3 -L 127.0.0.1:10001:hiveserver2.prod.company.com:10001 bastion.company.com
JDBC url: jdbc:hive2://127.0.0.1:10001/default;ssl=true
```
But it will throw exception after [HIVE-15025](https://issues.apache.org/jira/browse/HIVE-15025):
```
Exception in thread "main" java.sql.SQLException: Could not open client transport with JDBC Uri: jdbc:hive2://localhost:10001/default;ssl=true: javax.net.ssl.SSLHandshakeException: No subject alternative DNS name matching localhost found.
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:224)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:247)
at org.apache.spark.sql.TestJDBC$.main(TestJDBC.scala:47)
at org.apache.spark.sql.TestJDBC.main(TestJDBC.scala)
Caused by: org.apache.hive.org.apache.thrift.transport.TTransportException: javax.net.ssl.SSLHandshakeException: No subject alternative DNS name matching localhost found.
at org.apache.hive.org.apache.thrift.transport.TIOStreamTransport.flush(TIOStreamTransport.java:161)
at org.apache.hive.org.apache.thrift.transport.TSaslTransport.sendSaslMessage(TSaslTransport.java:166)
at org.apache.hive.org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:100)
at org.apache.hive.org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271)
at org.apache.hive.org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:311)
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:196)
... 5 more
```
This PR disables ssl hostname verification for 127.0.0.1 to workaround this issue.
### How was this patch tested?
Manual test.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes#7008 from turboFei/ssl.
Closes#7008
6ae1b7b82 [Wang, Fei] Backport HIVE-27817: Disable ssl hostname verification for 127.0.0.1
Authored-by: Wang, Fei <fwang12@ebay.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### Why are the changes needed?
add Utils.CLIENT_IP_ADDRESS to cache local ipAddress
closes#6918
### How was this patch tested?
minor fix
### Was this patch authored or co-authored using generative AI tooling?
No
Closes#6919 from wForget/KYUUBI-6918.
Closes#6918
5b0efe722 [Cheng Pan] Update kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/Utils.java
b8660c83b [wforget] address comment
835dfb8ef [wforget] address comment
29dd13a2e [wforget] fix style
843b29fe8 [wforget] [KYUUBI #6918] Cache client ipAddress in kyuubi jdbc connection
Lead-authored-by: wforget <643348094@qq.com>
Co-authored-by: Cheng Pan <pan3793@gmail.com>
Signed-off-by: wforget <643348094@qq.com>
### Why are the changes needed?
Backport 2b0e424daa
The commit does not have a Hive ticket.
### How was this patch tested?
Pass GHA to ensure it breaks nothing, and verify locally that the reported error message does contain the data when failing to parse the illegal date.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes#6828 from pan3793/hive-jdbc-2b0e424.
Closes#6828
b6e03a661 [Cheng Pan] KyuubiSQLException
63861af99 [Cheng Pan] Backport Hive 2b0e424 - Clean up and improve error message for KyuubiBaseResultSet
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
This pull request fixes #
## Describe Your Solution 🔧
Preparing v1.11.0-SNAPSHOT after branch-1.10 cut
```shell
build/mvn versions:set -DgenerateBackupPoms=false -DnewVersion="1.11.0-SNAPSHOT"
(cd kyuubi-server/web-ui && npm version "1.11.0-SNAPSHOT")
```
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6769 from bowenliang123/bump-1.11.
Closes#6769
6db219d28 [Bowen Liang] get latest_branch by sorting version in branch name
465276204 [Bowen Liang] update package.json
81f2865e5 [Bowen Liang] bump
Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
# 🔍 Description
## Issue References 🔗
This pull request fixes#6034
## Describe Your Solution 🔧
Currently, use beeline to connect kyuubiServer with HA mode, the strategy only support random, this will lead to a high load on the machine. So i make this pr to support choose strategy.
[description]
First, we need know, beeline connect kyuubiServer dependency on kyuubi-hive-jdbc, it is isolated from the kyuubi cluster, so the code only support random choose serverHost from zk node /${namespace}. Because kyuubi-hive-jdbc is a stateless module, only run once, cannot store var about get serverHost from zk node.
[Solution]
This pr, we could implement a interface named ChooseServerStrategy to choose serverHost. I implement two strategy
1. poll: it will create a zk node named ${namespace}-counter, when a beeline client want connect kyuubiServer, the node will increment 1, use this value to take the remainder from serverHosts, like counter % serverHost.size, so we could get a order serverHost
2. random: random get serverHost from serverHosts
3. User Definied Class: implemented the ChooseServerStrategy, then put the jar to beeline-jars, it can use your strategy to choose serverHost
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Test the Strategy in my test Cluster
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
[Use Case]
1. poll: `bin/beeline -u 'jdbc:hive2://xxx:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=kyuubi;zooKeeperStrategy=poll?spark.yarn.queue=root.kylin;spark.app.name=testspark;spark.shuffle.useOldFetchProtocol=true' -n mfw_hadoop --verbose=true --showNestedErrs=true`
2. random: `bin/beeline -u 'jdbc:hive2://xxx:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=kyuubi;zooKeeperStrategy=random?spark.yarn.queue=root.kylin;spark.app.name=testspark;spark.shuffle.useOldFetchProtocol=true' -n mfw_hadoop --verbose=true --showNestedErrs=true` or `bin/beeline -u 'jdbc:hive2://xxx:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=kyuubi?spark.yarn.queue=root.kylin;spark.app.name=testspark;spark.shuffle.useOldFetchProtocol=true' -n mfw_hadoop --verbose=true --showNestedErrs=true`
3. YourStrategy: `bin/beeline -u 'jdbc:hive2://xxx:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=kyuubi;zooKeeperStrategy=xxx.xxx.xxx.XxxChooseServerStrategy?spark.yarn.queue=root.kylin;spark.app.name=testspark;spark.shuffle.useOldFetchProtocol=true' -n mfw_hadoop --verbose=true --showNestedErrs=true`
[Result: The Cluster have two Server (221,233)]
1. poll:
1.1. zkNode: counterValue
1.2. result:
2. random:
3. YourStrategy(the test case only get the first serverHost):
#### Related Unit Tests
There is no Unit Tests.
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6213 from davidyuan1223/ha_zk_support_more_strategy.
Closes#6034
961d3e989 [Bowen Liang] rename ServerStrategyFactory to ServerSelectStrategyFactory
353f94059 [Bowen Liang] repeat
8822ad471 [Bowen Liang] repeat
619339402 [Bowen Liang] nit
e94f9e909 [Bowen Liang] nit
40f427ae5 [Bowen Liang] rename StrategyFactory to StrategyFactoryServerStrategyFactory
7668f99cc [Bowen Liang] test name
e194ea62f [Bowen Liang] remove ZooKeeperHiveClientException from method signature of chooseServer
265965e5d [Bowen Liang] polling
b39c56700 [Bowen Liang] style
1ab79b494 [Bowen Liang] strategyName
8f8ca28f2 [Bowen Liang] nit
228bf1091 [Bowen Liang] rename parameter zooKeeperStrategy to serverSelectStrategy
125c82358 [Bowen Liang] rename ChooseServerStrategy to ServerSelectStrategy
b4aeb3dbd [Bowen Liang] repeat testing on pollingChooseStrategy
465548005 [davidyuan] update
09a84f1f9 [david yuan] remove the distirbuted lock
93f4a2699 [davidyuan] remove reset
7b0c1b811 [davidyuan] fix var not valid and counter getAndIncrement
c95382a23 [davidyuan] fix var not valid and counter getAndIncrement
9ed2cac85 [david yuan] remove test comment
8eddd7682 [davidyuan] Add Strategy Unit Test Case and fix the polling strategy counter begin with 0
73952f878 [davidyuan] Kyuubi Server HA&ZK get server from serverHosts support more strategy
97b959776 [davidyuan] Kyuubi Server HA&ZK get server from serverHosts support more strategy
ee5a9ad68 [davidyuan] Kyuubi Server HA&ZK get server from serverHosts support more strategy
6a0445357 [davidyuan] Kyuubi Server HA&ZK get server from serverHosts support more strategy
1892f148d [davidyuan] add common method to get session level config
7c0c6058d [yuanfuyuan] fix_4186
Lead-authored-by: Bowen Liang <liangbowen@gf.com.cn>
Co-authored-by: davidyuan <yuanfuyuan@mafengwo.com>
Co-authored-by: davidyuan <davidyuan1223@gmail.com>
Co-authored-by: david yuan <davidyuan1223@gmail.com>
Co-authored-by: yuanfuyuan <1406957364@qq.com>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
# 🔍 Description
Fix#6594.
This PR ports HIVE-26633(https://github.com/apache/hive/pull/3674): Make thrift client maxMessageSize configurable to fix a regression after upgrading Thrift 0.16 in 1.9.0.
## Types of changes 🔖
- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6631 from pan3793/thrift-max-size.
Closes#6594
e4841c88e [Cheng Pan] [KYUUBI #6594] Port HIVE-26633: Make thrift client maxMessageSize configurable
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
Backport https://github.com/apache/hive/pull/4819
Note: it's only the JDBC driver side change, to make it work, we also need to modify the engines.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
We can not test this feature so far because we don't have engine support that. Pass GHA to ensure it breaks nothing.
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6639 from pan3793/HIVE-27815.
Closes#6639
4b3bcd65f [Cheng Pan] fix
c16dc28e8 [Cheng Pan] Port HIVE-27815: Support update numModifiedRows
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
This pull request fixes#6619
## Describe Your Solution 🔧
This is a subtask of #6590
This PR makes the following changes:
1. Instead of using SessionManager, the ThriftHttpService will use AuthorizationFilter to handle http client parameters. The purpose is to pass correct client IP address for bearer token authentication.
2. Partial backport Partial backport: https://github.com/apache/hive/pull/3006
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6624 from George314159/thrift.
Closes#6619
9761d6ddf [Wang, Fei] Partial backport HIVE-25575: Add support for JWT authentication in HTTP mode
30ede05d8 [Wang, Fei] Revert "suffix"
39c945802 [Wang, Fei] suffix
b99420633 [Wang, Fei] Revert "Fix"
ea6236a8d [George314159] Fix
a75025be4 [George314159] Refine UT
945433583 [George314159] Refine
8f4eeff03 [George314159] Update UT
6192d4713 [Wang, Fei] revert unused
c7f037c66 [Wang, Fei] Support bearer token in kyuubi jdbc
f7725f761 [George314159] Refine
f865abf01 [George314159] Refine
9d4edd50d [George314159] Add http header config for thrift bearer token auth and UT
6d0f97355 [Wang, Fei] refine
a7e785bcd [Wang, Fei] save
Lead-authored-by: Wang, Fei <fwang12@ebay.com>
Co-authored-by: George314159 <hua16732@gmail.com>
Signed-off-by: Wang, Fei <fwang12@ebay.com>
# 🔍 Description
## Issue References 🔗
This pull request fixes#6396
## Describe Your Solution 🔧
NPE will occur when kyuubiClientTicketCache is not specified. `ConcurrentHashMap` does not allow null key.
## Types of changes 🔖
- [X] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6420 from wForget/hotfix2.
Closes#6396
e5d5f9d33 [wforget] address comment
fe9ecd1b1 [wforget] Avoid NPE
Authored-by: wforget <643348094@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
This pull request fixes#6396
## Describe Your Solution 🔧
By using a cache to store CachingKerberosAuthentication objects keyed by the ticket cache path, we ensure that each unique ticket cache path generates a distinct authentication object.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6401 from lsm1/branch-kyuubi-6396.
Closes#6396
bb8f738e1 [senmiaoliu] fix kyuubiClientTicketCache
Authored-by: senmiaoliu <senmiaoliu@trip.com>
Signed-off-by: Shaoyun Chen <csy@apache.org>
# 🔍 Description
`Base64.getMimeEncoder`(RFC2045) might generate newline when encoded chars exceed 76, so I changed it to `Base64.getEncoder`(RFC4648).
## Types of changes 🔖
- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Use the changed BeeLine to connect to Kyuubi Server 1.9.0, everything goes well.
```
$ bin/beeline -u 'jdbc:kyuubi://0.0.0.0:10009/'
...
Connected to: Spark SQL (version 3.4.1)
Driver: Kyuubi Project Hive JDBC Client (version 1.10.0-SNAPSHOT)
Beeline version 1.10.0-SNAPSHOT by Apache Kyuubi
0: jdbc:kyuubi://0.0.0.0:10009/>
```
---
# Checklist 📝
- [X] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6408 from wForget/base64.
Closes#6408
a19f6f64f [wforget] change to Base64.getEncoder/getDecoder
7052a9346 [wforget] change to Base64.getEncoder/getDecoder
1b94ad991 [wforget] Change to Base64 urlEncoder/urlDecoder for handle guid and credential
Authored-by: wforget <643348094@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
This pull request fixes#6346
## Describe Your Solution 🔧
Change Kyuubi JDBC Driver default port and make it effective.
## Types of changes 🔖
- [X] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [X] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6347 from wForget/KYUUBI-6346.
Closes#6346
0a3085205 [wforget] [KYUUBI #6346] Change Kyuubi JDBC Driver default port and make it effective
Authored-by: wforget <643348094@qq.com>
Signed-off-by: wforget <643348094@qq.com>
# 🔍 Description
## Issue References 🔗
This pull request fixes#6221
## Describe Your Solution 🔧
Fix the static function that split the sql by parameter placeholder '?', add the handling of cases when the placeholder is within the double quote or comment.
## Types of changes 🔖
- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
org.apache.kyuubi.jdbc.hive.UtilsTest.testSplitSqlStatement
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6222 from TakawaAkirayo/fix_sql_split_issue.
Closes#6221
a501c38ea [tatian] [KYUUBI #6221] Let UT cover another case
3b9714569 [tatian] [KYUUBI #6221] Fix parameter replacement issue caused by incorrect sql split
Authored-by: tatian <tatian@ebay.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
This pull request fixes #
Fix https://github.com/apache/kyuubi/pull/6207#discussion_r1538539729
## Describe Your Solution 🔧
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6209 from turboFei/fix_union_type.
Closes#2782
622c838cd [Wang, Fei] miss
Authored-by: Wang, Fei <fwang12@ebay.com>
Signed-off-by: Wang, Fei <fwang12@ebay.com>
# 🔍 Description
## Issue References 🔗
This pull request fixes #
To fix below issue:
```
24/03/25 00:47:10 ERROR SparkTBinaryFrontendService: Error getting result set metadata:
java.lang.IllegalArgumentException: Unrecognized type name: struct<type:tinyint,size:int,indices:array<int>,values:array<double>>
```
<img width="1567" alt="image" src="https://github.com/apache/kyuubi/assets/6757692/9067d2d2-06d9-4937-b328-71434def34fd">
## Describe Your Solution 🔧
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
After this pr:
<img width="1728" alt="image" src="https://github.com/apache/kyuubi/assets/6757692/2d9f4f0b-9ac4-48e9-9e6a-4c0f1616edf9">
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6207 from turboFei/udt.
Closes#6207
b492568e3 [Wang, Fei] Revert "miss union type"
39ac1c42f [Wang, Fei] miss union type
8c32f54af [Wang, Fei] comment
00a469855 [Wang, Fei] getColumnTypeName
d7d291652 [Wang, Fei] ut
edce5cf1f [Wang, Fei] support udt
Authored-by: Wang, Fei <fwang12@ebay.com>
Signed-off-by: Wang, Fei <fwang12@ebay.com>
# 🔍 Description
## Issue References 🔗
This PR upgrades Kyuubi Shaded to 0.3.0, the major changes are:
- upgrade Thrift from 0.9.3-1 to 0.16.0
- upgrade JLine used by Zookeeper CLI from 0.9.94 to 2.14.6
- split kyuubi-relocated-thrift from kyuubi-relocated-hive-service-rpc
- introduce a light kyuubi-relocated-hive-metastore-client for getting DelegationToken
Due to THRIFT-5297 (fixed in 0.14.0) removes requestTimeout and beBackoffSlotLength, the following Kyuubi configurations are removed too.
- kyuubi.frontend.login.timeout
- kyuubi.frontend.thrift.login.timeout
- kyuubi.frontend.backoff.slot.length
- kyuubi.frontend.thrift.backoff.slot.length
Previously, Zookeeper 3.4 depends on JLine 0.9.94 while Zookeeper 3.6 and Hive Beeline depends on JLine 2.14.6, we pulls different versions of JLine into the binary distribution. Now we align them to a single JLine 2.14.6.
## Describe Your Solution 🔧
Upgrade dependencies and change code to adapt to the breaking changes, also upgrade migration guide to mention the removed configurations.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Pass GA.
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6117 from pan3793/shaded-0.3.
Closes#6117
5639c9fd8 [Cheng Pan] nit
d281cdb4b [Cheng Pan] fix
184e1b95b [Cheng Pan] fix
1628337c8 [Cheng Pan] remove unused conf
24db2d5ad [Cheng Pan] try fix
1e995bb34 [Cheng Pan] nit
1d8e9bce9 [Cheng Pan] dep list
7f33624e8 [Cheng Pan] Bump Kyuubi Shaded 0.3.0
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
This pull request fixes#6114
## Describe Your Solution 🔧
After fixing [HIVE-7680](https://issues.apache.org/jira/browse/HIVE-7680) issue the method getMoreResults() of HiveStatement and respectively KyuubiStatementclasses returns false instead of throwing exception. The javadoc of the Statement#getMoreResults()method says, that it should implicitly close any current ResultSet object, i.e. is similar to calling the KyuubiStatement#getMoreResults(int) method with Statement.CLOSE_CURRENT_RESULT argument. Therefore, in the latter case, we should also return false instead of failing with an exception. This fix also allows Kyuubi JDBC driver to be used in the JDBC engine.
## Types of changes 🔖
- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6115 from tigrulya-exe/bugfix/jdbc-driver-get-more-results.
Closes#6114
b67479166 [Tigran Manasyan] Close result set in case of KyuubiStatement#getMoreResults(CLOSE_CURRENT_RESULT)
eeedaf898 [Tigran Manasyan] Do not throw exception in KyuubiStatement#getMoreResults(int) in case of closing current result set
Authored-by: Tigran Manasyan <t.manasyan@arenadata.io>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
I get reported by a user that using Kyuubi JDBC driver 1.8.0 to access Spark Thrift Server 2.4 (with Hive 1.2.1) with getting an error on opening the session even with `clientProtocol=7`
```
org.apache.hive.service.cli.HiveSQLException: java.lang.IllegalArgumentException: hive configuration hive.server2.thrift.resultset.default.fetch.size does not exists.
at org.apache.hive.service.cli.session.HiveSessionImpl.configureSession(HiveSessionImpl.java:220)
at org.apache.hive.service.cli.session.HiveSessionImpl.open(HiveSessionImpl.java:154)
at org.apache.hive.service.cli.session.SessionManager.openSession(SessionManager.java:258)
... 13 more
```
## Describe Your Solution 🔧
When `hive.conf.validation` is `true` (it also is the default value), an IllegalArgumentException will be thrown if the provided configurations start with `hive.` and can not be recognized by the server.
One solution is to disable `hive.conf.validation` on the server side, but we can also address it by avoiding passing this configuration if we know that the server does not support it.
HIVE-14901 (2.3.0, HIVE_CLI_SERVICE_PROTOCOL_V10) introduces `hive.server2.thrift.resultset.default.fetch.size`, so we can set this configuration only when protocol >= HIVE_CLI_SERVICE_PROTOCOL_V10
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Verified locally by connecting to HS2 2.1.1 and HS2 2.3.9
#### Behavior Without This Pull Request ⚰️
IllegalArgumentException throws when using `jdbc:hive2://localhost:10000/default;clientProtocol=8` to connect HS2 2.1.1
Everything is OK when using `jdbc:hive2://localhost:10000/default` to connect HS2 2.3.9
#### Behavior With This Pull Request 🎉
Everything is OK when using `jdbc:hive2://localhost:10000/default;clientProtocol=8` to connect HS2 2.1.1
Everything is OK when using `jdbc:hive2://localhost:10000/default` to connect HS2 2.3.9
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6036 from pan3793/jdbc-fetchsize.
Closes#6036
7ea91f12c [Cheng Pan] nit
e6ea8291f [Cheng Pan] fix
1dbecbb48 [Cheng Pan] JDBC driver conditional sets fetchSize on opening session
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
Currently, the Kyuubi Beeline is not compatible with HS2 2.3 and lower versions of Kyuubi, because the server can not recognize `TGetInfoType.CLI_ODBC_KEYWORDS` and then corrupts the socket.
## Describe Your Solution 🔧
Conditional call GetInfo CLI_ODBC_KEYWORDS, when protocol version lower than TProtocolVersion.HIVE_CLI_SERVICE_PROTOCOL_V11, just throw an SQLFeatureNotSupportedException
## Types of changes 🔖
- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
Use beeline to connect a HS2 2.3
#### Behavior Without This Pull Request ⚰️
```
roothadoop-master1:/opt/kyuubi-dev# bin/beeline -u 'jdbc:hive2://localhost:10000/default'
Warn: Not find kyuubi environment file /etc/kyuubi/conf/kyuubi-env.sh, using default ones...
Connecting to jdbc:hive2://localhost:10000/default
Connected to: Apache Hive (version 2.3.9)
Driver: Kyuubi Project Hive JDBC Client (version 1.8.0)
Beeline version 1.8.0 by Apache Kyuubi
0: jdbc:hive2://localhost:10000/default> select 1;
Unexpected end of file when reading from HS2 server. The root cause might be too many concurrent connections. Please ask the administrator to check the number of active connections, and adjust hive.server2.thrift.max.worker.threads if applicable.
Error: org.apache.thrift.transport.TTransportException (state=08S01,code=0)
```
And the HS2 server-side error message is
```
24/01/25 03:50:53 [37217e6b-e8a9-4069-9973-10471df52e6d HiveServer2-Handler-Pool: Thread-37]: INFO conf.HiveConf: Using the default value passed in for log id: 37217e6b-e8a9-4069-9973-10471df52e6d
24/01/25 03:50:53 [HiveServer2-Handler-Pool: Thread-37]: ERROR server.TThreadPoolServer: Thrift error occurred during processing of message.
org.apache.thrift.protocol.TProtocolException: Missing version in readMessageBegin, old client?
at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:228) ~[hive-exec-2.3.9.jar:2.3.9]
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:27) ~[hive-exec-2.3.9.jar:2.3.9]
at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56) ~[hive-service-2.3.9.jar:2.3.9]
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) ~[hive-exec-2.3.9.jar:2.3.9]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_392]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_392]
at java.lang.Thread.run(Thread.java:750) [?:1.8.0_392]
24/01/25 03:50:53 [HiveServer2-Handler-Pool: Thread-37]: INFO thrift.ThriftCLIService: Session disconnected without closing properly.
24/01/25 03:50:53 [HiveServer2-Handler-Pool: Thread-37]: INFO thrift.ThriftCLIService: Closing the session: SessionHandle [37217e6b-e8a9-4069-9973-10471df52e6d]
```
#### Behavior With This Pull Request 🎉
```
roothadoop-master1:/opt/kyuubi-dev# bin/beeline -u 'jdbc:hive2://localhost:10000/default'
Warn: Not find kyuubi environment file /etc/kyuubi/conf/kyuubi-env.sh, using default ones...
Connecting to jdbc:hive2://localhost:10000/default
Connected to: Apache Hive (version 2.3.9)
Driver: Kyuubi Project Hive JDBC Client (version 1.9.0-SNAPSHOT)
Beeline version 1.9.0-SNAPSHOT by Apache Kyuubi
0: jdbc:hive2://localhost:10000/default> select 1;
+------+
| _c0 |
+------+
| 1 |
+------+
1 row selected (2.403 seconds)
0: jdbc:hive2://localhost:10000/default>
```
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#6016 from pan3793/jdbc-comp.
Closes#6016
413c7a40d [Cheng Pan] Conditional call GetInfo CLI_ODBC_KEYWORDS to restore compatible with lower version Kyuubi and HS2
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
Followup #5961
```
scala> Files.exists(Paths.get(null))
<console>:14: error: ambiguous reference to overloaded definition,
both method get in class Paths of type (x$1: java.net.URI)java.nio.file.Path
and method get in class Paths of type (x$1: String, x$2: String*)java.nio.file.Path
match argument types (Null) and expected result type java.nio.file.Path
Files.exists(Paths.get(null))
^
scala> Files.exists(Paths.get(""))
res0: Boolean = true
scala>
```
## Issue References 🔗
This pull request fixes #
## Describe Your Solution 🔧
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#5980 from turboFei/fix_npe.
Closes#5961
99d4a1695 [Fei Wang] fix npe
Authored-by: Fei Wang <fwang12@ebay.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
The use case:
- 1. the user code running in flink cluster and wrapped by transparent `ugi.doAs`
- 2. the current UserGroupInfomation is the proxy user not login/real user
- 3. user specify url with kyuubiClientTicketCache but does not work
- 4. the proxy UserGroupInfomation is used and then throw GSS issue.
```
Caused by: org.apache.kyuubi.shade.org.apache.thrift.transport.TTransportException: GSS initiate failed
at org.apache.kyuubi.shade.org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:232)
at org.apache.kyuubi.shade.org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:316)
at org.apache.kyuubi.shade.org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
at org.apache.kyuubi.jdbc.hive.auth.TSubjectTransport.lambda$open$0(TSubjectTransport.java:47)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.kyuubi.jdbc.hive.auth.TSubjectTransport.open(TSubjectTransport.java:42)
at org.apache.kyuubi.jdbc.hive.KyuubiConnection.openTransport(KyuubiConnection.java:458)
at org.apache.kyuubi.jdbc.hive.KyuubiConnection.<init>(KyuubiConnection.java:207)
... 22 more
```
The root cause is that, for this case, the result of `isHadoopUserGroupInformationDoAs` is true.
So, `isFromSubjectAuthMode` is true.
In this pr, I want to specify the kerberosAuthType to fromTicketCache, and do not check `isHadoopUserGroupInformationDoAs`.
After this pr, customer can specify `kerberosAuthType=fromTicketCache` to leverage ticket cache prefer than `ugi.doAs`.
## Issue References 🔗
This pull request fixes #
## Describe Your Solution 🔧
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklist 📝
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes#5961 from turboFei/warn_ticket_cache.
Closes#5961
6e065005d [Fei Wang] exception
c01a99e24 [Fei Wang] refine
098a37b2d [Fei Wang] do not infer fromSubject if fromTicketCache && ugi
042fa2249 [Fei Wang] warn
Authored-by: Fei Wang <fwang12@ebay.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
This pull request fixes#5849
## Describe Your Solution 🔧
When replacing the `host:port` information in the jdbc url, only the first matching string should be replaced.
## Types of changes 🔖
- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklists
## 📝 Author Self Checklist
- [x] My code follows the [style guidelines](https://kyuubi.readthedocs.io/en/master/contributing/code/style.html) of this project
- [x] I have performed a self-review
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my feature works
- [x] New and existing unit tests pass locally with my changes
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
## 📝 Committer Pre-Merge Checklist
- [x] Pull request title is okay.
- [x] No license issues.
- [x] Milestone correctly set?
- [x] Test coverage is ok
- [x] Assignees are selected.
- [x] Minimum number of approvals
- [x] No changes are requested
**Be nice. Be informative.**
Closes#5850 from hameizi/fix-url-parse.
Closes#5849
491407860 [wangzeyu] fix code style
32565f272 [wangzeyu] fix unit test
e1e6d0a63 [wangzeyu] add unit test
9af1d0596 [wangzeyu] fix
Authored-by: wangzeyu <hameizi369@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
This pull request fixes#5709
## Describe Your Solution 🔧
Add a new JDBC URL property, storePasswordPath, which points to a local JCE keystore file storing password aliases.If the password exists, use password. If it does not exist, try to use storePasswordPath. All these passwords can be hidden from the JDBC URL when you protect them in a local JCEKS keystore file and pass the JCEKS file path to the URL.
JDBC URL:
`jdbc:hive2://wapdfwudp001.webex.com:30009/default;ssl=true;sslTrustStore=kyuubissl.truststore.keystore.jks;storePasswordPath=jceks://file/client_trust_creds.jceks`
Hadoop credential command with trustStorePassword and keyStorePassword aliases like below
hadoop credential create ssl.kyuubi.trustStorePassword -value mytruststorepassword -provider localjceks://file/client_trust_creds.jceks
more info: https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients
Reference PR comes from HIVE-27308
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
Local test passed
<img width="1093" alt="Screenshot 2023-11-16 at 10 43 03" src="https://github.com/apache/kyuubi/assets/43336508/3be87238-dfd7-4484-8945-e62780c43d9a">
---
# Checklists
## 📝 Author Self Checklist
- [ ] My code follows the [style guidelines](https://kyuubi.readthedocs.io/en/master/contributing/code/style.html) of this project
- [ ] I have performed a self-review
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
## 📝 Committer Pre-Merge Checklist
- [x] Pull request title is okay.
- [x] No license issues.
- [x] Milestone correctly set?
- [ ] Test coverage is ok
- [x] Assignees are selected.
- [x] Minimum number of approvals
- [x] No changes are requested
**Be nice. Be informative.**
Closes#5710 from dev-lpq/enhance_url_password.
Closes#5709
98cba7ef4 [pengqli] resolve conflicts
1055dcc41 [pengqli] use reflection to check Hadoop classes
c3096146d [pengqli] use reflection to check Hadoop classes
f94f3024b [pengqli] Enhanced URL password
Authored-by: pengqli <pengqli@cisco.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
TL;DR there are some issues with shading Thrift RPC classes during the engine packaging phase, see details in the PR description of https://github.com/apache/kyuubi-shaded/pull/20.
## Describe Your Solution 🔧
This PR aims to migrate from vanilla `hive-service-rpc`, `libfb303`, `libthrift` to `kyuubi-relocated-hive-service-rpc` introduced in https://github.com/apache/kyuubi-shaded/pull/20, the detailed works are:
- replace imported deps in `pom.xml` and rename the package prefix in all modules, except for
- `kyuubi-server` there are a few places use vanilla thrift classes to access HMS to get token
- `kyuubi-hive-sql-engine` Hive method invocation
- update relocations rules in modules that creates shaded jar
- introduce `HiveRpcUtils` in `kyuubi-hive-sql-engine` module for object conversion.
As part of the whole change, this PR upgrades from the Kyuubi Shaded 0.1.0 to 0.2.0, which changes the jars name. see https://kyuubi.apache.org/shaded-release/0.2.0.html
## Types of changes 🔖
- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
Pass all Hive UT with Hive 3.1.3, and IT with Hive 3.1.3 and 2.3.9 (also tested with 2.1.1-cdh6.3.2)
---
# Checklists
## 📝 Author Self Checklist
- [ ] My code follows the [style guidelines](https://kyuubi.readthedocs.io/en/master/contributing/code/style.html) of this project
- [x] I have performed a self-review
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my feature works
- [x] New and existing unit tests pass locally with my changes
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
## 📝 Committer Pre-Merge Checklist
- [x] Pull request title is okay.
- [x] No license issues.
- [x] Milestone correctly set?
- [x] Test coverage is ok
- [x] Assignees are selected.
- [x] Minimum number of approvals
- [x] No changes are requested
**Be nice. Be informative.**
Closes#5783 from pan3793/rpc-shaded.
Closes#5783
b45d4deaa [Cheng Pan] remove staging repo
890076a20 [Cheng Pan] Kyuubi Shaded 0.2.0 RC0
071945d45 [Cheng Pan] Rebase
199794ed9 [Cheng Pan] fix
fc128b170 [Cheng Pan] fix
26d313896 [Cheng Pan] fix
632984c92 [Cheng Pan] fix
428305589 [Cheng Pan] fix
6301e28fd [Cheng Pan] fix
955cdb33b [Cheng Pan] Switch to kyuubi-shaded-hive-service-rpc
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
## Issue References 🔗
This pull request fixes#5650
## Describe Your Solution 🔧
implemented getDate, getTimestamp and getTime with Calendar in Kyuubi JDBC driver side
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklists
## 📝 Author Self Checklist
- [x] My code follows the [style guidelines](https://kyuubi.readthedocs.io/en/master/contributing/code/style.html) of this project
- [x] I have performed a self-review
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
## 📝 Committer Pre-Merge Checklist
- [ ] Pull request title is okay.
- [ ] No license issues.
- [ ] Milestone correctly set?
- [ ] Test coverage is ok
- [ ] Assignees are selected.
- [ ] Minimum number of approvals
- [ ] No changes are requested
**Be nice. Be informative.**
Closes#5752 from zhaohehuhu/dev-1122.
Closes#5752
ddff6a7aa [Cheng Pan] Update kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/KyuubiArrowBasedResultSet.java
595226d04 [Cheng Pan] Update kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/KyuubiArrowBasedResultSet.java
8b386f151 [Cheng Pan] Update kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/KyuubiArrowBasedResultSet.java
190c6b051 [Cheng Pan] Update kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/KyuubiBaseResultSet.java
dbc1e1ea7 [Cheng Pan] Update kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/KyuubiBaseResultSet.java
e1a36d3e8 [Cheng Pan] Update kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/KyuubiBaseResultSet.java
36710d0b1 [hezhao2] refactor
cd2e36f7c [hezhao2] add UTs in SparkDataTypeTests
fe60822b5 [hezhao2] add UTs in SparkDataTypeTests
77313c224 [hezhao2] reformat code style
33e4305e3 [hezhao2] JDBC driver implements Result#getTimestamp and Result#getTime with Calendar
7623b854e [hezhao2] add Overwrite annotation and remove this method from the parent abstract class
757f03701 [hezhao2] JDBC driver implements Result#getDate with Calendar
Lead-authored-by: hezhao2 <hezhao2@cisco.com>
Co-authored-by: Cheng Pan <pan3793@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
# 🔍 Description
Backport https://github.com/apache/hive/pull/4262
## Issue References 🔗
This pull request fixes ##5713
## Describe Your Solution 🔧
trustStorePassword is not a necessary parameter in connection URL. Connection can be established without it.
From the javadocs [Link](https://docs.oracle.com/javase/7/docs/api/java/security/KeyStore.html#load(java.io.InputStream,%20char%5B%5D)) A password may be given to unlock the keystore (e.g. the keystore resides on a hardware token device), or to check the integrity of the keystore data. If a password is not given for integrity checking, then integrity checking is not performed.
In order to create an empty keystore, or if the keystore cannot be initialized from a stream, pass null as the stream argument.
Reference PR comes from HIVE-27271
## Types of changes 🔖
- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklists
## 📝 Author Self Checklist
- [ ] My code follows the [style guidelines](https://kyuubi.readthedocs.io/en/master/contributing/code/style.html) of this project
- [ ] I have performed a self-review
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
## 📝 Committer Pre-Merge Checklist
- [x] Pull request title is okay.
- [x] No license issues.
- [x] Milestone correctly set?
- [ ] Test coverage is ok
- [x] Assignees are selected.
- [x] Minimum number of approvals
- [x] No changes are requested
**Be nice. Be informative.**
Closes#5712 from dev-lpq/ssl_http_store.
Closes#5713
c1011e487 [pengqli] Support client connection when transportMode=http,ssl=true, sslTrustStore specified without trustStorePassword in the JDBC URL
Authored-by: pengqli <pengqli@cisco.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Support to config customized ticket cache location in kyuubi connection URL.
Before, we can only set it via environment variable `KRB5CCNAME`.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No.
Closes#5564 from turboFei/ticket_cache.
Closes#5564
140ae6ec2 [fwang12] since 1.8.0
caaf33a44 [fwang12] check is blank
0fae9f9c5 [fwang12] doc
05b9e9df8 [fwang12] ticket cache
Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
When using Kyuubi Hive JDBC to Hive Server2, TProtocolVersion.HIVE_CLI_SERVICE_PROTOCOL_V10 is used and can not be changed.
When we connected to Hive Server2 with version lower than 2.2.0, we got the following error:
```
org.apache.kyuubi.shade.org.apache.thrift.TApplicationException:
Required field 'client_protocol' is unset!
Struct:TOpenSessionReq(client_protocol:null, configuration:{kyuubi.client.version=1.7.3, set:hiveconf:hive.server2.thrift.resultset.default.fetch.size=1000, kyuubi.client.ipAddress=172.16.19.113, use:database=default})
```
In this PR, we introduced a session conf `clientProtocolVersion`.
By adding `clientProtocolVersion=8` to jdbc url, the error got fixed.
Changes of `kyuubi_jdbc.rst`
<img width="867" alt="image" src="https://github.com/apache/kyuubi/assets/88070094/8f98edf9-15c4-4d1b-9299-83b24136352b">
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Manual tests against Hive Server2 version 2.1.1-cdh6.3.0
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No.
Closes#5480 from zhouyifan279/protocol-version.
Closes#5480
0ee7b1f68 [zhouyifan279] Support setting kyuubi hive jdbc client protocol version
61b8038d3 [zhouyifan279] Support setting kyuubi hive jdbc client protocol version
ed4c29f2b [Cheng Pan] Update docs/deployment/migration-guide.md
dc16a05fc [zhouyifan279] Support setting kyuubi hive jdbc client protocol version
1a6bfd88c [zhouyifan279] Support setting kyuubi hive jdbc client protocol version
cb00edc37 [zhouyifan279] Support setting kyuubi hive jdbc client protocol version
c99fc4802 [zhouyifan279] Support setting kyuubi hive jdbc client protocol version
Lead-authored-by: zhouyifan279 <zhouyifan279@gmail.com>
Co-authored-by: Cheng Pan <pan3793@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
The Apache Spark Community found a performance regression with log4j2. See https://github.com/apache/spark/pull/36747.
This PR to fix the performance issue on our side.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No.
Closes#5400 from ITzhangqiang/KYUUBI_5365.
Closes#5365
dbb9d8b32 [ITzhangqiang] [KYUUBI #5365] Don't use Log4j2's extended throwable conversion pattern in default logging configurations
Authored-by: ITzhangqiang <itzhangqiang@163.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
After https://github.com/apache/kyuubi/pull/3354, thrift `out of sequence` exception will be thrown when using a higher version of beeline to access a lower version of Kyuubi Server.
```
2023-09-26 11:10:16.426 ERROR org.apache.kyuubi.server.KyuubiTBinaryFrontendService: Error fetching results:
org.apache.thrift.protocol.TProtocolException: Required field 'operationHandle' is unset! Struct:TFetchResultsReq(operationHandle:null, orientation:FETCH_NEXT, maxRows:1000, fetchType:0)
at org.apache.hive.service.rpc.thrift.TFetchResultsReq.validate(TFetchResultsReq.java:548) ~[hive-service-rpc-3.1.3.jar:3.1.3]
at org.apache.hive.service.rpc.thrift.TCLIService$FetchResults_args.validate(TCLIService.java:15755) ~[hive-service-rpc-3.1.3.jar:3.1.3]
at org.apache.hive.service.rpc.thrift.TCLIService$FetchResults_args$FetchResults_argsStandardScheme.write(TCLIService.java:15812) ~[hive-service-rpc-3.1.3.jar:3.1.3]
at org.apache.hive.service.rpc.thrift.TCLIService$FetchResults_args$FetchResults_argsStandardScheme.write(TCLIService.java:15781) ~[hive-service-rpc-3.1.3.jar:3.1.3]
at org.apache.hive.service.rpc.thrift.TCLIService$FetchResults_args.write(TCLIService.java:15732) ~[hive-service-rpc-3.1.3.jar:3.1.3]
at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:71) ~[libthrift-0.9.3.jar:0.9.3]
at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:62) ~[libthrift-0.9.3.jar:0.9.3]
at org.apache.hive.service.rpc.thrift.TCLIService$Client.send_FetchResults(TCLIService.java:561) ~[hive-service-rpc-3.1.3.jar:3.1.3]
at org.apache.hive.service.rpc.thrift.TCLIService$Client.FetchResults(TCLIService.java:553) ~[hive-service-rpc-3.1.3.jar:3.1.3]
at org.apache.kyuubi.client.KyuubiSyncThriftClient.$anonfun$fetchResults$1(KyuubiSyncThriftClient.scala:393) ~[kyuubi-server_2.12-1.6.1-incubating.jar:1.6.1-incubating]
at org.apache.kyuubi.client.KyuubiSyncThriftClient.$anonfun$withLockAcquiredAsyncRequest$2(KyuubiSyncThriftClient.scala:136) ~[kyuubi-server_2.12-1.6.1-incubating.jar:1.6.1-incubating]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_152]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_152]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[?:1.8.0_152]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_152]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_152]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_152]
```
This PR introduces a negotiation mechanism to make the JDBC client adaptively call `fetchLaunchEngineResult` to address such a compatible issue.
It supposes that the higher version of JDBC client could seamlessly communicate with the lower version of Kyuubi Server after this patch, but the JDBC client fetches the engine information only when the connected Server applies this patch too.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No.
Closes#5333 from pan3793/launch-engine-result.
Closes#5333
ca9bd5361 [Cheng Pan] [JDBC] Adaptive call fetchLaunchEngineResult
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Currently, `getNextRowSetInternal` returns `TRowSet` which is not friendly to explicit EOS in streaming result fetch.
This PR changes the return type to `TFetchResultsResp` to allow the engines to determine the EOS.
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
Closes#5160 from link3280/refactor_result.
Closes#5160
09822f2ee [Paul Lin] Fix hasMoreRows missing
c94907e2b [Paul Lin] Explicitly set `resp.setHasMoreRows(false)` for operations
4d193fb1d [Paul Lin] Revert unrelated changes in FlinkOperation
ffd0367b3 [Paul Lin] Refactor getNextRowSetInternal to support fetch streaming data
Authored-by: Paul Lin <paullin3280@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- remove duplicated assignment for the same variable in adjacent lines in `FastHiveDecimalImpl`
- replace redundant `putAll` with collection initialization in `BatchRestApi`
- use `try-with-resources` statement with the reader and avoid declaring two variables in the same line of code in `KyuubiCommands`
- fix `warning: Tag 'return:' is not recognised` compilation warning in `KyuubiGetSqlClassification:L53`
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
Closes#5117 from bowenliang123/fastsignum.
Closes#5117
595b5747d [liangbowen] simplify
be530fac4 [liangbowen] fix warning: Tag '@return:' is not recognised compilation warning in KyuubiGetSqlClassification:L53
249706905 [liangbowen] use try-with-resources in KyuubiCommands
a54a97fdd [liangbowen] remove redundant addAll call to collection initialization
cc76d5d0f [liangbowen] remove repeated assignment
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- remove 2 unused string builders in `KyuubiQueryResultSet` and `KyuubiArrowQueryResultSet`, which are only appended separator only and never queried again
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
Closes#5101 from bowenliang123/unused-sb.
Closes#5101
ccb6fb77d [liangbowen] remove never queried StringBuilders
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
- Remove the existing single quote in message format which causes the argument 0 is not used
- `A single quote itself must be represented by doubled single quotes '' throughout a String.` https://docs.oracle.com/javase/8/docs/api/java/text/MessageFormat.html
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
Closes#5100 from bowenliang123/datatype-msg.
Closes#5100
8135ff146 [liangbowen] fix
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
To prevent conflicts in case that other dependency also has a resource named `version.properties`.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
Closes#5054 from turboFei/version_conflicts.
Closes#5054
fe1333dbe [fwang12] prevent version properties conflicts
Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: fwang12 <fwang12@ebay.com>
### _Why are the changes needed?_
- apply the usage of `ReflectUtils` and `Dyn*` to the modules of engines and plugins (eg. Spark engine, Authz plugin, lineage plugin, beeline)
- remove similar redundant methods for calling reflected methods or getting field values
- unified reflection helper methods with type casting support, as `getField[T]` for getting field values from `getFields`, `invokeAs[T]` for invoking methods in `getMethods`.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4879 from bowenliang123/reflect-use.
Closes#4879
c685fb67d [liangbowen] bug fix for "Cannot bind static field options" when executing "bin/beeline"
fc1fdf1de [liangbowen] import
59c3dd032 [liangbowen] comment
c435c131d [liangbowen] reflect util usage
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
This PR aims to migrate the vanilla Zookeeper and Curator to the Kyuubi Shaded Zookeeper. It's the first step to adapting JDK 17.
There is a known issue [ZOOKEEPER-3779](https://issues.apache.org/jira/browse/ZOOKEEPER-3779) that Zookeeper 3.4 client can not run on JDK 14 and above, in https://github.com/apache/kyuubi-shaded/pull/5, we fixed this issue by a surgical.
With the above fixing, zk-3.4 and zk-3.6 clients both work well on JDK 17, we just randomly pick some cases to make sure zk-3.6 is tested
zk-3.4 client supports zk-3.4+ server, but zk-3.6 client only supports zk-3.5+ server; in the meanwhile, zk-3.4 is adopted widely, (CDH 5/6, HDP, EMR created before 2023).
We are sticky to zk-3.4 to ensure that Kyuubi can be out-of-box in the most existing Hadoop cluster but also provide zk-3.6 as an alternative(simply replace the kyuubi-shaded-zk-3.4 jar w/ kyuubi-shaded-zk-3.6, or build w/ -Pzookeeper-3.6) for users who concerns that zk-3.4 is EOL.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4852 from pan3793/shaded-zk.
Closes#4852
d960cc945 [Cheng Pan] remove staging repo
1b3622080 [Cheng Pan] Switch to Kyuubi Shaded Zookeeper
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
```
Cause: org.apache.http.HttpException: Cannot invoke "String.toCharArray()" because "message" is null
at org.apache.kyuubi.jdbc.hive.auth.HttpRequestInterceptorBase.process(HttpRequestInterceptorBase.java:113)
at org.apache.http.protocol.ImmutableHttpProcessor.process(ImmutableHttpProcessor.java:133)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.ServiceUnavailableRetryExec.execute(ServiceUnavailableRetryExec.java:85)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:118)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at org.apache.thrift.transport.THttpClient.flushUsingHttpClient(THttpClient.java:251)
...
Cause: java.lang.NullPointerException: Cannot invoke "String.toCharArray()" because "message" is null
at org.apache.http.HttpException.clean(HttpException.java:48)
at org.apache.http.HttpException.<init>(HttpException.java:105)
at org.apache.kyuubi.jdbc.hive.auth.HttpKerberosRequestInterceptor.addHttpAuthHeader(HttpKerberosRequestInterceptor.java:68)
at org.apache.kyuubi.jdbc.hive.auth.HttpRequestInterceptorBase.process(HttpRequestInterceptorBase.java:82)
at org.apache.http.protocol.ImmutableHttpProcessor.process(ImmutableHttpProcessor.java:133)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.ServiceUnavailableRetryExec.execute(ServiceUnavailableRetryExec.java:85)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
...
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4859 from pan3793/http-exception.
Closes#4859
94c83a362 [Cheng Pan] HttpException message should not be null
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
close#4325
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4417 from yehere/kyuubi-4325.
Closes#4325
7b2864b53 [yehere] [KYUUBI #4325] Support replace preparedStatement for Trino-jdbc,update the dependency file, run './build/dependency.sh --replace'
749b1c15c [yehere] [KYUUBI #4325] Support replace preparedStatement for Trino-jdbc,Code optimization
33ea9ba2b [yehere] [KYUUBI #4325] Support replace preparedStatement for Trino-jdbc,Code optimization
568418a21 [yehere] [KYUUBI #4325] Support replace preparedStatement for Trino-jdbc,Add test case for DEALLOCATE PREPARE
358a8e3b8 [yehere] [KYUUBI #4325] Support replace preparedStatement for Trino-jdbc,Support DEALLOCATE PREPARE
7d4a32402 [yehere] [KYUUBI #4325] Support replace preparedStatement for Trino-jdbc
91392add6 [yehere] [KYUUBI #4325] Support replace preparedStatement for Trino-jdbc
63bf8c462 [yehere] [KYUUBI #4325] Support replace preparedStatement for Trino-jdbc
f5b7fb786 [yehere] [KYUUBI #4325] Support replace preparedStatement for Trino-jdbc
b0476a79d [yehere] [KYUUBI #4325] Support replace preparedStatement for Trino-jdbc
1a8f147a0 [yehere] [KYUUBI #4325] Support replace preparedStatement for Trino-jdbc
Authored-by: yehere <867171931@qq.com>
Signed-off-by: ulyssesyou <ulyssesyou@apache.org>
### _Why are the changes needed?_
Close#4609
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4610 from lightning-L/kyuubi-4609.
Closes#4609
c40fd71ed [Tianlin Liao] fix test case
8f2d96cf0 [Tianlin Liao] [KYUUBI #4609] get engineRefId in KyuubiConnection
Authored-by: Tianlin Liao <tiliao@ebay.com>
Signed-off-by: fwang12 <fwang12@ebay.com>
### _Why are the changes needed?_
A typical use case of Hadoop UGI w/ Kyuubi Hive JDBC is
```
UserGroupInformation ugi = UserGroupInformation.getCurrentUser();
ugi.doAs(() -> {
Connection conn = DriverManager.getConnection(
"jdbc:kyuubi://host:10009/default;principal=kyuubi_HOST/ABC.ORG");
...
});
```
After https://github.com/apache/kyuubi/pull/3023, Kyuubi Hive JDBC implements the Kerberos authentication by using JDK directly instead of Hadoop `UserGroupInformation`, but it also introduce a breaking change for Hadoop users, including the above case. As workaround, user should add `kerberosAuthType=fromSubject` alongside w/ `principal=kyuubi_HOST/ABC.ORG` to make it work.
This PR propose to restore the behavior before https://github.com/apache/kyuubi/pull/3023 by handling UGI.doAs explicitly.
And this PR makes the `clientPrincipal` `clientKeytab` as the highest priority, so in below cases, `clientPrincipal` `clientKeytab` take effects instead of UGI.
```
UserGroupInformation ugi = UserGroupInformation.getCurrentUser();
ugi.doAs(() -> {
Connection conn = DriverManager.getConnection(
"jdbc:kyuubi://host:10009/default;principal=kyuubi_HOST/ABC.ORG;" +
"clientPrincipal=tom_HOST/ABC.ORG;clientKeytab=/path/xxx.keytab");
...
});
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4479 from pan3793/detect-ugi.
Closes#4479
0e169abc6 [Cheng Pan] nit
19036e3d7 [Cheng Pan] reorder
e8faf9c56 [Cheng Pan] Restore JDBC kerberos authentication behavior for UGI.doAs
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: fwang12 <fwang12@ebay.com>
### _Why are the changes needed?_
As described in the discussion: <https://github.com/apache/kyuubi/discussions/4436>
When the zk namespace contains leading and trailing slashes, the exception will cause the client connection to fail, and the exception prompt information is not clear.
So we strip the redundant leading and tailing slashes of getZooKeeperNamespace and add exception stack information at the same time.
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4454 from merrily01/zk-namespace.
Closes#4452
62290545e [maruilei] Address comments.
cee43cd61 [maruilei] [KYUUBI #4452] Strip the redundant leading and tailing slash of getZooKeeperNamespace.
Authored-by: maruilei <maruilei@58.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
