[KYUUBI #6117] Bump Kyuubi Shaded 0.3.0

# 🔍 Description
## Issue References 🔗

This PR upgrades Kyuubi Shaded to 0.3.0, the major changes are:

- upgrade Thrift from 0.9.3-1 to 0.16.0
- upgrade JLine used by Zookeeper CLI from 0.9.94 to 2.14.6
- split kyuubi-relocated-thrift from kyuubi-relocated-hive-service-rpc
- introduce a light kyuubi-relocated-hive-metastore-client for getting DelegationToken

Due to THRIFT-5297 (fixed in 0.14.0) removes requestTimeout and beBackoffSlotLength, the following Kyuubi configurations are removed too.

- kyuubi.frontend.login.timeout
- kyuubi.frontend.thrift.login.timeout
- kyuubi.frontend.backoff.slot.length
- kyuubi.frontend.thrift.backoff.slot.length

Previously, Zookeeper 3.4 depends on JLine 0.9.94 while Zookeeper 3.6 and Hive Beeline depends on JLine 2.14.6, we pulls different versions of JLine into the binary distribution. Now we align them to a single JLine 2.14.6.

## Describe Your Solution 🔧

Upgrade dependencies and change code to adapt to the breaking changes, also upgrade migration guide to mention the removed configurations.

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

Pass GA.

---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6117 from pan3793/shaded-0.3.

Closes #6117

5639c9fd8 [Cheng Pan] nit
d281cdb4b [Cheng Pan] fix
184e1b95b [Cheng Pan] fix
1628337c8 [Cheng Pan] remove unused conf
24db2d5ad [Cheng Pan] try fix
1e995bb34 [Cheng Pan] nit
1d8e9bce9 [Cheng Pan] dep list
7f33624e8 [Cheng Pan] Bump Kyuubi Shaded 0.3.0

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
This commit is contained in:
Cheng Pan 2024-03-08 10:19:30 +08:00
parent 8cc9b98e25
commit 182f4349a7
No known key found for this signature in database
GPG Key ID: 8001952629BCC75D
18 changed files with 58 additions and 66 deletions

View File

@ -105,7 +105,7 @@ jetty-server/9.4.52.v20230823//jetty-server-9.4.52.v20230823.jar
jetty-servlet/9.4.52.v20230823//jetty-servlet-9.4.52.v20230823.jar jetty-servlet/9.4.52.v20230823//jetty-servlet-9.4.52.v20230823.jar
jetty-util-ajax/9.4.52.v20230823//jetty-util-ajax-9.4.52.v20230823.jar jetty-util-ajax/9.4.52.v20230823//jetty-util-ajax-9.4.52.v20230823.jar
jetty-util/9.4.52.v20230823//jetty-util-9.4.52.v20230823.jar jetty-util/9.4.52.v20230823//jetty-util-9.4.52.v20230823.jar
jline/0.9.94//jline-0.9.94.jar jline/2.14.6//jline-2.14.6.jar
jul-to-slf4j/1.7.36//jul-to-slf4j-1.7.36.jar jul-to-slf4j/1.7.36//jul-to-slf4j-1.7.36.jar
kafka-clients/3.5.1//kafka-clients-3.5.1.jar kafka-clients/3.5.1//kafka-clients-3.5.1.jar
kubernetes-client-api/6.8.1//kubernetes-client-api-6.8.1.jar kubernetes-client-api/6.8.1//kubernetes-client-api-6.8.1.jar

View File

@ -230,11 +230,9 @@ You can configure the Kyuubi properties in `$KYUUBI_HOME/conf/kyuubi-defaults.co
| Key | Default | Meaning | Type | Since | | Key | Default | Meaning | Type | Since |
|--------------------------------------------------------|--------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|-------| |--------------------------------------------------------|--------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|-------|
| kyuubi.frontend.advertised.host | &lt;undefined&gt; | Hostname or IP of the Kyuubi server's frontend services to publish to external systems such as the service discovery ensemble and metadata store. Use it when you want to advertise a different hostname or IP than the bind host. | string | 1.8.0 | | kyuubi.frontend.advertised.host | &lt;undefined&gt; | Hostname or IP of the Kyuubi server's frontend services to publish to external systems such as the service discovery ensemble and metadata store. Use it when you want to advertise a different hostname or IP than the bind host. | string | 1.8.0 |
| kyuubi.frontend.backoff.slot.length | PT0.1S | (deprecated) Time to back off during login to the thrift frontend service. | duration | 1.0.0 |
| kyuubi.frontend.bind.host | &lt;undefined&gt; | Hostname or IP of the machine on which to run the frontend services. | string | 1.0.0 | | kyuubi.frontend.bind.host | &lt;undefined&gt; | Hostname or IP of the machine on which to run the frontend services. | string | 1.0.0 |
| kyuubi.frontend.bind.port | 10009 | (deprecated) Port of the machine on which to run the thrift frontend service via the binary protocol. | int | 1.0.0 | | kyuubi.frontend.bind.port | 10009 | (deprecated) Port of the machine on which to run the thrift frontend service via the binary protocol. | int | 1.0.0 |
| kyuubi.frontend.connection.url.use.hostname | true | When true, frontend services prefer hostname, otherwise, ip address. Note that, the default value is set to `false` when engine running on Kubernetes to prevent potential network issues. | boolean | 1.5.0 | | kyuubi.frontend.connection.url.use.hostname | true | When true, frontend services prefer hostname, otherwise, ip address. Note that, the default value is set to `false` when engine running on Kubernetes to prevent potential network issues. | boolean | 1.5.0 |
| kyuubi.frontend.login.timeout | PT20S | (deprecated) Timeout for Thrift clients during login to the thrift frontend service. | duration | 1.0.0 |
| kyuubi.frontend.max.message.size | 104857600 | (deprecated) Maximum message size in bytes a Kyuubi server will accept. | int | 1.0.0 | | kyuubi.frontend.max.message.size | 104857600 | (deprecated) Maximum message size in bytes a Kyuubi server will accept. | int | 1.0.0 |
| kyuubi.frontend.max.worker.threads | 999 | (deprecated) Maximum number of threads in the frontend worker thread pool for the thrift frontend service | int | 1.0.0 | | kyuubi.frontend.max.worker.threads | 999 | (deprecated) Maximum number of threads in the frontend worker thread pool for the thrift frontend service | int | 1.0.0 |
| kyuubi.frontend.min.worker.threads | 9 | (deprecated) Minimum number of threads in the frontend worker thread pool for the thrift frontend service | int | 1.0.0 | | kyuubi.frontend.min.worker.threads | 9 | (deprecated) Minimum number of threads in the frontend worker thread pool for the thrift frontend service | int | 1.0.0 |
@ -254,7 +252,6 @@ You can configure the Kyuubi properties in `$KYUUBI_HOME/conf/kyuubi-defaults.co
| kyuubi.frontend.ssl.keystore.password | &lt;undefined&gt; | SSL certificate keystore password. | string | 1.7.0 | | kyuubi.frontend.ssl.keystore.password | &lt;undefined&gt; | SSL certificate keystore password. | string | 1.7.0 |
| kyuubi.frontend.ssl.keystore.path | &lt;undefined&gt; | SSL certificate keystore location. | string | 1.7.0 | | kyuubi.frontend.ssl.keystore.path | &lt;undefined&gt; | SSL certificate keystore location. | string | 1.7.0 |
| kyuubi.frontend.ssl.keystore.type | &lt;undefined&gt; | SSL certificate keystore type. | string | 1.7.0 | | kyuubi.frontend.ssl.keystore.type | &lt;undefined&gt; | SSL certificate keystore type. | string | 1.7.0 |
| kyuubi.frontend.thrift.backoff.slot.length | PT0.1S | Time to back off during login to the thrift frontend service. | duration | 1.4.0 |
| kyuubi.frontend.thrift.binary.bind.host | &lt;undefined&gt; | Hostname or IP of the machine on which to run the thrift frontend service via the binary protocol. | string | 1.4.0 | | kyuubi.frontend.thrift.binary.bind.host | &lt;undefined&gt; | Hostname or IP of the machine on which to run the thrift frontend service via the binary protocol. | string | 1.4.0 |
| kyuubi.frontend.thrift.binary.bind.port | 10009 | Port of the machine on which to run the thrift frontend service via the binary protocol. | int | 1.4.0 | | kyuubi.frontend.thrift.binary.bind.port | 10009 | Port of the machine on which to run the thrift frontend service via the binary protocol. | int | 1.4.0 |
| kyuubi.frontend.thrift.binary.ssl.disallowed.protocols | SSLv2,SSLv3 | SSL versions to disallow for Kyuubi thrift binary frontend. | set | 1.7.0 | | kyuubi.frontend.thrift.binary.ssl.disallowed.protocols | SSLv2,SSLv3 | SSL versions to disallow for Kyuubi thrift binary frontend. | set | 1.7.0 |
@ -278,7 +275,6 @@ You can configure the Kyuubi properties in `$KYUUBI_HOME/conf/kyuubi-defaults.co
| kyuubi.frontend.thrift.http.ssl.protocol.blacklist | SSLv2,SSLv3 | SSL Versions to disable when using HTTP transport mode. | seq | 1.6.0 | | kyuubi.frontend.thrift.http.ssl.protocol.blacklist | SSLv2,SSLv3 | SSL Versions to disable when using HTTP transport mode. | seq | 1.6.0 |
| kyuubi.frontend.thrift.http.use.SSL | false | Set this to true for using SSL encryption in http mode. | boolean | 1.6.0 | | kyuubi.frontend.thrift.http.use.SSL | false | Set this to true for using SSL encryption in http mode. | boolean | 1.6.0 |
| kyuubi.frontend.thrift.http.xsrf.filter.enabled | false | If enabled, Kyuubi will block any requests made to it over HTTP if an X-XSRF-HEADER header is not present | boolean | 1.6.0 | | kyuubi.frontend.thrift.http.xsrf.filter.enabled | false | If enabled, Kyuubi will block any requests made to it over HTTP if an X-XSRF-HEADER header is not present | boolean | 1.6.0 |
| kyuubi.frontend.thrift.login.timeout | PT20S | Timeout for Thrift clients during login to the thrift frontend service. | duration | 1.4.0 |
| kyuubi.frontend.thrift.max.message.size | 104857600 | Maximum message size in bytes a Kyuubi server will accept. | int | 1.4.0 | | kyuubi.frontend.thrift.max.message.size | 104857600 | Maximum message size in bytes a Kyuubi server will accept. | int | 1.4.0 |
| kyuubi.frontend.thrift.max.worker.threads | 999 | Maximum number of threads in the frontend worker thread pool for the thrift frontend service | int | 1.4.0 | | kyuubi.frontend.thrift.max.worker.threads | 999 | Maximum number of threads in the frontend worker thread pool for the thrift frontend service | int | 1.4.0 |
| kyuubi.frontend.thrift.min.worker.threads | 9 | Minimum number of threads in the frontend worker thread pool for the thrift frontend service | int | 1.4.0 | | kyuubi.frontend.thrift.min.worker.threads | 9 | Minimum number of threads in the frontend worker thread pool for the thrift frontend service | int | 1.4.0 |

View File

@ -23,6 +23,7 @@
* Since Kyuubi 1.9.0, the support of Derby is removal for Kyuubi metastore. * Since Kyuubi 1.9.0, the support of Derby is removal for Kyuubi metastore.
* Since Kyuubi 1.9.0, the support of Spark SQL engine for Spark 3.1 is deprecated, and will be removed in the future. * Since Kyuubi 1.9.0, the support of Spark SQL engine for Spark 3.1 is deprecated, and will be removed in the future.
* Since Kyuubi 1.9.0, the support of Spark extensions for Spark 3.1 is removed, please use Spark 3.2 or higher versions. * Since Kyuubi 1.9.0, the support of Spark extensions for Spark 3.1 is removed, please use Spark 3.2 or higher versions.
* Since Kyuubi 1.9.0, `kyuubi.frontend.login.timeout`, `kyuubi.frontend.thrift.login.timeout`, `kyuubi.frontend.backoff.slot.length`, `kyuubi.frontend.thrift.backoff.slot.length` are removed.
## Upgrading from Kyuubi 1.8.0 to 1.8.1 ## Upgrading from Kyuubi 1.8.0 to 1.8.1

View File

@ -98,6 +98,11 @@
<artifactId>commons-lang3</artifactId> <artifactId>commons-lang3</artifactId>
</dependency> </dependency>
<dependency>
<groupId>org.apache.kyuubi</groupId>
<artifactId>kyuubi-relocated-thrift</artifactId>
</dependency>
<dependency> <dependency>
<groupId>org.apache.kyuubi</groupId> <groupId>org.apache.kyuubi</groupId>
<artifactId>kyuubi-relocated-hive-service-rpc</artifactId> <artifactId>kyuubi-relocated-hive-service-rpc</artifactId>

View File

@ -605,34 +605,6 @@ object KyuubiConf {
.version("1.4.0") .version("1.4.0")
.fallbackConf(FRONTEND_MAX_MESSAGE_SIZE) .fallbackConf(FRONTEND_MAX_MESSAGE_SIZE)
@deprecated("using kyuubi.frontend.thrift.login.timeout instead", "1.4.0")
val FRONTEND_LOGIN_TIMEOUT: ConfigEntry[Long] =
buildConf("kyuubi.frontend.login.timeout")
.doc("(deprecated) Timeout for Thrift clients during login to the thrift frontend service.")
.version("1.0.0")
.timeConf
.createWithDefault(Duration.ofSeconds(20).toMillis)
val FRONTEND_THRIFT_LOGIN_TIMEOUT: ConfigEntry[Long] =
buildConf("kyuubi.frontend.thrift.login.timeout")
.doc("Timeout for Thrift clients during login to the thrift frontend service.")
.version("1.4.0")
.fallbackConf(FRONTEND_LOGIN_TIMEOUT)
@deprecated("using kyuubi.frontend.thrift.backoff.slot.length instead", "1.4.0")
val FRONTEND_LOGIN_BACKOFF_SLOT_LENGTH: ConfigEntry[Long] =
buildConf("kyuubi.frontend.backoff.slot.length")
.doc("(deprecated) Time to back off during login to the thrift frontend service.")
.version("1.0.0")
.timeConf
.createWithDefault(Duration.ofMillis(100).toMillis)
val FRONTEND_THRIFT_LOGIN_BACKOFF_SLOT_LENGTH: ConfigEntry[Long] =
buildConf("kyuubi.frontend.thrift.backoff.slot.length")
.doc("Time to back off during login to the thrift frontend service.")
.version("1.4.0")
.fallbackConf(FRONTEND_LOGIN_BACKOFF_SLOT_LENGTH)
val FRONTEND_THRIFT_HTTP_REQUEST_HEADER_SIZE: ConfigEntry[Int] = val FRONTEND_THRIFT_HTTP_REQUEST_HEADER_SIZE: ConfigEntry[Int] =
buildConf("kyuubi.frontend.thrift.http.request.header.size") buildConf("kyuubi.frontend.thrift.http.request.header.size")
.doc("Request header size in bytes, when using HTTP transport mode. Jetty defaults used.") .doc("Request header size in bytes, when using HTTP transport mode. Jetty defaults used.")
@ -3144,14 +3116,6 @@ object KyuubiConf {
FRONTEND_MAX_MESSAGE_SIZE.key, FRONTEND_MAX_MESSAGE_SIZE.key,
"1.4.0", "1.4.0",
s"Use ${FRONTEND_THRIFT_MAX_MESSAGE_SIZE.key} instead"), s"Use ${FRONTEND_THRIFT_MAX_MESSAGE_SIZE.key} instead"),
DeprecatedConfig(
FRONTEND_LOGIN_TIMEOUT.key,
"1.4.0",
s"Use ${FRONTEND_THRIFT_LOGIN_TIMEOUT.key} instead"),
DeprecatedConfig(
FRONTEND_LOGIN_BACKOFF_SLOT_LENGTH.key,
"1.4.0",
s"Use ${FRONTEND_THRIFT_LOGIN_BACKOFF_SLOT_LENGTH.key} instead"),
DeprecatedConfig( DeprecatedConfig(
SESSION_TIMEOUT.key, SESSION_TIMEOUT.key,
"1.2.0", "1.2.0",

View File

@ -101,17 +101,13 @@ abstract class TBinaryFrontendService(name: String)
} }
_actualPort = tServerSocket.getServerSocket.getLocalPort _actualPort = tServerSocket.getServerSocket.getLocalPort
val maxMessageSize = conf.get(FRONTEND_THRIFT_MAX_MESSAGE_SIZE) val maxMessageSize = conf.get(FRONTEND_THRIFT_MAX_MESSAGE_SIZE)
val requestTimeout = conf.get(FRONTEND_THRIFT_LOGIN_TIMEOUT).toInt
val beBackoffSlotLength = conf.get(FRONTEND_THRIFT_LOGIN_BACKOFF_SLOT_LENGTH).toInt
val args = new TThreadPoolServer.Args(tServerSocket) val args = new TThreadPoolServer.Args(tServerSocket)
.processorFactory(tProcFactory) .processorFactory(tProcFactory)
.transportFactory(transFactory) .transportFactory(transFactory)
.protocolFactory(new TBinaryProtocol.Factory) .protocolFactory(new TBinaryProtocol.Factory)
.inputProtocolFactory( .inputProtocolFactory(
new TBinaryProtocol.Factory(true, true, maxMessageSize, maxMessageSize)) new TBinaryProtocol.Factory(true, true, maxMessageSize, maxMessageSize))
.requestTimeout(requestTimeout).requestTimeoutUnit(TimeUnit.MILLISECONDS) // THRIFT-5297 (fixed in 0.14.0) removes requestTimeout and beBackoffSlotLength
.beBackoffSlotLength(beBackoffSlotLength)
.beBackoffSlotLengthUnit(TimeUnit.MILLISECONDS)
.executorService(executor) .executorService(executor)
// TCP Server // TCP Server
server = Some(new TThreadPoolServer(args)) server = Some(new TThreadPoolServer(args))

View File

@ -649,5 +649,9 @@ private[kyuubi] object TFrontendService {
} }
def getSessionHandle: SessionHandle = sessionHandle def getSessionHandle: SessionHandle = sessionHandle
override def unwrap[T](aClass: Class[T]): T = null.asInstanceOf[T]
override def isWrapperFor(aClass: Class[_]): Boolean = false
} }
} }

View File

@ -131,7 +131,7 @@ object HadoopThriftAuthBridgeServer {
class TUGIAssumingProcessor( class TUGIAssumingProcessor(
wrapped: TProcessor, wrapped: TProcessor,
secretMgr: KyuubiDelegationTokenManager) extends TProcessor with Logging { secretMgr: KyuubiDelegationTokenManager) extends TProcessor with Logging {
override def process(in: TProtocol, out: TProtocol): Boolean = { override def process(in: TProtocol, out: TProtocol): Unit = {
val transport = in.getTransport val transport = in.getTransport
transport match { transport match {
case saslTrans: TSaslServerTransport => case saslTrans: TSaslServerTransport =>

View File

@ -28,7 +28,7 @@ class TSetIpAddressProcessor[I <: Iface](
import TSetIpAddressProcessor._ import TSetIpAddressProcessor._
@throws[TException] @throws[TException]
override def process(in: TProtocol, out: TProtocol): Boolean = { override def process(in: TProtocol, out: TProtocol): Unit = {
setIpAddress(in) setIpAddress(in)
setUserName(in) setUserName(in)
try { try {

View File

@ -135,13 +135,6 @@
<dependency> <dependency>
<groupId>jline</groupId> <groupId>jline</groupId>
<artifactId>jline</artifactId> <artifactId>jline</artifactId>
<version>${hive.client.jline.version}</version>
<exclusions>
<exclusion>
<groupId>*</groupId>
<artifactId>*</artifactId>
</exclusion>
</exclusions>
</dependency> </dependency>
<dependency> <dependency>

View File

@ -97,6 +97,11 @@
<artifactId>httpclient</artifactId> <artifactId>httpclient</artifactId>
</dependency> </dependency>
<dependency>
<groupId>org.apache.kyuubi</groupId>
<artifactId>kyuubi-relocated-thrift</artifactId>
</dependency>
<dependency> <dependency>
<groupId>org.apache.kyuubi</groupId> <groupId>org.apache.kyuubi</groupId>
<artifactId>kyuubi-relocated-hive-service-rpc</artifactId> <artifactId>kyuubi-relocated-hive-service-rpc</artifactId>

View File

@ -22,6 +22,7 @@ import javax.security.auth.Subject;
import javax.security.sasl.SaslException; import javax.security.sasl.SaslException;
import org.apache.kyuubi.shaded.thrift.transport.TSaslClientTransport; import org.apache.kyuubi.shaded.thrift.transport.TSaslClientTransport;
import org.apache.kyuubi.shaded.thrift.transport.TTransport; import org.apache.kyuubi.shaded.thrift.transport.TTransport;
import org.apache.kyuubi.shaded.thrift.transport.TTransportException;
import org.slf4j.Logger; import org.slf4j.Logger;
import org.slf4j.LoggerFactory; import org.slf4j.LoggerFactory;
@ -34,7 +35,7 @@ public final class KerberosSaslHelper {
String host, String host,
TTransport underlyingTransport, TTransport underlyingTransport,
Map<String, String> saslProps) Map<String, String> saslProps)
throws SaslException { throws SaslException, TTransportException {
String resolvedPrincipal = KerberosUtils.canonicalPrincipal(serverPrincipal, host); String resolvedPrincipal = KerberosUtils.canonicalPrincipal(serverPrincipal, host);
String[] names = KerberosUtils.splitPrincipal(resolvedPrincipal); String[] names = KerberosUtils.splitPrincipal(resolvedPrincipal);
TTransport saslTransport = TTransport saslTransport =

View File

@ -22,11 +22,13 @@ import javax.security.auth.callback.*;
import javax.security.sasl.SaslException; import javax.security.sasl.SaslException;
import org.apache.kyuubi.shaded.thrift.transport.TSaslClientTransport; import org.apache.kyuubi.shaded.thrift.transport.TSaslClientTransport;
import org.apache.kyuubi.shaded.thrift.transport.TTransport; import org.apache.kyuubi.shaded.thrift.transport.TTransport;
import org.apache.kyuubi.shaded.thrift.transport.TTransportException;
public final class PlainSaslHelper { public final class PlainSaslHelper {
public static TTransport getPlainTransport( public static TTransport getPlainTransport(
String username, String password, TTransport underlyingTransport) throws SaslException { String username, String password, TTransport underlyingTransport)
throws SaslException, TTransportException {
return new TSaslClientTransport( return new TSaslClientTransport(
"PLAIN", "PLAIN",
null, null,

View File

@ -17,6 +17,7 @@
package org.apache.kyuubi.jdbc.hive.auth; package org.apache.kyuubi.jdbc.hive.auth;
import org.apache.kyuubi.shaded.thrift.TConfiguration;
import org.apache.kyuubi.shaded.thrift.transport.TTransport; import org.apache.kyuubi.shaded.thrift.transport.TTransport;
import org.apache.kyuubi.shaded.thrift.transport.TTransportException; import org.apache.kyuubi.shaded.thrift.transport.TTransportException;
@ -95,4 +96,19 @@ public class TFilterTransport extends TTransport {
public void consumeBuffer(int len) { public void consumeBuffer(int len) {
wrapped.consumeBuffer(len); wrapped.consumeBuffer(len);
} }
@Override
public TConfiguration getConfiguration() {
return wrapped.getConfiguration();
}
@Override
public void updateKnownMessageSize(long l) throws TTransportException {
wrapped.updateKnownMessageSize(l);
}
@Override
public void checkReadBytesAvailable(long l) throws TTransportException {
wrapped.checkReadBytesAvailable(l);
}
} }

View File

@ -19,6 +19,7 @@ package org.apache.kyuubi.jdbc.hive.auth;
import javax.net.ssl.SSLParameters; import javax.net.ssl.SSLParameters;
import javax.net.ssl.SSLSocket; import javax.net.ssl.SSLSocket;
import org.apache.kyuubi.shaded.thrift.TConfiguration;
import org.apache.kyuubi.shaded.thrift.transport.TSSLTransportFactory; import org.apache.kyuubi.shaded.thrift.transport.TSSLTransportFactory;
import org.apache.kyuubi.shaded.thrift.transport.TSocket; import org.apache.kyuubi.shaded.thrift.transport.TSocket;
import org.apache.kyuubi.shaded.thrift.transport.TTransport; import org.apache.kyuubi.shaded.thrift.transport.TTransport;
@ -30,8 +31,8 @@ import org.apache.kyuubi.shaded.thrift.transport.TTransportException;
*/ */
public class ThriftUtils { public class ThriftUtils {
public static TTransport getSocketTransport( public static TTransport getSocketTransport(
String host, int port, int connectTimeout, int socketTimeout) { String host, int port, int connectTimeout, int socketTimeout) throws TTransportException {
return new TSocket(host, port, socketTimeout, connectTimeout); return new TSocket(TConfiguration.DEFAULT, host, port, socketTimeout, connectTimeout);
} }
public static TTransport getSSLSocket( public static TTransport getSSLSocket(

View File

@ -36,6 +36,7 @@ import org.apache.kyuubi.operation.FetchOrientation.FetchOrientation
import org.apache.kyuubi.service.authentication.PlainSASLHelper import org.apache.kyuubi.service.authentication.PlainSASLHelper
import org.apache.kyuubi.session.SessionHandle import org.apache.kyuubi.session.SessionHandle
import org.apache.kyuubi.shaded.hive.service.rpc.thrift._ import org.apache.kyuubi.shaded.hive.service.rpc.thrift._
import org.apache.kyuubi.shaded.thrift.TConfiguration
import org.apache.kyuubi.shaded.thrift.protocol.{TBinaryProtocol, TProtocol} import org.apache.kyuubi.shaded.thrift.protocol.{TBinaryProtocol, TProtocol}
import org.apache.kyuubi.shaded.thrift.transport.TSocket import org.apache.kyuubi.shaded.thrift.transport.TSocket
import org.apache.kyuubi.util.{ThreadUtils, ThriftUtils} import org.apache.kyuubi.util.{ThreadUtils, ThriftUtils}
@ -457,7 +458,7 @@ private[kyuubi] object KyuubiSyncThriftClient extends Logging {
port: Int, port: Int,
socketTimeout: Int, socketTimeout: Int,
connectionTimeout: Int): TProtocol = { connectionTimeout: Int): TProtocol = {
val tSocket = new TSocket(host, port, socketTimeout, connectionTimeout) val tSocket = new TSocket(TConfiguration.DEFAULT, host, port, socketTimeout, connectionTimeout)
val tTransport = PlainSASLHelper.getPlainTransport(user, passwd, tSocket) val tTransport = PlainSASLHelper.getPlainTransport(user, passwd, tSocket)
tTransport.open() tTransport.open()
new TBinaryProtocol(tTransport) new TBinaryProtocol(tTransport)

View File

@ -80,7 +80,9 @@ class KyuubiOperationPerConnectionSuite extends WithKyuubiServer with HiveJDBCTe
assert(executeStmtResp.getOperationHandle === null) assert(executeStmtResp.getOperationHandle === null)
val errMsg = executeStmtResp.getStatus.getErrorMessage val errMsg = executeStmtResp.getStatus.getErrorMessage
assert(errMsg.contains("Caused by: java.net.SocketException: Connection reset") || assert(errMsg.contains("Caused by: java.net.SocketException: Connection reset") ||
errMsg.contains(s"Socket for ${SessionHandle(handle)} is closed")) errMsg.contains(s"Socket for ${SessionHandle(handle)} is closed") ||
errMsg.contains("Socket is closed by peer") ||
errMsg.contains("SparkContext was shut down"))
} }
} }
@ -332,9 +334,10 @@ class KyuubiOperationPerConnectionSuite extends WithKyuubiServer with HiveJDBCTe
assert(executeStmtResp.getStatus.getStatusCode === TStatusCode.ERROR_STATUS) assert(executeStmtResp.getStatus.getStatusCode === TStatusCode.ERROR_STATUS)
val errorMsg = executeStmtResp.getStatus.getErrorMessage val errorMsg = executeStmtResp.getStatus.getErrorMessage
assert(errorMsg.contains("java.net.SocketException") || assert(errorMsg.contains("java.net.SocketException") ||
errorMsg.contains("org.apache.thrift.transport.TTransportException") || errorMsg.contains("org.apache.kyuubi.shaded.thrift.transport.TTransportException") ||
errorMsg.contains("connection does not exist") || errorMsg.contains("connection does not exist") ||
errorMsg.contains(s"Socket for ${SessionHandle(handle)} is closed")) errorMsg.contains(s"Socket for ${SessionHandle(handle)} is closed") ||
errorMsg.contains("Error submitting query in background, query rejected"))
val elapsedTime = System.currentTimeMillis() - startTime val elapsedTime = System.currentTimeMillis() - startTime
assert(elapsedTime < 20 * 1000) assert(elapsedTime < 20 * 1000)
eventually(timeout(3.seconds)) { eventually(timeout(3.seconds)) {

10
pom.xml
View File

@ -169,11 +169,11 @@
<jersey.version>2.39.1</jersey.version> <jersey.version>2.39.1</jersey.version>
<jetcd.version>0.7.7</jetcd.version> <jetcd.version>0.7.7</jetcd.version>
<jetty.version>9.4.52.v20230823</jetty.version> <jetty.version>9.4.52.v20230823</jetty.version>
<jline.version>0.9.94</jline.version> <jline.version>2.14.6</jline.version>
<junit.version>4.13.2</junit.version> <junit.version>4.13.2</junit.version>
<kafka.version>3.5.1</kafka.version> <kafka.version>3.5.1</kafka.version>
<kubernetes-client.version>6.8.1</kubernetes-client.version> <kubernetes-client.version>6.8.1</kubernetes-client.version>
<kyuubi-relocated.version>0.2.0</kyuubi-relocated.version> <kyuubi-relocated.version>0.3.0</kyuubi-relocated.version>
<kyuubi-relocated-zookeeper.artifacts>kyuubi-relocated-zookeeper-34</kyuubi-relocated-zookeeper.artifacts> <kyuubi-relocated-zookeeper.artifacts>kyuubi-relocated-zookeeper-34</kyuubi-relocated-zookeeper.artifacts>
<ldapsdk.version>6.0.5</ldapsdk.version> <ldapsdk.version>6.0.5</ldapsdk.version>
<log4j.version>2.20.0</log4j.version> <log4j.version>2.20.0</log4j.version>
@ -223,7 +223,6 @@
<pnpm.version>v8.6.1</pnpm.version> <pnpm.version>v8.6.1</pnpm.version>
<!-- apply to kyuubi-hive-jdbc/kyuubi-hive-beeline module --> <!-- apply to kyuubi-hive-jdbc/kyuubi-hive-beeline module -->
<hive.client.jline.version>2.12</hive.client.jline.version>
<hive.client.supercsv.version>2.2.0</hive.client.supercsv.version> <hive.client.supercsv.version>2.2.0</hive.client.supercsv.version>
<hive.jdbc.artifact>kyuubi-hive-jdbc</hive.jdbc.artifact> <hive.jdbc.artifact>kyuubi-hive-jdbc</hive.jdbc.artifact>
@ -279,6 +278,11 @@
<dependencyManagement> <dependencyManagement>
<dependencies> <dependencies>
<dependency>
<groupId>org.apache.kyuubi</groupId>
<artifactId>kyuubi-relocated-thrift</artifactId>
<version>${kyuubi-relocated.version}</version>
</dependency>
<dependency> <dependency>
<groupId>org.apache.kyuubi</groupId> <groupId>org.apache.kyuubi</groupId>
<artifactId>kyuubi-relocated-hive-service-rpc</artifactId> <artifactId>kyuubi-relocated-hive-service-rpc</artifactId>