[KYUUBI #6117] Bump Kyuubi Shaded 0.3.0
# 🔍 Description ## Issue References 🔗 This PR upgrades Kyuubi Shaded to 0.3.0, the major changes are: - upgrade Thrift from 0.9.3-1 to 0.16.0 - upgrade JLine used by Zookeeper CLI from 0.9.94 to 2.14.6 - split kyuubi-relocated-thrift from kyuubi-relocated-hive-service-rpc - introduce a light kyuubi-relocated-hive-metastore-client for getting DelegationToken Due to THRIFT-5297 (fixed in 0.14.0) removes requestTimeout and beBackoffSlotLength, the following Kyuubi configurations are removed too. - kyuubi.frontend.login.timeout - kyuubi.frontend.thrift.login.timeout - kyuubi.frontend.backoff.slot.length - kyuubi.frontend.thrift.backoff.slot.length Previously, Zookeeper 3.4 depends on JLine 0.9.94 while Zookeeper 3.6 and Hive Beeline depends on JLine 2.14.6, we pulls different versions of JLine into the binary distribution. Now we align them to a single JLine 2.14.6. ## Describe Your Solution 🔧 Upgrade dependencies and change code to adapt to the breaking changes, also upgrade migration guide to mention the removed configurations. ## Types of changes 🔖 - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) ## Test Plan 🧪 Pass GA. --- # Checklist 📝 - [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html) **Be nice. Be informative.** Closes #6117 from pan3793/shaded-0.3. Closes #6117 5639c9fd8 [Cheng Pan] nit d281cdb4b [Cheng Pan] fix 184e1b95b [Cheng Pan] fix 1628337c8 [Cheng Pan] remove unused conf 24db2d5ad [Cheng Pan] try fix 1e995bb34 [Cheng Pan] nit 1d8e9bce9 [Cheng Pan] dep list 7f33624e8 [Cheng Pan] Bump Kyuubi Shaded 0.3.0 Authored-by: Cheng Pan <chengpan@apache.org> Signed-off-by: Cheng Pan <chengpan@apache.org>
This commit is contained in:
parent
8cc9b98e25
commit
182f4349a7
@ -105,7 +105,7 @@ jetty-server/9.4.52.v20230823//jetty-server-9.4.52.v20230823.jar
|
|||||||
jetty-servlet/9.4.52.v20230823//jetty-servlet-9.4.52.v20230823.jar
|
jetty-servlet/9.4.52.v20230823//jetty-servlet-9.4.52.v20230823.jar
|
||||||
jetty-util-ajax/9.4.52.v20230823//jetty-util-ajax-9.4.52.v20230823.jar
|
jetty-util-ajax/9.4.52.v20230823//jetty-util-ajax-9.4.52.v20230823.jar
|
||||||
jetty-util/9.4.52.v20230823//jetty-util-9.4.52.v20230823.jar
|
jetty-util/9.4.52.v20230823//jetty-util-9.4.52.v20230823.jar
|
||||||
jline/0.9.94//jline-0.9.94.jar
|
jline/2.14.6//jline-2.14.6.jar
|
||||||
jul-to-slf4j/1.7.36//jul-to-slf4j-1.7.36.jar
|
jul-to-slf4j/1.7.36//jul-to-slf4j-1.7.36.jar
|
||||||
kafka-clients/3.5.1//kafka-clients-3.5.1.jar
|
kafka-clients/3.5.1//kafka-clients-3.5.1.jar
|
||||||
kubernetes-client-api/6.8.1//kubernetes-client-api-6.8.1.jar
|
kubernetes-client-api/6.8.1//kubernetes-client-api-6.8.1.jar
|
||||||
|
|||||||
@ -230,11 +230,9 @@ You can configure the Kyuubi properties in `$KYUUBI_HOME/conf/kyuubi-defaults.co
|
|||||||
| Key | Default | Meaning | Type | Since |
|
| Key | Default | Meaning | Type | Since |
|
||||||
|--------------------------------------------------------|--------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|-------|
|
|--------------------------------------------------------|--------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|-------|
|
||||||
| kyuubi.frontend.advertised.host | <undefined> | Hostname or IP of the Kyuubi server's frontend services to publish to external systems such as the service discovery ensemble and metadata store. Use it when you want to advertise a different hostname or IP than the bind host. | string | 1.8.0 |
|
| kyuubi.frontend.advertised.host | <undefined> | Hostname or IP of the Kyuubi server's frontend services to publish to external systems such as the service discovery ensemble and metadata store. Use it when you want to advertise a different hostname or IP than the bind host. | string | 1.8.0 |
|
||||||
| kyuubi.frontend.backoff.slot.length | PT0.1S | (deprecated) Time to back off during login to the thrift frontend service. | duration | 1.0.0 |
|
|
||||||
| kyuubi.frontend.bind.host | <undefined> | Hostname or IP of the machine on which to run the frontend services. | string | 1.0.0 |
|
| kyuubi.frontend.bind.host | <undefined> | Hostname or IP of the machine on which to run the frontend services. | string | 1.0.0 |
|
||||||
| kyuubi.frontend.bind.port | 10009 | (deprecated) Port of the machine on which to run the thrift frontend service via the binary protocol. | int | 1.0.0 |
|
| kyuubi.frontend.bind.port | 10009 | (deprecated) Port of the machine on which to run the thrift frontend service via the binary protocol. | int | 1.0.0 |
|
||||||
| kyuubi.frontend.connection.url.use.hostname | true | When true, frontend services prefer hostname, otherwise, ip address. Note that, the default value is set to `false` when engine running on Kubernetes to prevent potential network issues. | boolean | 1.5.0 |
|
| kyuubi.frontend.connection.url.use.hostname | true | When true, frontend services prefer hostname, otherwise, ip address. Note that, the default value is set to `false` when engine running on Kubernetes to prevent potential network issues. | boolean | 1.5.0 |
|
||||||
| kyuubi.frontend.login.timeout | PT20S | (deprecated) Timeout for Thrift clients during login to the thrift frontend service. | duration | 1.0.0 |
|
|
||||||
| kyuubi.frontend.max.message.size | 104857600 | (deprecated) Maximum message size in bytes a Kyuubi server will accept. | int | 1.0.0 |
|
| kyuubi.frontend.max.message.size | 104857600 | (deprecated) Maximum message size in bytes a Kyuubi server will accept. | int | 1.0.0 |
|
||||||
| kyuubi.frontend.max.worker.threads | 999 | (deprecated) Maximum number of threads in the frontend worker thread pool for the thrift frontend service | int | 1.0.0 |
|
| kyuubi.frontend.max.worker.threads | 999 | (deprecated) Maximum number of threads in the frontend worker thread pool for the thrift frontend service | int | 1.0.0 |
|
||||||
| kyuubi.frontend.min.worker.threads | 9 | (deprecated) Minimum number of threads in the frontend worker thread pool for the thrift frontend service | int | 1.0.0 |
|
| kyuubi.frontend.min.worker.threads | 9 | (deprecated) Minimum number of threads in the frontend worker thread pool for the thrift frontend service | int | 1.0.0 |
|
||||||
@ -254,7 +252,6 @@ You can configure the Kyuubi properties in `$KYUUBI_HOME/conf/kyuubi-defaults.co
|
|||||||
| kyuubi.frontend.ssl.keystore.password | <undefined> | SSL certificate keystore password. | string | 1.7.0 |
|
| kyuubi.frontend.ssl.keystore.password | <undefined> | SSL certificate keystore password. | string | 1.7.0 |
|
||||||
| kyuubi.frontend.ssl.keystore.path | <undefined> | SSL certificate keystore location. | string | 1.7.0 |
|
| kyuubi.frontend.ssl.keystore.path | <undefined> | SSL certificate keystore location. | string | 1.7.0 |
|
||||||
| kyuubi.frontend.ssl.keystore.type | <undefined> | SSL certificate keystore type. | string | 1.7.0 |
|
| kyuubi.frontend.ssl.keystore.type | <undefined> | SSL certificate keystore type. | string | 1.7.0 |
|
||||||
| kyuubi.frontend.thrift.backoff.slot.length | PT0.1S | Time to back off during login to the thrift frontend service. | duration | 1.4.0 |
|
|
||||||
| kyuubi.frontend.thrift.binary.bind.host | <undefined> | Hostname or IP of the machine on which to run the thrift frontend service via the binary protocol. | string | 1.4.0 |
|
| kyuubi.frontend.thrift.binary.bind.host | <undefined> | Hostname or IP of the machine on which to run the thrift frontend service via the binary protocol. | string | 1.4.0 |
|
||||||
| kyuubi.frontend.thrift.binary.bind.port | 10009 | Port of the machine on which to run the thrift frontend service via the binary protocol. | int | 1.4.0 |
|
| kyuubi.frontend.thrift.binary.bind.port | 10009 | Port of the machine on which to run the thrift frontend service via the binary protocol. | int | 1.4.0 |
|
||||||
| kyuubi.frontend.thrift.binary.ssl.disallowed.protocols | SSLv2,SSLv3 | SSL versions to disallow for Kyuubi thrift binary frontend. | set | 1.7.0 |
|
| kyuubi.frontend.thrift.binary.ssl.disallowed.protocols | SSLv2,SSLv3 | SSL versions to disallow for Kyuubi thrift binary frontend. | set | 1.7.0 |
|
||||||
@ -278,7 +275,6 @@ You can configure the Kyuubi properties in `$KYUUBI_HOME/conf/kyuubi-defaults.co
|
|||||||
| kyuubi.frontend.thrift.http.ssl.protocol.blacklist | SSLv2,SSLv3 | SSL Versions to disable when using HTTP transport mode. | seq | 1.6.0 |
|
| kyuubi.frontend.thrift.http.ssl.protocol.blacklist | SSLv2,SSLv3 | SSL Versions to disable when using HTTP transport mode. | seq | 1.6.0 |
|
||||||
| kyuubi.frontend.thrift.http.use.SSL | false | Set this to true for using SSL encryption in http mode. | boolean | 1.6.0 |
|
| kyuubi.frontend.thrift.http.use.SSL | false | Set this to true for using SSL encryption in http mode. | boolean | 1.6.0 |
|
||||||
| kyuubi.frontend.thrift.http.xsrf.filter.enabled | false | If enabled, Kyuubi will block any requests made to it over HTTP if an X-XSRF-HEADER header is not present | boolean | 1.6.0 |
|
| kyuubi.frontend.thrift.http.xsrf.filter.enabled | false | If enabled, Kyuubi will block any requests made to it over HTTP if an X-XSRF-HEADER header is not present | boolean | 1.6.0 |
|
||||||
| kyuubi.frontend.thrift.login.timeout | PT20S | Timeout for Thrift clients during login to the thrift frontend service. | duration | 1.4.0 |
|
|
||||||
| kyuubi.frontend.thrift.max.message.size | 104857600 | Maximum message size in bytes a Kyuubi server will accept. | int | 1.4.0 |
|
| kyuubi.frontend.thrift.max.message.size | 104857600 | Maximum message size in bytes a Kyuubi server will accept. | int | 1.4.0 |
|
||||||
| kyuubi.frontend.thrift.max.worker.threads | 999 | Maximum number of threads in the frontend worker thread pool for the thrift frontend service | int | 1.4.0 |
|
| kyuubi.frontend.thrift.max.worker.threads | 999 | Maximum number of threads in the frontend worker thread pool for the thrift frontend service | int | 1.4.0 |
|
||||||
| kyuubi.frontend.thrift.min.worker.threads | 9 | Minimum number of threads in the frontend worker thread pool for the thrift frontend service | int | 1.4.0 |
|
| kyuubi.frontend.thrift.min.worker.threads | 9 | Minimum number of threads in the frontend worker thread pool for the thrift frontend service | int | 1.4.0 |
|
||||||
|
|||||||
@ -23,6 +23,7 @@
|
|||||||
* Since Kyuubi 1.9.0, the support of Derby is removal for Kyuubi metastore.
|
* Since Kyuubi 1.9.0, the support of Derby is removal for Kyuubi metastore.
|
||||||
* Since Kyuubi 1.9.0, the support of Spark SQL engine for Spark 3.1 is deprecated, and will be removed in the future.
|
* Since Kyuubi 1.9.0, the support of Spark SQL engine for Spark 3.1 is deprecated, and will be removed in the future.
|
||||||
* Since Kyuubi 1.9.0, the support of Spark extensions for Spark 3.1 is removed, please use Spark 3.2 or higher versions.
|
* Since Kyuubi 1.9.0, the support of Spark extensions for Spark 3.1 is removed, please use Spark 3.2 or higher versions.
|
||||||
|
* Since Kyuubi 1.9.0, `kyuubi.frontend.login.timeout`, `kyuubi.frontend.thrift.login.timeout`, `kyuubi.frontend.backoff.slot.length`, `kyuubi.frontend.thrift.backoff.slot.length` are removed.
|
||||||
|
|
||||||
## Upgrading from Kyuubi 1.8.0 to 1.8.1
|
## Upgrading from Kyuubi 1.8.0 to 1.8.1
|
||||||
|
|
||||||
|
|||||||
@ -98,6 +98,11 @@
|
|||||||
<artifactId>commons-lang3</artifactId>
|
<artifactId>commons-lang3</artifactId>
|
||||||
</dependency>
|
</dependency>
|
||||||
|
|
||||||
|
<dependency>
|
||||||
|
<groupId>org.apache.kyuubi</groupId>
|
||||||
|
<artifactId>kyuubi-relocated-thrift</artifactId>
|
||||||
|
</dependency>
|
||||||
|
|
||||||
<dependency>
|
<dependency>
|
||||||
<groupId>org.apache.kyuubi</groupId>
|
<groupId>org.apache.kyuubi</groupId>
|
||||||
<artifactId>kyuubi-relocated-hive-service-rpc</artifactId>
|
<artifactId>kyuubi-relocated-hive-service-rpc</artifactId>
|
||||||
|
|||||||
@ -605,34 +605,6 @@ object KyuubiConf {
|
|||||||
.version("1.4.0")
|
.version("1.4.0")
|
||||||
.fallbackConf(FRONTEND_MAX_MESSAGE_SIZE)
|
.fallbackConf(FRONTEND_MAX_MESSAGE_SIZE)
|
||||||
|
|
||||||
@deprecated("using kyuubi.frontend.thrift.login.timeout instead", "1.4.0")
|
|
||||||
val FRONTEND_LOGIN_TIMEOUT: ConfigEntry[Long] =
|
|
||||||
buildConf("kyuubi.frontend.login.timeout")
|
|
||||||
.doc("(deprecated) Timeout for Thrift clients during login to the thrift frontend service.")
|
|
||||||
.version("1.0.0")
|
|
||||||
.timeConf
|
|
||||||
.createWithDefault(Duration.ofSeconds(20).toMillis)
|
|
||||||
|
|
||||||
val FRONTEND_THRIFT_LOGIN_TIMEOUT: ConfigEntry[Long] =
|
|
||||||
buildConf("kyuubi.frontend.thrift.login.timeout")
|
|
||||||
.doc("Timeout for Thrift clients during login to the thrift frontend service.")
|
|
||||||
.version("1.4.0")
|
|
||||||
.fallbackConf(FRONTEND_LOGIN_TIMEOUT)
|
|
||||||
|
|
||||||
@deprecated("using kyuubi.frontend.thrift.backoff.slot.length instead", "1.4.0")
|
|
||||||
val FRONTEND_LOGIN_BACKOFF_SLOT_LENGTH: ConfigEntry[Long] =
|
|
||||||
buildConf("kyuubi.frontend.backoff.slot.length")
|
|
||||||
.doc("(deprecated) Time to back off during login to the thrift frontend service.")
|
|
||||||
.version("1.0.0")
|
|
||||||
.timeConf
|
|
||||||
.createWithDefault(Duration.ofMillis(100).toMillis)
|
|
||||||
|
|
||||||
val FRONTEND_THRIFT_LOGIN_BACKOFF_SLOT_LENGTH: ConfigEntry[Long] =
|
|
||||||
buildConf("kyuubi.frontend.thrift.backoff.slot.length")
|
|
||||||
.doc("Time to back off during login to the thrift frontend service.")
|
|
||||||
.version("1.4.0")
|
|
||||||
.fallbackConf(FRONTEND_LOGIN_BACKOFF_SLOT_LENGTH)
|
|
||||||
|
|
||||||
val FRONTEND_THRIFT_HTTP_REQUEST_HEADER_SIZE: ConfigEntry[Int] =
|
val FRONTEND_THRIFT_HTTP_REQUEST_HEADER_SIZE: ConfigEntry[Int] =
|
||||||
buildConf("kyuubi.frontend.thrift.http.request.header.size")
|
buildConf("kyuubi.frontend.thrift.http.request.header.size")
|
||||||
.doc("Request header size in bytes, when using HTTP transport mode. Jetty defaults used.")
|
.doc("Request header size in bytes, when using HTTP transport mode. Jetty defaults used.")
|
||||||
@ -3144,14 +3116,6 @@ object KyuubiConf {
|
|||||||
FRONTEND_MAX_MESSAGE_SIZE.key,
|
FRONTEND_MAX_MESSAGE_SIZE.key,
|
||||||
"1.4.0",
|
"1.4.0",
|
||||||
s"Use ${FRONTEND_THRIFT_MAX_MESSAGE_SIZE.key} instead"),
|
s"Use ${FRONTEND_THRIFT_MAX_MESSAGE_SIZE.key} instead"),
|
||||||
DeprecatedConfig(
|
|
||||||
FRONTEND_LOGIN_TIMEOUT.key,
|
|
||||||
"1.4.0",
|
|
||||||
s"Use ${FRONTEND_THRIFT_LOGIN_TIMEOUT.key} instead"),
|
|
||||||
DeprecatedConfig(
|
|
||||||
FRONTEND_LOGIN_BACKOFF_SLOT_LENGTH.key,
|
|
||||||
"1.4.0",
|
|
||||||
s"Use ${FRONTEND_THRIFT_LOGIN_BACKOFF_SLOT_LENGTH.key} instead"),
|
|
||||||
DeprecatedConfig(
|
DeprecatedConfig(
|
||||||
SESSION_TIMEOUT.key,
|
SESSION_TIMEOUT.key,
|
||||||
"1.2.0",
|
"1.2.0",
|
||||||
|
|||||||
@ -101,17 +101,13 @@ abstract class TBinaryFrontendService(name: String)
|
|||||||
}
|
}
|
||||||
_actualPort = tServerSocket.getServerSocket.getLocalPort
|
_actualPort = tServerSocket.getServerSocket.getLocalPort
|
||||||
val maxMessageSize = conf.get(FRONTEND_THRIFT_MAX_MESSAGE_SIZE)
|
val maxMessageSize = conf.get(FRONTEND_THRIFT_MAX_MESSAGE_SIZE)
|
||||||
val requestTimeout = conf.get(FRONTEND_THRIFT_LOGIN_TIMEOUT).toInt
|
|
||||||
val beBackoffSlotLength = conf.get(FRONTEND_THRIFT_LOGIN_BACKOFF_SLOT_LENGTH).toInt
|
|
||||||
val args = new TThreadPoolServer.Args(tServerSocket)
|
val args = new TThreadPoolServer.Args(tServerSocket)
|
||||||
.processorFactory(tProcFactory)
|
.processorFactory(tProcFactory)
|
||||||
.transportFactory(transFactory)
|
.transportFactory(transFactory)
|
||||||
.protocolFactory(new TBinaryProtocol.Factory)
|
.protocolFactory(new TBinaryProtocol.Factory)
|
||||||
.inputProtocolFactory(
|
.inputProtocolFactory(
|
||||||
new TBinaryProtocol.Factory(true, true, maxMessageSize, maxMessageSize))
|
new TBinaryProtocol.Factory(true, true, maxMessageSize, maxMessageSize))
|
||||||
.requestTimeout(requestTimeout).requestTimeoutUnit(TimeUnit.MILLISECONDS)
|
// THRIFT-5297 (fixed in 0.14.0) removes requestTimeout and beBackoffSlotLength
|
||||||
.beBackoffSlotLength(beBackoffSlotLength)
|
|
||||||
.beBackoffSlotLengthUnit(TimeUnit.MILLISECONDS)
|
|
||||||
.executorService(executor)
|
.executorService(executor)
|
||||||
// TCP Server
|
// TCP Server
|
||||||
server = Some(new TThreadPoolServer(args))
|
server = Some(new TThreadPoolServer(args))
|
||||||
|
|||||||
@ -649,5 +649,9 @@ private[kyuubi] object TFrontendService {
|
|||||||
}
|
}
|
||||||
|
|
||||||
def getSessionHandle: SessionHandle = sessionHandle
|
def getSessionHandle: SessionHandle = sessionHandle
|
||||||
|
|
||||||
|
override def unwrap[T](aClass: Class[T]): T = null.asInstanceOf[T]
|
||||||
|
|
||||||
|
override def isWrapperFor(aClass: Class[_]): Boolean = false
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@ -131,7 +131,7 @@ object HadoopThriftAuthBridgeServer {
|
|||||||
class TUGIAssumingProcessor(
|
class TUGIAssumingProcessor(
|
||||||
wrapped: TProcessor,
|
wrapped: TProcessor,
|
||||||
secretMgr: KyuubiDelegationTokenManager) extends TProcessor with Logging {
|
secretMgr: KyuubiDelegationTokenManager) extends TProcessor with Logging {
|
||||||
override def process(in: TProtocol, out: TProtocol): Boolean = {
|
override def process(in: TProtocol, out: TProtocol): Unit = {
|
||||||
val transport = in.getTransport
|
val transport = in.getTransport
|
||||||
transport match {
|
transport match {
|
||||||
case saslTrans: TSaslServerTransport =>
|
case saslTrans: TSaslServerTransport =>
|
||||||
|
|||||||
@ -28,7 +28,7 @@ class TSetIpAddressProcessor[I <: Iface](
|
|||||||
import TSetIpAddressProcessor._
|
import TSetIpAddressProcessor._
|
||||||
|
|
||||||
@throws[TException]
|
@throws[TException]
|
||||||
override def process(in: TProtocol, out: TProtocol): Boolean = {
|
override def process(in: TProtocol, out: TProtocol): Unit = {
|
||||||
setIpAddress(in)
|
setIpAddress(in)
|
||||||
setUserName(in)
|
setUserName(in)
|
||||||
try {
|
try {
|
||||||
|
|||||||
@ -135,13 +135,6 @@
|
|||||||
<dependency>
|
<dependency>
|
||||||
<groupId>jline</groupId>
|
<groupId>jline</groupId>
|
||||||
<artifactId>jline</artifactId>
|
<artifactId>jline</artifactId>
|
||||||
<version>${hive.client.jline.version}</version>
|
|
||||||
<exclusions>
|
|
||||||
<exclusion>
|
|
||||||
<groupId>*</groupId>
|
|
||||||
<artifactId>*</artifactId>
|
|
||||||
</exclusion>
|
|
||||||
</exclusions>
|
|
||||||
</dependency>
|
</dependency>
|
||||||
|
|
||||||
<dependency>
|
<dependency>
|
||||||
|
|||||||
@ -97,6 +97,11 @@
|
|||||||
<artifactId>httpclient</artifactId>
|
<artifactId>httpclient</artifactId>
|
||||||
</dependency>
|
</dependency>
|
||||||
|
|
||||||
|
<dependency>
|
||||||
|
<groupId>org.apache.kyuubi</groupId>
|
||||||
|
<artifactId>kyuubi-relocated-thrift</artifactId>
|
||||||
|
</dependency>
|
||||||
|
|
||||||
<dependency>
|
<dependency>
|
||||||
<groupId>org.apache.kyuubi</groupId>
|
<groupId>org.apache.kyuubi</groupId>
|
||||||
<artifactId>kyuubi-relocated-hive-service-rpc</artifactId>
|
<artifactId>kyuubi-relocated-hive-service-rpc</artifactId>
|
||||||
|
|||||||
@ -22,6 +22,7 @@ import javax.security.auth.Subject;
|
|||||||
import javax.security.sasl.SaslException;
|
import javax.security.sasl.SaslException;
|
||||||
import org.apache.kyuubi.shaded.thrift.transport.TSaslClientTransport;
|
import org.apache.kyuubi.shaded.thrift.transport.TSaslClientTransport;
|
||||||
import org.apache.kyuubi.shaded.thrift.transport.TTransport;
|
import org.apache.kyuubi.shaded.thrift.transport.TTransport;
|
||||||
|
import org.apache.kyuubi.shaded.thrift.transport.TTransportException;
|
||||||
import org.slf4j.Logger;
|
import org.slf4j.Logger;
|
||||||
import org.slf4j.LoggerFactory;
|
import org.slf4j.LoggerFactory;
|
||||||
|
|
||||||
@ -34,7 +35,7 @@ public final class KerberosSaslHelper {
|
|||||||
String host,
|
String host,
|
||||||
TTransport underlyingTransport,
|
TTransport underlyingTransport,
|
||||||
Map<String, String> saslProps)
|
Map<String, String> saslProps)
|
||||||
throws SaslException {
|
throws SaslException, TTransportException {
|
||||||
String resolvedPrincipal = KerberosUtils.canonicalPrincipal(serverPrincipal, host);
|
String resolvedPrincipal = KerberosUtils.canonicalPrincipal(serverPrincipal, host);
|
||||||
String[] names = KerberosUtils.splitPrincipal(resolvedPrincipal);
|
String[] names = KerberosUtils.splitPrincipal(resolvedPrincipal);
|
||||||
TTransport saslTransport =
|
TTransport saslTransport =
|
||||||
|
|||||||
@ -22,11 +22,13 @@ import javax.security.auth.callback.*;
|
|||||||
import javax.security.sasl.SaslException;
|
import javax.security.sasl.SaslException;
|
||||||
import org.apache.kyuubi.shaded.thrift.transport.TSaslClientTransport;
|
import org.apache.kyuubi.shaded.thrift.transport.TSaslClientTransport;
|
||||||
import org.apache.kyuubi.shaded.thrift.transport.TTransport;
|
import org.apache.kyuubi.shaded.thrift.transport.TTransport;
|
||||||
|
import org.apache.kyuubi.shaded.thrift.transport.TTransportException;
|
||||||
|
|
||||||
public final class PlainSaslHelper {
|
public final class PlainSaslHelper {
|
||||||
|
|
||||||
public static TTransport getPlainTransport(
|
public static TTransport getPlainTransport(
|
||||||
String username, String password, TTransport underlyingTransport) throws SaslException {
|
String username, String password, TTransport underlyingTransport)
|
||||||
|
throws SaslException, TTransportException {
|
||||||
return new TSaslClientTransport(
|
return new TSaslClientTransport(
|
||||||
"PLAIN",
|
"PLAIN",
|
||||||
null,
|
null,
|
||||||
|
|||||||
@ -17,6 +17,7 @@
|
|||||||
|
|
||||||
package org.apache.kyuubi.jdbc.hive.auth;
|
package org.apache.kyuubi.jdbc.hive.auth;
|
||||||
|
|
||||||
|
import org.apache.kyuubi.shaded.thrift.TConfiguration;
|
||||||
import org.apache.kyuubi.shaded.thrift.transport.TTransport;
|
import org.apache.kyuubi.shaded.thrift.transport.TTransport;
|
||||||
import org.apache.kyuubi.shaded.thrift.transport.TTransportException;
|
import org.apache.kyuubi.shaded.thrift.transport.TTransportException;
|
||||||
|
|
||||||
@ -95,4 +96,19 @@ public class TFilterTransport extends TTransport {
|
|||||||
public void consumeBuffer(int len) {
|
public void consumeBuffer(int len) {
|
||||||
wrapped.consumeBuffer(len);
|
wrapped.consumeBuffer(len);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public TConfiguration getConfiguration() {
|
||||||
|
return wrapped.getConfiguration();
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public void updateKnownMessageSize(long l) throws TTransportException {
|
||||||
|
wrapped.updateKnownMessageSize(l);
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public void checkReadBytesAvailable(long l) throws TTransportException {
|
||||||
|
wrapped.checkReadBytesAvailable(l);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@ -19,6 +19,7 @@ package org.apache.kyuubi.jdbc.hive.auth;
|
|||||||
|
|
||||||
import javax.net.ssl.SSLParameters;
|
import javax.net.ssl.SSLParameters;
|
||||||
import javax.net.ssl.SSLSocket;
|
import javax.net.ssl.SSLSocket;
|
||||||
|
import org.apache.kyuubi.shaded.thrift.TConfiguration;
|
||||||
import org.apache.kyuubi.shaded.thrift.transport.TSSLTransportFactory;
|
import org.apache.kyuubi.shaded.thrift.transport.TSSLTransportFactory;
|
||||||
import org.apache.kyuubi.shaded.thrift.transport.TSocket;
|
import org.apache.kyuubi.shaded.thrift.transport.TSocket;
|
||||||
import org.apache.kyuubi.shaded.thrift.transport.TTransport;
|
import org.apache.kyuubi.shaded.thrift.transport.TTransport;
|
||||||
@ -30,8 +31,8 @@ import org.apache.kyuubi.shaded.thrift.transport.TTransportException;
|
|||||||
*/
|
*/
|
||||||
public class ThriftUtils {
|
public class ThriftUtils {
|
||||||
public static TTransport getSocketTransport(
|
public static TTransport getSocketTransport(
|
||||||
String host, int port, int connectTimeout, int socketTimeout) {
|
String host, int port, int connectTimeout, int socketTimeout) throws TTransportException {
|
||||||
return new TSocket(host, port, socketTimeout, connectTimeout);
|
return new TSocket(TConfiguration.DEFAULT, host, port, socketTimeout, connectTimeout);
|
||||||
}
|
}
|
||||||
|
|
||||||
public static TTransport getSSLSocket(
|
public static TTransport getSSLSocket(
|
||||||
|
|||||||
@ -36,6 +36,7 @@ import org.apache.kyuubi.operation.FetchOrientation.FetchOrientation
|
|||||||
import org.apache.kyuubi.service.authentication.PlainSASLHelper
|
import org.apache.kyuubi.service.authentication.PlainSASLHelper
|
||||||
import org.apache.kyuubi.session.SessionHandle
|
import org.apache.kyuubi.session.SessionHandle
|
||||||
import org.apache.kyuubi.shaded.hive.service.rpc.thrift._
|
import org.apache.kyuubi.shaded.hive.service.rpc.thrift._
|
||||||
|
import org.apache.kyuubi.shaded.thrift.TConfiguration
|
||||||
import org.apache.kyuubi.shaded.thrift.protocol.{TBinaryProtocol, TProtocol}
|
import org.apache.kyuubi.shaded.thrift.protocol.{TBinaryProtocol, TProtocol}
|
||||||
import org.apache.kyuubi.shaded.thrift.transport.TSocket
|
import org.apache.kyuubi.shaded.thrift.transport.TSocket
|
||||||
import org.apache.kyuubi.util.{ThreadUtils, ThriftUtils}
|
import org.apache.kyuubi.util.{ThreadUtils, ThriftUtils}
|
||||||
@ -457,7 +458,7 @@ private[kyuubi] object KyuubiSyncThriftClient extends Logging {
|
|||||||
port: Int,
|
port: Int,
|
||||||
socketTimeout: Int,
|
socketTimeout: Int,
|
||||||
connectionTimeout: Int): TProtocol = {
|
connectionTimeout: Int): TProtocol = {
|
||||||
val tSocket = new TSocket(host, port, socketTimeout, connectionTimeout)
|
val tSocket = new TSocket(TConfiguration.DEFAULT, host, port, socketTimeout, connectionTimeout)
|
||||||
val tTransport = PlainSASLHelper.getPlainTransport(user, passwd, tSocket)
|
val tTransport = PlainSASLHelper.getPlainTransport(user, passwd, tSocket)
|
||||||
tTransport.open()
|
tTransport.open()
|
||||||
new TBinaryProtocol(tTransport)
|
new TBinaryProtocol(tTransport)
|
||||||
|
|||||||
@ -80,7 +80,9 @@ class KyuubiOperationPerConnectionSuite extends WithKyuubiServer with HiveJDBCTe
|
|||||||
assert(executeStmtResp.getOperationHandle === null)
|
assert(executeStmtResp.getOperationHandle === null)
|
||||||
val errMsg = executeStmtResp.getStatus.getErrorMessage
|
val errMsg = executeStmtResp.getStatus.getErrorMessage
|
||||||
assert(errMsg.contains("Caused by: java.net.SocketException: Connection reset") ||
|
assert(errMsg.contains("Caused by: java.net.SocketException: Connection reset") ||
|
||||||
errMsg.contains(s"Socket for ${SessionHandle(handle)} is closed"))
|
errMsg.contains(s"Socket for ${SessionHandle(handle)} is closed") ||
|
||||||
|
errMsg.contains("Socket is closed by peer") ||
|
||||||
|
errMsg.contains("SparkContext was shut down"))
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -332,9 +334,10 @@ class KyuubiOperationPerConnectionSuite extends WithKyuubiServer with HiveJDBCTe
|
|||||||
assert(executeStmtResp.getStatus.getStatusCode === TStatusCode.ERROR_STATUS)
|
assert(executeStmtResp.getStatus.getStatusCode === TStatusCode.ERROR_STATUS)
|
||||||
val errorMsg = executeStmtResp.getStatus.getErrorMessage
|
val errorMsg = executeStmtResp.getStatus.getErrorMessage
|
||||||
assert(errorMsg.contains("java.net.SocketException") ||
|
assert(errorMsg.contains("java.net.SocketException") ||
|
||||||
errorMsg.contains("org.apache.thrift.transport.TTransportException") ||
|
errorMsg.contains("org.apache.kyuubi.shaded.thrift.transport.TTransportException") ||
|
||||||
errorMsg.contains("connection does not exist") ||
|
errorMsg.contains("connection does not exist") ||
|
||||||
errorMsg.contains(s"Socket for ${SessionHandle(handle)} is closed"))
|
errorMsg.contains(s"Socket for ${SessionHandle(handle)} is closed") ||
|
||||||
|
errorMsg.contains("Error submitting query in background, query rejected"))
|
||||||
val elapsedTime = System.currentTimeMillis() - startTime
|
val elapsedTime = System.currentTimeMillis() - startTime
|
||||||
assert(elapsedTime < 20 * 1000)
|
assert(elapsedTime < 20 * 1000)
|
||||||
eventually(timeout(3.seconds)) {
|
eventually(timeout(3.seconds)) {
|
||||||
|
|||||||
10
pom.xml
10
pom.xml
@ -169,11 +169,11 @@
|
|||||||
<jersey.version>2.39.1</jersey.version>
|
<jersey.version>2.39.1</jersey.version>
|
||||||
<jetcd.version>0.7.7</jetcd.version>
|
<jetcd.version>0.7.7</jetcd.version>
|
||||||
<jetty.version>9.4.52.v20230823</jetty.version>
|
<jetty.version>9.4.52.v20230823</jetty.version>
|
||||||
<jline.version>0.9.94</jline.version>
|
<jline.version>2.14.6</jline.version>
|
||||||
<junit.version>4.13.2</junit.version>
|
<junit.version>4.13.2</junit.version>
|
||||||
<kafka.version>3.5.1</kafka.version>
|
<kafka.version>3.5.1</kafka.version>
|
||||||
<kubernetes-client.version>6.8.1</kubernetes-client.version>
|
<kubernetes-client.version>6.8.1</kubernetes-client.version>
|
||||||
<kyuubi-relocated.version>0.2.0</kyuubi-relocated.version>
|
<kyuubi-relocated.version>0.3.0</kyuubi-relocated.version>
|
||||||
<kyuubi-relocated-zookeeper.artifacts>kyuubi-relocated-zookeeper-34</kyuubi-relocated-zookeeper.artifacts>
|
<kyuubi-relocated-zookeeper.artifacts>kyuubi-relocated-zookeeper-34</kyuubi-relocated-zookeeper.artifacts>
|
||||||
<ldapsdk.version>6.0.5</ldapsdk.version>
|
<ldapsdk.version>6.0.5</ldapsdk.version>
|
||||||
<log4j.version>2.20.0</log4j.version>
|
<log4j.version>2.20.0</log4j.version>
|
||||||
@ -223,7 +223,6 @@
|
|||||||
<pnpm.version>v8.6.1</pnpm.version>
|
<pnpm.version>v8.6.1</pnpm.version>
|
||||||
|
|
||||||
<!-- apply to kyuubi-hive-jdbc/kyuubi-hive-beeline module -->
|
<!-- apply to kyuubi-hive-jdbc/kyuubi-hive-beeline module -->
|
||||||
<hive.client.jline.version>2.12</hive.client.jline.version>
|
|
||||||
<hive.client.supercsv.version>2.2.0</hive.client.supercsv.version>
|
<hive.client.supercsv.version>2.2.0</hive.client.supercsv.version>
|
||||||
|
|
||||||
<hive.jdbc.artifact>kyuubi-hive-jdbc</hive.jdbc.artifact>
|
<hive.jdbc.artifact>kyuubi-hive-jdbc</hive.jdbc.artifact>
|
||||||
@ -279,6 +278,11 @@
|
|||||||
|
|
||||||
<dependencyManagement>
|
<dependencyManagement>
|
||||||
<dependencies>
|
<dependencies>
|
||||||
|
<dependency>
|
||||||
|
<groupId>org.apache.kyuubi</groupId>
|
||||||
|
<artifactId>kyuubi-relocated-thrift</artifactId>
|
||||||
|
<version>${kyuubi-relocated.version}</version>
|
||||||
|
</dependency>
|
||||||
<dependency>
|
<dependency>
|
||||||
<groupId>org.apache.kyuubi</groupId>
|
<groupId>org.apache.kyuubi</groupId>
|
||||||
<artifactId>kyuubi-relocated-hive-service-rpc</artifactId>
|
<artifactId>kyuubi-relocated-hive-service-rpc</artifactId>
|
||||||
|
|||||||
Loading…
Reference in New Issue
Block a user