apache/kyuubi
1
0
Fork 0
Code Issues Pull Requests Actions 5 Packages Projects Releases Wiki Activity
a3d9ca31d3
kyuubi/dev/dependencyList
SteNicholas 770499ca54
[KYUUBI #1948] Upgrade thrift version to 0.16.0 
### _Why are the changes needed?_

Upgrade libthrift to 0.16.0 due to [CVE-2020-13949](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-13949) and the coming upstream change of Spark https://github.com/apache/spark/pull/34362

### _What changed in this PR?_

- Upgrade libthrift to 0.16.0

- Shade and relocate `thrift` and `hive-service-rpc` classes in `kyuubi-spark-engine`, it's necessary to avoide conflicting with old thrift libs bundled in Spark binary releases.

- Due to thrift change the method signature, the subclasses those interfaces in Kyuubi also need to modify to pass compile.
We rely on Hive 2.3.9 jars in some components, e.g. `kyuubi-hive-jdbc`, `LocalMetaServer` in `kyuubi-server` test classes.

Some classes in Hive jars compiled against old thrift interfaces which are not compatible with thrift 0.16.0, it causes runtime link error, we found the following classes which breaks the test and copied them with necessary modification to make it work with thrift 0.16.0.

    - `TFramedTransport`
    - `TFilterTransport`
    - `TUGIAssumingTransport`
    - `TUGIContainingTransport`

- Next Steps, I think it's worth to do in separated PRs.

    - Recover the `HiveDelegationTokenProviderSuite`, one approach is use an isolate classloader to load HMS classes and thrift 0.9.3 classes from Maven, this approach can also be used for the planed Zoopkeeper upgrading to help us verficating the compatibility of Zookeeper Server 3.4.x.
    - Rewrite `kyuubi-hive-jdbc` to make it decouple with Hive jars, because there maybe other places which may not work with thrift 0.16.0 but the UTs does not cover.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1953 from SteNicholas/KYUUBI-1948.

Closes #1948

de5d1ea2 [SteNicholas] [KYUUBI #1948] Upgrade thrift version to 0.16.0
898effcd [SteNicholas] [KYUUBI #1948] Upgrade thrift version to 0.16.0
803e270c [SteNicholas] [KYUUBI #1948] Upgrade thrift version to 0.16.0

Authored-by: SteNicholas <programgeek@163.com>
Signed-off-by: Kent Yao <yao@apache.org>
2022-02-23 20:51:59 +08:00

120 lines
6.4 KiB
Plaintext
Raw Blame History

#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
aopalliance-repackaged/2.6.1//aopalliance-repackaged-2.6.1.jar
classgraph/4.8.95//classgraph-4.8.95.jar
commons-codec/1.15//commons-codec-1.15.jar
commons-lang/2.6//commons-lang-2.6.jar
commons-lang3/3.10//commons-lang3-3.10.jar
curator-client/2.12.0//curator-client-2.12.0.jar
curator-framework/2.12.0//curator-framework-2.12.0.jar
curator-recipes/2.12.0//curator-recipes-2.12.0.jar
guava/30.1-jre//guava-30.1-jre.jar
hadoop-client-api/3.3.1//hadoop-client-api-3.3.1.jar
hadoop-client-runtime/3.3.1//hadoop-client-runtime-3.3.1.jar
hive-common/2.3.9//hive-common-2.3.9.jar
hive-metastore/2.3.9//hive-metastore-2.3.9.jar
hive-serde/2.3.9//hive-serde-2.3.9.jar
hive-service-rpc/2.3.9//hive-service-rpc-2.3.9.jar
hive-shims-0.23/2.3.9//hive-shims-0.23-2.3.9.jar
hive-shims-common/2.3.9//hive-shims-common-2.3.9.jar
hk2-api/2.6.1//hk2-api-2.6.1.jar
hk2-locator/2.6.1//hk2-locator-2.6.1.jar
hk2-utils/2.6.1//hk2-utils-2.6.1.jar
htrace-core4/4.1.0-incubating//htrace-core4-4.1.0-incubating.jar
httpclient/4.5.13//httpclient-4.5.13.jar
httpcore/4.4.15//httpcore-4.4.15.jar
jackson-annotations/2.13.1//jackson-annotations-2.13.1.jar
jackson-core/2.13.1//jackson-core-2.13.1.jar
jackson-databind/2.13.1//jackson-databind-2.13.1.jar
jackson-dataformat-yaml/2.13.1//jackson-dataformat-yaml-2.13.1.jar
jackson-datatype-jsr310/2.13.1//jackson-datatype-jsr310-2.13.1.jar
jackson-jaxrs-base/2.13.1//jackson-jaxrs-base-2.13.1.jar
jackson-jaxrs-json-provider/2.13.1//jackson-jaxrs-json-provider-2.13.1.jar
jackson-module-jaxb-annotations/2.13.1//jackson-module-jaxb-annotations-2.13.1.jar
jackson-module-scala_2.12/2.13.1//jackson-module-scala_2.12-2.13.1.jar
jakarta.activation-api/1.2.1//jakarta.activation-api-1.2.1.jar
jakarta.annotation-api/1.3.5//jakarta.annotation-api-1.3.5.jar
jakarta.inject/2.6.1//jakarta.inject-2.6.1.jar
jakarta.servlet-api/4.0.4//jakarta.servlet-api-4.0.4.jar
jakarta.validation-api/2.0.2//jakarta.validation-api-2.0.2.jar
jakarta.ws.rs-api/2.1.6//jakarta.ws.rs-api-2.1.6.jar
jakarta.xml.bind-api/2.3.2//jakarta.xml.bind-api-2.3.2.jar
javassist/3.25.0-GA//javassist-3.25.0-GA.jar
jcl-over-slf4j/1.7.35//jcl-over-slf4j-1.7.35.jar
jersey-client/2.34//jersey-client-2.34.jar
jersey-common/2.34//jersey-common-2.34.jar
jersey-container-servlet-core/2.34//jersey-container-servlet-core-2.34.jar
jersey-entity-filtering/2.34//jersey-entity-filtering-2.34.jar
jersey-hk2/2.34//jersey-hk2-2.34.jar
jersey-media-json-jackson/2.34//jersey-media-json-jackson-2.34.jar
jersey-server/2.34//jersey-server-2.34.jar
jetty-http/9.4.41.v20210516//jetty-http-9.4.41.v20210516.jar
jetty-io/9.4.41.v20210516//jetty-io-9.4.41.v20210516.jar
jetty-security/9.4.41.v20210516//jetty-security-9.4.41.v20210516.jar
jetty-server/9.4.41.v20210516//jetty-server-9.4.41.v20210516.jar
jetty-servlet/9.4.41.v20210516//jetty-servlet-9.4.41.v20210516.jar
jetty-util-ajax/9.4.41.v20210516//jetty-util-ajax-9.4.41.v20210516.jar
jetty-util/9.4.41.v20210516//jetty-util-9.4.41.v20210516.jar
jline/0.9.94//jline-0.9.94.jar
libfb303/0.9.3//libfb303-0.9.3.jar
libthrift/0.16.0//libthrift-0.16.0.jar
log4j-1.2-api/2.17.1//log4j-1.2-api-2.17.1.jar
log4j-api/2.17.1//log4j-api-2.17.1.jar
log4j-core/2.17.1//log4j-core-2.17.1.jar
log4j-slf4j-impl/2.17.1//log4j-slf4j-impl-2.17.1.jar
metrics-core/4.2.8//metrics-core-4.2.8.jar
metrics-jmx/4.2.8//metrics-jmx-4.2.8.jar
metrics-json/4.2.8//metrics-json-4.2.8.jar
metrics-jvm/4.2.8//metrics-jvm-4.2.8.jar
netty-all/4.1.73.Final//netty-all-4.1.73.Final.jar
netty-buffer/4.1.73.Final//netty-buffer-4.1.73.Final.jar
netty-codec/4.1.73.Final//netty-codec-4.1.73.Final.jar
netty-common/4.1.73.Final//netty-common-4.1.73.Final.jar
netty-handler/4.1.73.Final//netty-handler-4.1.73.Final.jar
netty-resolver/4.1.73.Final//netty-resolver-4.1.73.Final.jar
netty-tcnative-classes/2.0.46.Final//netty-tcnative-classes-2.0.46.Final.jar
netty-transport-classes-epoll/4.1.73.Final//netty-transport-classes-epoll-4.1.73.Final.jar
netty-transport-classes-kqueue/4.1.73.Final//netty-transport-classes-kqueue-4.1.73.Final.jar
netty-transport-native-epoll/4.1.73.Final/linux-aarch_64/netty-transport-native-epoll-4.1.73.Final-linux-aarch_64.jar
netty-transport-native-epoll/4.1.73.Final/linux-x86_64/netty-transport-native-epoll-4.1.73.Final-linux-x86_64.jar
netty-transport-native-kqueue/4.1.73.Final/osx-aarch_64/netty-transport-native-kqueue-4.1.73.Final-osx-aarch_64.jar
netty-transport-native-kqueue/4.1.73.Final/osx-x86_64/netty-transport-native-kqueue-4.1.73.Final-osx-x86_64.jar
netty-transport-native-unix-common/4.1.73.Final//netty-transport-native-unix-common-4.1.73.Final.jar
netty-transport/4.1.73.Final//netty-transport-4.1.73.Final.jar
osgi-resource-locator/1.0.3//osgi-resource-locator-1.0.3.jar
paranamer/2.8//paranamer-2.8.jar
scala-library/2.12.15//scala-library-2.12.15.jar
scopt_2.12/4.0.1//scopt_2.12-4.0.1.jar
simpleclient/0.14.1//simpleclient-0.14.1.jar
simpleclient_common/0.14.1//simpleclient_common-0.14.1.jar
simpleclient_dropwizard/0.14.1//simpleclient_dropwizard-0.14.1.jar
simpleclient_servlet/0.14.1//simpleclient_servlet-0.14.1.jar
simpleclient_servlet_common/0.14.1//simpleclient_servlet_common-0.14.1.jar
simpleclient_tracer_common/0.14.1//simpleclient_tracer_common-0.14.1.jar
simpleclient_tracer_otel/0.14.1//simpleclient_tracer_otel-0.14.1.jar
simpleclient_tracer_otel_agent/0.14.1//simpleclient_tracer_otel_agent-0.14.1.jar
slf4j-api/1.7.35//slf4j-api-1.7.35.jar
snakeyaml/1.28//snakeyaml-1.28.jar
swagger-annotations/2.1.11//swagger-annotations-2.1.11.jar
swagger-core/2.1.11//swagger-core-2.1.11.jar
swagger-integration/2.1.11//swagger-integration-2.1.11.jar
swagger-jaxrs2/2.1.11//swagger-jaxrs2-2.1.11.jar
swagger-models/2.1.11//swagger-models-2.1.11.jar
swagger-ui/4.1.3//swagger-ui-4.1.3.jar
zookeeper/3.4.14//zookeeper-3.4.14.jar
Reference in New Issue View Git Blame Copy Permalink