0cfbcd2e08
40 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Cheng Pan
|
f8efcb7125
|
[KYUUBI #2103] Revert "[KYUUBI #1948] Upgrade thrift version to 0.16.0"
### _Why are the changes needed?_ This PR reverts "[KYUUBI #1948] Upgrade thrift version to 0.16.0" in branch-1.5 and some subsequent commits due to issues caused by breaking changes in Thrift, e.g. #2101, #2093 We need more time to evaluate it and add more test cases to ensure stability before the next release. ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #2103 from pan3793/revert. Closes #2103 Closes #1948 575dfe8d [Cheng Pan] properties order 4c94481e [Cheng Pan] fix deps bfbdeb89 [Cheng Pan] Add log4j deps for kyuubi-hive-jdbc test bad4e90e [Cheng Pan] surefire f2ae8b9b [Cheng Pan] Revert "[KYUUBI #1948] Upgrade thrift version to 0.16.0" 4501f743 [Cheng Pan] Revert "[KYUUBI #1948][FOLLOWUP] Remove TFramedTransport in extension-spark-3-1 and extension-spark-3-2" cc04df49 [Cheng Pan] Revert "[KYUUBI #1996] Clean up log4j2 conf and enhance surefire plugin conf" 52a20e17 [Cheng Pan] Revert "[KYUUBI #1948][FOLLOWUP] Relocate fb303 classes" Authored-by: Cheng Pan <chengpan@apache.org> Signed-off-by: Cheng Pan <chengpan@apache.org> |
||
|
Fu Chen
|
9bf8ff83d8
|
[KYUUBI #2061] Implementation of the very basic UI on current Jetty server
### _Why are the changes needed?_ thanks to the pr original author yaooqinn https://github.com/apache/incubator-kyuubi/pull/1670 After this pr, the endpoint of OpenAPI will change from`http://${ip}:${port}/openapi.json` to `http://${ip}:${port}/api/openapi.json` ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [x] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request   Closes #2061 from cfmcgrady/pr-1670. Closes #2061 969b7d3d [Fu Chen] update dev/dependencyList ff4b6bc2 [Fu Chen] fix ut 207553b6 [Fu Chen] addressed comment a3baefe0 [Fu Chen] new line at end of file ff36f957 [Fu Chen] fix bb6abb3f [Fu Chen] language b22be2ec [Fu Chen] style f0c9562f [Fu Chen] Merge remote-tracking branch 'upstream/master' into pr-1670 d4ac4cba [Fu Chen] fix cd5ee2b8 [Fu Chen] Merge remote-tracking branch 'upstream/master' into pr-1670 1b108a04 [Kent Yao] ui 85e31519 [Kent Yao] ui 0c20fcff [Kent Yao] ui c2abde93 [Kent Yao] ui b4dff513 [Kent Yao] ui 43dffab8 [Kent Yao] Merge branch 'master' into web 7cd273a3 [Kent Yao] swagger ui enabled 518882e2 [Kent Yao] swagger ui enabled Lead-authored-by: Fu Chen <cfmcgrady@gmail.com> Co-authored-by: Kent Yao <yao@apache.org> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
--replace-all
|
dc6085e060
|
[KYUUBI #2043] Upgrade log4j/2.x/ to 2.17.2
### _Why are the changes needed?_ Close #2043 ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [x] Add screenshots for manual tests if appropriate  - [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #2046 from RishiKumarRay/master. Closes #2043 bf9a4a0b [--replace-all] fixed sql issue c119e5c2 [Rishi Kumar Ray] Merge branch 'apache:master' into master 323d81c7 [--replace-all] Upgrade log4j/2.x/ to 2.17.2 Lead-authored-by: --replace-all <rishi.ray@knoldus.com> Co-authored-by: Rishi Kumar Ray <87641376+RishiKumarRay@users.noreply.github.com> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
SteNicholas
|
ce8462705f
|
[KYUUBI #1961] Upgrade hive-service-rpc to 3.1.2
### _Why are the changes needed?_ Upgrade `hive-service-rpc` to 3.1.2. ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #2031 from SteNicholas/KYUUBI-1961. Closes #1961 88c29557 [SteNicholas] [KYUUBI #1961] Upgrade hive-service-rpc to 3.1.2 Authored-by: SteNicholas <programgeek@163.com> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
Ada Wong
|
e250c4ae35
|
[KYUUBI #2000][DEPS] Bump Hadoop 3.3.2
### _Why are the changes needed?_ Hadoop 3.3.2 is out. ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #2003 from deadwind4/KYUUBI-2000. Closes #2000 3b5b3f85 [Ada Wong] [KYUUBI #2000][DEPS] Bump Hadoop 3.3.2 Authored-by: Ada Wong <rsl4@foxmail.com> Signed-off-by: Cheng Pan <chengpan@apache.org> |
||
|
Kent Yao
|
d6bbe32a01
|
[KYUUBI #1981] Remove unused artifacts for common and parent module
### _Why are the changes needed?_ ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1981 from yaooqinn/branch-dep. Closes #1981 07bf0e29 [Kent Yao] codec 96e41e7d [Kent Yao] codec 1da26432 [Kent Yao] guava 6cf0abb3 [Kent Yao] base64 & sha1 3b45ec2a [Kent Yao] base64 5490cfa7 [Kent Yao] http 6f4b48b8 [Kent Yao] n 2a87a4cf [Kent Yao] d a5ad3ccc [Kent Yao] Refine Deps in common module 24d5299e [Kent Yao] Refine Deps in common module Authored-by: Kent Yao <yao@apache.org> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
SteNicholas
|
770499ca54
|
[KYUUBI #1948] Upgrade thrift version to 0.16.0
### _Why are the changes needed?_ Upgrade libthrift to 0.16.0 due to [CVE-2020-13949](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-13949) and the coming upstream change of Spark https://github.com/apache/spark/pull/34362 ### _What changed in this PR?_ - Upgrade libthrift to 0.16.0 - Shade and relocate `thrift` and `hive-service-rpc` classes in `kyuubi-spark-engine`, it's necessary to avoide conflicting with old thrift libs bundled in Spark binary releases. - Due to thrift change the method signature, the subclasses those interfaces in Kyuubi also need to modify to pass compile. We rely on Hive 2.3.9 jars in some components, e.g. `kyuubi-hive-jdbc`, `LocalMetaServer` in `kyuubi-server` test classes. Some classes in Hive jars compiled against old thrift interfaces which are not compatible with thrift 0.16.0, it causes runtime link error, we found the following classes which breaks the test and copied them with necessary modification to make it work with thrift 0.16.0. - `TFramedTransport` - `TFilterTransport` - `TUGIAssumingTransport` - `TUGIContainingTransport` - Next Steps, I think it's worth to do in separated PRs. - Recover the `HiveDelegationTokenProviderSuite`, one approach is use an isolate classloader to load HMS classes and thrift 0.9.3 classes from Maven, this approach can also be used for the planed Zoopkeeper upgrading to help us verficating the compatibility of Zookeeper Server 3.4.x. - Rewrite `kyuubi-hive-jdbc` to make it decouple with Hive jars, because there maybe other places which may not work with thrift 0.16.0 but the UTs does not cover. ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1953 from SteNicholas/KYUUBI-1948. Closes #1948 de5d1ea2 [SteNicholas] [KYUUBI #1948] Upgrade thrift version to 0.16.0 898effcd [SteNicholas] [KYUUBI #1948] Upgrade thrift version to 0.16.0 803e270c [SteNicholas] [KYUUBI #1948] Upgrade thrift version to 0.16.0 Authored-by: SteNicholas <programgeek@163.com> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
ulysses-you
|
73c84d42e7
|
[KYUUBI #1868] Upgrate netty from 4.1.68 to 4.1.73
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> close https://github.com/apache/incubator-kyuubi/issues/1868 ### _How was this patch tested?_ Pass CI Closes #1869 from ulysses-you/netty. Closes #1868 c197d0e3 [ulysses-you] Upgrate netty from 4.1.68 to 4.1.73 Authored-by: ulysses-you <ulyssesyou18@gmail.com> Signed-off-by: ulysses-you <ulyssesyou@apache.org> |
||
|
Kent Yao
|
08fc087e25
|
[KYUUBI #1875] Upgrade codahale.metrics.version to 4.2.8 and its deps
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> Upgrade codahale.metrics.version to 4.2.8, jackson to 2.13.1 abd slf4j to 1.7.35 ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1875 from yaooqinn/md. Closes #1875 12a21729 [Kent Yao] Upgrade codahale.metrics.version to 4.2.8 Authored-by: Kent Yao <yao@apache.org> Signed-off-by: ulysses-you <ulyssesyou@apache.org> |
||
|
Cheng Pan
|
839c5c5bf4
|
[KYUUBI #1861] Migrate bind-api and activation-api from javax to jakarta
### _Why are the changes needed?_ The Java world is migrating JavaEE components from `javax` to `jakarta`, we should follow it. ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1861 from pan3793/jaxb. Closes #1861 3563b773 [Cheng Pan] Migrate bind-api activation-api to jakarta Authored-by: Cheng Pan <chengpan@apache.org> Signed-off-by: Cheng Pan <chengpan@apache.org> |
||
|
Cheng Pan
|
b2bad04d75 |
[KYUUBI #1769][FOLLOWUP] Exclude log4j 12 deps from spark-hive and fix detect deps change workflow
### _Why are the changes needed?_ The previous PR did not handle log4j 1.2 deps of `spark-hive`, and the current detect deps change workflow has the dirty cache issue, this PR also fix it. ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1856 from pan3793/log4j2. Closes #1769 2785acf4 [Cheng Pan] nit 58affcb5 [Cheng Pan] update workflow 301acb6f [Cheng Pan] update check deps script 7611c7e0 [Cheng Pan] revert exclusion 239e7cc4 [Cheng Pan] Remove duplicated scalatest e143fecc [Cheng Pan] dependencyList a7254771 [Cheng Pan] [KYUUBI #1769][FOLLOWUP] Exclude log4j 12 deps from test deps Authored-by: Cheng Pan <chengpan@apache.org> Signed-off-by: Fei Wang <fwang12@ebay.com> |
||
|
SteNicholas
|
463f851351
|
[KYUUBI #1850] Bump Prometheus 0.14.1
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ Bump Prometheus 0.14.1. There are some breaking changes, please go to https://github.com/prometheus/client_java/releases to learn details. ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1857 from SteNicholas/KYUUBI-1850. Closes #1850 41c6f682 [SteNicholas] [KYUUBI #1850] Bump Prometheus 0.14.1 Authored-by: SteNicholas <programgeek@163.com> Signed-off-by: Cheng Pan <chengpan@apache.org> |
||
|
Fei Wang
|
885ace06c8 |
[KYUUBI #1769][FOLLOWUP] Some cleanups for log4j properties
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> This is a followup of #1769. Cleanup the log4j properties files and using log4j2.properties for UT. ### _How was this patch tested?_ Passed UT. Closes #1842 from turboFei/remove_log4j1. Closes #1769 b6757555 [Fei Wang] unused change 365fbec7 [Fei Wang] revert license 14c64ec9 [Fei Wang] exclude log4j 1.2.17 in slf4j-log4j12 034b04db [Fei Wang] recover jcl-over-slf4j and slf4j-api for k8s it 2beae75e [Fei Wang] remove log4j 1.2.17 b00f07b5 [Fei Wang] remove from license e55fd2ec [Fei Wang] remove unused dependencies ab86f023 [Fei Wang] [KYUUBI #1769][FOLLOWUP] Some cleanups for log4j properties Authored-by: Fei Wang <fwang12@ebay.com> Signed-off-by: Fei Wang <fwang12@ebay.com> |
||
|
Fei Wang
|
53d59a02bf |
[KYUUBI #1769] [BUILD] Migrate from log4j1 to log4j2
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> This patch proposes to migrate from log4j1 to log4j2 in Kyuubi. Refer the spark patch: https://github.com/apache/spark/pull/34895 ### Does this PR introduce any user-facing change? Yes. Users may need to rewrite their log4j properties file for log4j2. As of version 2.4, log4j now supports configuration via properties files. Note that the property syntax is not the same as the syntax used in log4j 1, but during the migration I found the syntax is pretty close so the migration should be straightforward. ### _How was this patch tested?_ Passed all existing tests. Closes #1769 from turboFei/log4j2. Closes #1769 8613779c [Fei Wang] remove log4j dependencies from spark-sql-engine b2fe6dba [Fei Wang] Use String to present default log level 762e2d03 [Fei Wang] only remove org.apache.logging.log4j:log4j-slf4j-impl 8a912086 [Fei Wang] remove dependencies from spark-sql-engine 7e3a4980 [Fei Wang] address commments 051f49f5 [Fei Wang] address comments 85316a0b [Fei Wang] Keep compatible with log4j12 01d1a84e [Fei Wang] for log4j1 b9e17e1b [Fei Wang] refactor e24391ed [Fei Wang] revert log count 38803002 [Fei Wang] add log4j2.properties.template 4f0b22fc [Fei Wang] save 7ce84119 [Fei Wang] modify log level 1ea5ca53 [Fei Wang] add log4j to engine c4a86d4d [Fei Wang] use AbstractFilter 27b08b6a [Fei Wang] remove more 8cc15ae7 [Fei Wang] reformat c13ec29e [Fei Wang] save temporarily 33a38e2e [Fei Wang] exclude log4j12 from spark-sql 9129a64a [Fei Wang] refactor 5362b43d [Fei Wang] make it run at first 7f27f519 [Fei Wang] more 56f4f1ff [Fei Wang] fix logging a74b6d37 [Fei Wang] start appender dea964aa [Fei Wang] fix build erorr at first e20b7500 [Fei Wang] address comments 2ec02b4d [Fei Wang] fix LogDivertAppender dded1290 [Fei Wang] more c63e0008 [Fei Wang] add log4j2.properties Authored-by: Fei Wang <fwang12@ebay.com> Signed-off-by: Fei Wang <fwang12@ebay.com> |
||
|
Kent Yao
|
c1a6cd587c
|
[KYUUBI #1716] Unify Hive deps for server and client
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> This PR make the hive deps be consistent between server and clients for a single release, we shall not play client-server compatibility tricky in one release ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1716 from yaooqinn/hive. Closes #1716 2eb3bdf7 [Kent Yao] Merge branch 'master' into hive c1bf6369 [Kent Yao] dep c7f0f26f [Kent Yao] dep 645d76ab [Kent Yao] fix test 3ac4f55c [Kent Yao] Merge branch 'master' into hive f1ca475a [Kent Yao] delta test 01d30291 [Kent Yao] refine 1e93f33a [Kent Yao] Merge branch 'master' of github.com:apache/incubator-kyuubi into hive 5bd96313 [Kent Yao] Unify Hive deps for server and client Authored-by: Kent Yao <yao@apache.org> Signed-off-by: ulysses-you <ulyssesyou@apache.org> |
||
|
Kent Yao
|
d1de9ca529
|
[KYUUBI #1658] Revamp swagger ui dependencies
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> This PR mainly removes the swagger-scala-module which is currently fine to remove. It also introduces transitive deps with `-jakarta` which could introduce jar conflicts like , ```scala Caused by: java.lang.NoClassDefFoundError: jakarta/xml/bind/annotation/XmlElement at io.swagger.v3.core.jackson.SwaggerAnnotationIntrospector.hasRequiredMarker(SwaggerAnnotationIntrospector.java:29) at com.fasterxml.jackson.databind.introspect.AnnotationIntrospectorPair.hasRequiredMarker(AnnotationIntrospectorPair.java:319) at com.fasterxml.jackson.databind.introspect.AnnotationIntrospectorPair.hasRequiredMarker(AnnotationIntrospectorPair.java:319) at com.fasterxml.jackson.databind.introspect.AnnotationIntrospectorPair.hasRequiredMarker(AnnotationIntrospectorPair.java:319) at com.fasterxml.jackson.databind.introspect.POJOPropertyBuilder.getMetadata(POJOPropertyBuilder.java:229) ``` ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1658 from yaooqinn/dep. Closes #1658 d5ab0765 [Kent Yao] Merge branch 'master' into dep d1ddd5fd [Kent Yao] Revamp swagger ui dependencies ecfcc92a [Kent Yao] Revamp swagger ui dependencies 59196b94 [Kent Yao] Revamp swagger ui dependencies d7ca6a5f [Kent Yao] Revamp swagger ui dependencies 215b1b53 [Kent Yao] Revamp swagger ui dependencies Authored-by: Kent Yao <yao@apache.org> Signed-off-by: ulysses-you <ulyssesyou@apache.org> |
||
|
rayliu
|
2af105a417 |
[KYUUBI #1556] Upgrade swagger-ui version from 4.1.0 to 4.1.3
### _Why are the changes needed?_ Close https://github.com/apache/incubator-kyuubi/issues/1612 ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1630 from ldzhjn/1612. Closes #1556 71fe05bc [rayliu] Upgrade swagger-ui version from 4.1.0 to 4.1.3 Authored-by: rayliu <rayliu@cisco.com> Signed-off-by: yanghua <yanghua1127@gmail.com> |
||
|
Cheng Pan
|
77a278707f
|
[KYUUBI #1559] Bump Jersey 2.34
### _Why are the changes needed?_ #1561 and align Spark 3.2.0 ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1559 from pan3793/jersey. Closes #1559 a86d4d9f [Cheng Pan] Bump Jersey 2.34 Authored-by: Cheng Pan <chengpan@apache.org> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
Cheng Pan
|
f841da4b4e
|
[KYUUBI #1558] Bump Jackson 2.12.5
### _Why are the changes needed?_ Because Spark 3.2, Trino 363, Jersey 2.34 (we use Jersey 2.30 now, will update to 2.34 soon) use Jackson 2.12.x ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1558 from pan3793/jackson. Closes #1558 7bdd65b0 [Cheng Pan] Bump Jackson 2.12.5 Authored-by: Cheng Pan <chengpan@apache.org> Signed-off-by: Cheng Pan <chengpan@apache.org> |
||
|
Cheng Pan
|
6fa1327f4e
|
[KYUUBI #1560] Align all Jackson libs version
### _Why are the changes needed?_ Fix missing version of `jackson-module-jaxb-annotations` in `dependencyManagement` ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1560 from pan3793/jackson-1. Closes #1560 49529d47 [Cheng Pan] Update deps b3610007 [Cheng Pan] fix 67044840 [Cheng Pan] Algin all Jackson libs version Authored-by: Cheng Pan <chengpan@apache.org> Signed-off-by: Cheng Pan <chengpan@apache.org> |
||
|
sunfangbin
|
fb8bf49dbc
|
[KYUUBI #1413] Improve usage of zookeeper cli
### _Why are the changes needed?_ This change enabled the JLine support so that the `kyuubi-zookeeper-shell` command can be easier to use with history, prompt and completion, etc. Note that the `jline` version `0.9.94` is dependent on `zookeeper` version `3.4.14` which is a lower version. BTW, users can consider upgrading the `zookeeper` version for the sake of performance improvements and new features. ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1413 from murong00/branch-1413. Closes #1413 2d1ec084 [sunfangbin] Rename shell script to zk-cli d09f64dc [sunfangbin] Fixes failed ci ce88357a [sunfangbin] Improve usage of zookeeper cli Authored-by: sunfangbin <sunfangbin@cmss.chinamobile.com> Signed-off-by: Cheng Pan <chengpan@apache.org> |
||
|
Fu Chen
|
a5ef94accc
|
[KYUUBI #1354] Expose OpenAPI Specifications
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> 1. add a new endpoint `/openapi.json` which describes the entire restful API of Kyuubi. 2. add a static web page `/api/v1/swagger-ui`. see [swagger-ui](https://github.com/swagger-api/swagger-ui) ``` curl http://localhost:10099/openapi.json | jq . ``` ```json { "openapi": "3.0.1", "info": { "title": "Apache Kyuubi RESTful Api Documentation", "description": "Apache Kyuubi RESTful Api Documentation", "contact": { "name": "Apache Kyuubi Community", "url": "https://kyuubi.apache.org/issue_tracking.html", "email": "devkyuubi.apache.org" }, "license": { "name": "Apache 2.0", "url": "https://www.apache.org/licenses/LICENSE-2.0.html" }, "version": "1.4.0" }, "tags": [ { "name": "Session" } ], "paths": { "/api/v1/ping": { "get": { "operationId": "ping", "responses": { "default": { "description": "default response", "content": { "text/plain": { "schema": { "type": "string" } } } } } } }, "/api/v1/sessions/{sessionHandle}": { "get": { "tags": [ "Session" ], "operationId": "sessionInfo", "parameters": [ { "name": "sessionHandle", "in": "path", "required": true, "schema": { "type": "string" } } ], "responses": { "200": { "description": "get a session via session handle identifier", "content": { "application/json": {} } } } }, "delete": { "tags": [ "Session" ], "operationId": "closeSession", "parameters": [ { "name": "sessionHandle", "in": "path", "required": true, "schema": { "type": "string" } } ], "responses": { "200": { "description": "close a Session", "content": { "application/json": {} } } } } }, "/api/v1/sessions/execPool/statistic": { "get": { "tags": [ "Session" ], "operationId": "execPoolStatistic", "responses": { "200": { "description": "get some statistic info of sessions", "content": { "application/json": {} } } } } }, "/api/v1/sessions/{sessionHandle}/info/{infoType}": { "get": { "tags": [ "Session" ], "operationId": "getInfo", "parameters": [ { "name": "sessionHandle", "in": "path", "required": true, "schema": { "type": "string" } }, { "name": "infoType", "in": "path", "required": true, "schema": { "type": "integer", "format": "int32" } } ], "responses": { "200": { "description": "get a information detail via session handle identifier and a specific information type", "content": { "application/json": {} } } } } }, "/api/v1/sessions": { "get": { "tags": [ "Session" ], "operationId": "sessionInfoList", "responses": { "200": { "description": "get all the session list hosted in SessionManager", "content": { "application/json": {} } } } }, "post": { "tags": [ "Session" ], "operationId": "openSession", "requestBody": { "content": { "application/json": { "schema": { "$ref": "#/components/schemas/SessionOpenRequest" } } } }, "responses": { "200": { "description": "Open(create) a Session", "content": { "application/json": {} } } } } }, "/api/v1/sessions/count": { "get": { "tags": [ "Session" ], "operationId": "sessionCount", "responses": { "200": { "description": "get open session count", "content": { "application/json": {} } } } } }, "/api/v1/exception": { "get": { "operationId": "test", "responses": { "default": { "description": "default response", "content": { "text/plain": {} } } } } }, "/application.wadl/{path}": { "get": { "operationId": "getExternalGrammar", "parameters": [ { "name": "path", "in": "path", "required": true, "schema": { "type": "string" } } ], "responses": { "default": { "description": "default response", "content": { "application/xml": {} } } } } }, "/application.wadl": { "get": { "operationId": "getWadl", "responses": { "default": { "description": "default response", "content": { "application/vnd.sun.wadl+xml": {}, "application/xml": {} } } } } } }, "components": { "schemas": { "ExecPoolStatistic": { "required": [ "execPoolSize" ], "type": "object", "properties": { "execPoolSize": { "type": "integer", "format": "int32" }, "execPoolActiveCount": { "type": "integer", "format": "int32" } } }, "InfoDetail": { "required": [ "infoType" ], "type": "object", "properties": { "infoType": { "type": "string" }, "infoValue": { "type": "string" } } }, "HandleIdentifier": { "required": [ "publicId" ], "type": "object", "properties": { "publicId": { "type": "string", "format": "uuid" }, "secretId": { "type": "string", "format": "uuid" } } }, "SessionHandle": { "required": [ "identifier", "protocol" ], "type": "object", "properties": { "identifier": { "$ref": "#/components/schemas/HandleIdentifier" }, "protocol": { "type": "string", "enum": [ "HIVE_CLI_SERVICE_PROTOCOL_V1", "HIVE_CLI_SERVICE_PROTOCOL_V2", "HIVE_CLI_SERVICE_PROTOCOL_V3", "HIVE_CLI_SERVICE_PROTOCOL_V4", "HIVE_CLI_SERVICE_PROTOCOL_V5", "HIVE_CLI_SERVICE_PROTOCOL_V6", "HIVE_CLI_SERVICE_PROTOCOL_V7", "HIVE_CLI_SERVICE_PROTOCOL_V8", "HIVE_CLI_SERVICE_PROTOCOL_V9", "HIVE_CLI_SERVICE_PROTOCOL_V10" ] } } }, "MapStringString": { "type": "object", "additionalProperties": { "type": "string" } }, "SessionOpenRequest": { "required": [ "configs", "protocolVersion", "user" ], "type": "object", "properties": { "protocolVersion": { "type": "integer", "format": "int32" }, "user": { "type": "string" }, "password": { "type": "string" }, "ipAddr": { "type": "string" }, "configs": { "type": "object", "additionalProperties": { "type": "string" } } } }, "SessionOpenCount": { "required": [ "openSessionCount" ], "type": "object", "properties": { "openSessionCount": { "type": "integer", "format": "int32" } } }, "SessionDetail": { "required": [ "configs", "createTime", "user" ], "type": "object", "properties": { "user": { "type": "string" }, "ipAddr": { "type": "string" }, "createTime": { "type": "integer", "format": "int64" }, "sessionHandle": { "$ref": "#/components/schemas/SessionHandle" }, "lastAccessTime": { "type": "integer", "format": "int64" }, "lastIdleTime": { "type": "integer", "format": "int64" }, "noOperationTime": { "type": "integer", "format": "int64" }, "configs": { "type": "object", "additionalProperties": { "type": "string" } } } }, "SeqSessionOverview": { "type": "array", "items": { "$ref": "#/components/schemas/SessionOverview" } }, "SessionList": { "required": [ "sessionList" ], "type": "object", "properties": { "sessionList": { "type": "array", "items": { "$ref": "#/components/schemas/SessionOverview" } } } }, "SessionOverview": { "required": [ "createTime", "user" ], "type": "object", "properties": { "user": { "type": "string" }, "ipAddr": { "type": "string" }, "createTime": { "type": "integer", "format": "int64" }, "sessionHandle": { "$ref": "#/components/schemas/SessionHandle" } } } } } } ``` http://localhost:10099/api/v1/swagger-ui > Note that this request will redirect to http://localhost:10099/swagger-ui-redirected/index.html?url=http://localhost:10099/openapi.json  ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1363 from cfmcgrady/kyuubi-1354. Closes #1354 8895d7fc [Fu Chen] typo 8f5dea9a [Fu Chen] addressed comment 4d325a45 [Fu Chen] constant MediaType.APPLICATION_JSON 0d96951e [Fu Chen] RESTful Api -> REST API be7d02dc [Fu Chen] add LIABILITY-edl.txt 4ff1053e [Fu Chen] update dev/dependencyList b126f7e6 [Fu Chen] swagger-ui 4.1.0 a35c2f29 [Fu Chen] addressed comment 7bde0534 [Fu Chen] update LICENSE-binary 38397dc2 [Fu Chen] align jackson version 923f3e7e [Fu Chen] update dev/dependencyList 7d5338c8 [Fu Chen] remove kyuubi-server static files 5cb66b33 [Fu Chen] update dev/dependencyList e8a8c421 [Fu Chen] fix style 5b13f69c [Fu Chen] add license and remove .map files c04d7a75 [Fu Chen] rebase master 2388dae8 [Fu Chen] fix style 8dcbd3db [Fu Chen] add static resource 4b0d6d0a [Fu Chen] expose OpenAPI Authored-by: Fu Chen <cfmcgrady@gmail.com> Signed-off-by: ulysses-you <ulyssesyou@apache.org> |
||
|
Cheng Pan
|
c122a23a0d
|
[KYUUBI #1348] [DEPS] Add netty and mysql jdbc driver to kyuubi-server module
### _Why are the changes needed?_ This PR is one part of #1334. Add netty to `kyuubi-server` module compile scope, and mysql jdbc driver to test scope, it's required for implementing MySQL frontend protocol ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1348 from pan3793/mysql-fe-deps. Closes #1348 288ad9c0 [Cheng Pan] [DEPS] Add netty to kyuubi-server module Authored-by: Cheng Pan <chengpan@apache.org> Signed-off-by: Cheng Pan <chengpan@apache.org> |
||
|
Brian Yue
|
7665339244
|
[KYUUBI #460] Upgrade Hive dependency to 2.3.9
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> Close #460. ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1179 from byyue/feature-460. Closes #460 92eb71b8 [Brian Yue] Upgrade Hive dependency to 2.3.9 Authored-by: Brian Yue <code.byyue@gmail.com> Signed-off-by: Cheng Pan <chengpan@apache.org> |
||
|
Cheng Pan
|
f03a1443a3
|
[KYUUBI #757] [DEPS] Bump Hadoop 3.3.1
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/NetEase/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> Spark 3.2.0 is out, which bundles Hadoop 3.3.1 shaded client in default. https://github.com/apache/spark/pull/30135 The test failed when using Hadoop 3.3.1 client connects to Yarn Mini Cluster 3.2.2 ``` Cause: java.lang.RuntimeException: org.apache.kyuubi.KyuubiSQLException:java.lang.ClassCastException: org.apache.hadoop.yarn.proto.YarnServiceProtos$GetClusterMetricsRequestProto cannot be cast to org.apache.hadoop.shaded.com.google.protobuf.Message at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:233) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:123) at com.sun.proxy.$Proxy12.getClusterMetrics(Unknown Source) at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getClusterMetrics(ApplicationClientProtocolPBClientImpl.java:271) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359) at com.sun.proxy.$Proxy13.getClusterMetrics(Unknown Source) at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getYarnClusterMetrics(YarnClientImpl.java:638) at org.apache.spark.deploy.yarn.Client.$anonfun$submitApplication$1(Client.scala:179) at org.apache.spark.internal.Logging.logInfo(Logging.scala:57) at org.apache.spark.internal.Logging.logInfo$(Logging.scala:56) at org.apache.spark.deploy.yarn.Client.logInfo(Client.scala:65) at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:179) at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:62) at org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:220) at org.apache.spark.SparkContext.<init>(SparkContext.scala:581) at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2690) at org.apache.spark.sql.SparkSession$Builder.$anonfun$getOrCreate$2(SparkSession.scala:949) at scala.Option.getOrElse(Option.scala:189) at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:943) at org.apache.kyuubi.engine.spark.SparkSQLEngine$.createSpark(SparkSQLEngine.scala:103) at org.apache.kyuubi.engine.spark.SparkSQLEngine$.main(SparkSQLEngine.scala:155) at org.apache.kyuubi.engine.spark.SparkSQLEngine.main(SparkSQLEngine.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52) at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:955) at org.apache.spark.deploy.SparkSubmit$$anon$1.run(SparkSubmit.scala:165) at org.apache.spark.deploy.SparkSubmit$$anon$1.run(SparkSubmit.scala:163) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1878) at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:163) at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203) at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90) at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1043) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1052) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) ``` ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request Closes #757 from pan3793/hadoop-3.3. Closes #757 7ec9313e [Cheng Pan] [DEPS] Bump Hadoop 3.3.1 Authored-by: Cheng Pan <379377944@qq.com> Signed-off-by: ulysses-you <ulyssesyou@apache.org> |
||
|
zhouyifan279
|
114ec9a046
|
[KYUUBI #1009][FOLLOWUP] Kyuubi server failed to start when connecting to secured HMS
### _Why are the changes needed?_ Fix #1161 [Bug] Kyuubi server failed to start when connecting to a secured Hive metastore ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate  - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1162 from zhouyifan279/KYUUBI#1161. Closes #1009 f5b54818 [zhouyifan279] [KYUUBI #1161][Bug] Kyuubi server failed to start when connecting to a secured Hive metastore Authored-by: zhouyifan279 <zhouyifan279@gmail.com> Signed-off-by: Cheng Pan <chengpan@apache.org> |
||
|
Brian Yue
|
cbe5bee6ec
|
[KYUUBI #1112] Upgrade scala to 2.12.15
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> Close #1112. ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1120 from byyue/feature-1112. Closes #1112 9bb10858 [Brian Yue] [FEATURE] [Dependency] Upgrade scala to 2.14.15 Authored-by: Brian Yue <code.byyue@gmail.com> Signed-off-by: Cheng Pan <chengpan@apache.org> |
||
|
zhouyifan279
|
1b89272749
|
[KYUUBI #1009] Implement Hive metastore server Delegation Token Provider
### _Why are the changes needed?_ This PR finishes the work of issue #1009 ### _How was this patch tested?_ - [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1053 from zhouyifan279/KYUUBI#1009. Closes #1009 342fca91 [zhouyifan279] [KYUUBI #1009] Implement Hive metastore server Delegation Token Provider a9e1b34d [zhouyifan279] [KYUUBI #1009] Implement Hive metastore server Delegation Token Provider 82da9007 [zhouyifan279] [KYUUBI #1009] Implement Hive metastore server Delegation Token Provider 2c00dd3a [zhouyifan279] [KYUUBI #1009] Implement Hive metastore server Delegation Token Provider Authored-by: zhouyifan279 <zhouyifan279@gmail.com> Signed-off-by: ulysses-you <ulyssesyou@apache.org> |
||
|
yanghua
|
10d8ab90dd |
[KYUUBI #1031] Implement api: /sessions/count
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1038 from yanghua/KYUUBI-1031. Closes #1031 36ad5b97 [yanghua] Addressed the review concerns 66d8b0fe [yanghua] [KYUUBI #1031] Implement api: /sessions/count Authored-by: yanghua <yanghua1127@gmail.com> Signed-off-by: ulysses-you <ulyssesyou18@gmail.com> |
||
|
Cheng Pan
|
19072f9642 |
[KYUUBI #1042] [DEPS] Bump jersey 2.30
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> Align with `spark-core` https://mvnrepository.com/artifact/org.apache.spark/spark-core_2.12/3.1.2 https://mvnrepository.com/artifact/org.apache.spark/spark-core_2.12/3.0.3 ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1042 from pan3793/jersey. Closes #1042 f5a05cd0 [Cheng Pan] Update dependencyList abb517e2 [Cheng Pan] [DEPS] Bump jersey 2.30 Authored-by: Cheng Pan <chengpan@apache.org> Signed-off-by: ulysses-you <ulyssesyou18@gmail.com> |
||
|
yanghua
|
32111a3014
|
[KYUUBI #999] Build the basic framework for rest frontend service
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1003 from yanghua/KYUUBI-999. Closes #999 78fdf542 [yanghua] Changed rest frontend service port from 10009 to 10099 73929d5d [yanghua] [KYUUBI #999] Build the basic framework for rest frontend service Authored-by: yanghua <yanghua1127@gmail.com> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
hongdongdong
|
d704e4d49d |
[KYUUBI #922] Add OperationEvent for server to track operation state
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> Track operation state change and support output to custom system. ### _How was this patch tested?_ - [X] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #968 from hddong/add-server-events. Closes #922 8be8f268 [hongdongdong] Add prefix for StatementEvent 9fdcc504 [hongdongdong] fix comment a0cdb35d [hongdongdong] fix test 4d340c67 [hongdongdong] add dependency 00920751 [hongdongdong] [KYUUBI#922]Add OperationEvent for server to track operation state Authored-by: hongdongdong <hongdongdong@cmss.chinamobile.com> Signed-off-by: ulysses-you <ulyssesyou18@gmail.com> |
||
|
timothy65535
|
5526898f7b |
[KYUUBI #894] Enhance kyuubi-zookeeper module
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> ### 1. Describe the feature Kyuubi use `TestingServer` as the EmbeddedZookeeper service in codebase, and require many args to init the service. These `electionPort`, `quorumPort`, `serverId` parameters only work in cluster mode, these ports are not bound to the host. From hadoop and bookkeeper project, they use `ZooKeeperServer` as a built-in zk service, especially the bookeeper project. - https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-registry/src/main/java/org/apache/hadoop/registry/server/services/MicroZookeeperService.java - https://github.com/apache/bookkeeper/blob/master/bookkeeper-server/src/main/java/org/apache/bookkeeper/shims/zk/ZooKeeperServerShimImpl.java ### 2. Describe the solution Improve - Remove `electionPort`, `quorumPort`, `serverId` which are nerver be used, hadoop and bookkeer do the same - Replace `TestingServer` with `ZooKeeperServer` - Remove `curator-test` dependency ### 3. `deleteDataDirectoryOnClose` The name style for `deleteDataDirectoryOnClose` option, refer to hadoop `core-default.xml` https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/core-default.xml ``` hadoop.security.dns.log-slow-lookups.enabled hadoop.security.dns.log-slow-lookups.threshold.ms dfs.client.block.write.replace-datanode-on-failure.policy ``` ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request Closes #907 from timothy65535/ky-894. Closes #894 7d7fde9f [timothy65535] remove useless config ce06a0fd [timothy65535] limit travis build only master branch ff825c3f [timothy65535] revert deleteDataDirectoryOnClose config a336544a [timothy65535] remove javassist 3da0bd55 [timothy65535] remove javassist 0abb44df [timothy65535] [KYUUBI #894] Enhance kyuubi-zookeeper module Authored-by: timothy65535 <timothy65535@163.com> Signed-off-by: ulysses-you <ulyssesyou18@gmail.com> |
||
|
hongdongdong
|
16b93e4720
|
[KYUUBI #753] Use scopt to parse arguments
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/NetEase/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> Use [`scopt`](https://github.com/scopt/scopt) replace custom parser to parse arguments. ### _How was this patch tested?_ - [X] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request Closes #754 from hddong/kyuubi-753. Closes #753 3d2645d8 [hongdongdong] fix check style fd181452 [hongdongdong] fix output ef7c44e4 [hongdongdong] use scopt help 682b3a2f [hongdongdong] [KYUUBI#753] Use scopt to parse arguments Authored-by: hongdongdong <hongdongdong@cmss.chinamobile.com> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
Cheng Pan
|
5c5d43b126
|
[KYUUBI #693] [BUILD] [GA] Support JDK 11 and enable it in GitHub Action
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/NetEase/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> Close #550, relate to #559 #688 #689. Add `javax.xml.bind:jaxb-api` and `javax.activation:activation` since those apis has been removed by JDK 11. In GA, we saw a significant performance downgrade in the unit tests, and even with some changes like enlarging timeout, using in-memory catalog instead of hive catalog, the JDK11(~43min) tests cost 2 times than JDK8(~22min), this performance regression is tracked in #745. ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request Closes #693 from pan3793/jdk11. Closes #693 5706daf1 [Cheng Pan] shrink scalatest memory 6ebac68e [Cheng Pan] address comments 2b479391 [Cheng Pan] address comments 5fa74aa0 [Cheng Pan] disable test in kyuubi-extension-spark_3.1 9f070a40 [Cheng Pan] maven profile for jdk versions 801e4cd1 [Cheng Pan] jdk11 in GA e86893cc [Cheng Pan] enlarge YarnCluster timeout c880edd1 [Cheng Pan] minor 38d2775c [Cheng Pan] enlarge engine crash timeout a945ade3 [Cheng Pan] use in-memory catalog in EngineRefSuite 51604d73 [Cheng Pan] enlarge ENGINE_INIT_TIMEOUT 7baeb965 [Cheng Pan] minicluster a7e4bdf1 [Cheng Pan] GA matrix b29aa2f4 [Cheng Pan] license for jaxb 4283777b [Cheng Pan] Add dependency jaxb-api c5b0e580 [Cheng Pan] [GA] Enable JDK 11 in GA Lead-authored-by: Cheng Pan <chengpan@apache.org> Co-authored-by: Cheng Pan <379377944@qq.com> Signed-off-by: Cheng Pan <chengpan@apache.org> |
||
|
wuchunfu
|
4d2fee5dcc
|
[KYUUBI #694] Upgrade Scala to 2.12.14
### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request Closes #721 from wuchunfu/694. Closes #694 10b6774a [wuchunfu] [KYUUBI #694] replace dev/dependencyList efafeab5 [wuchunfu] [KYUUBI #694] Upgrade Scala to 2.12.14 Authored-by: wuchunfu <319355703@qq.com> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
Cheng Pan
|
5276cfbecb
|
[KYUUBI #711] [BUILD] Bump jetty 9.4.41.v20210516
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/NetEase/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> close #710 ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request Closes #711 from pan3793/jetty. Closes #711 95bd9a32 [Cheng Pan] [BUILD] Bump jetty 9.4.41.v20210516 Authored-by: Cheng Pan <379377944@qq.com> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
Cheng Pan
|
0b4f4f242c
|
[KYUUBI #630] [METRICS] Support expose Prometheus metrics
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/NetEase/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> Support expose Prometheus metrics ### _How was this patch tested?_ - [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request Closes #630 from pan3793/prometheus. Closes #630 fff68b0 [Cheng Pan] nit 47c19eb [Cheng Pan] simplify 203e776 [Cheng Pan] address comments 10a877c [Cheng Pan] synchronized 26eb26e [Cheng Pan] simplify random port 827427c [Cheng Pan] fix import 476ee3a [Cheng Pan] address comments 9787c77 [Cheng Pan] update ut c8e1b0e [Cheng Pan] update license 2387a97 [Cheng Pan] [METRICS] Support expose Prometheus metrics Authored-by: Cheng Pan <379377944@qq.com> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
Kent Yao
|
9e7649296d
|
[KYUUBI #449] Add Basic Metrics Service framework
 [](https://github.com/yaooqinn/kyuubi/pull/450)    [<img width="16" alt="Powered by Pull Request Badge" src="https://user-images.githubusercontent.com/1393946/111216524-d2bb8e00-85d4-11eb-821b-ed4c00989c02.png">](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info)<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT --> <!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> This PR adds a future-proofing Metrics Service framework close #499 add some reporters, like json file, jmx, etc.. add some basic metrics, like statement.error ### _How was this patch tested?_ - [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible WIP - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request Closes #450 from yaooqinn/449. ea21a43 [Kent Yao] add tests 6513d07 [Kent Yao] update codecov 7cfe318 [Kent Yao] update docs a8454d0 [Kent Yao] update docs 91c33f4 [Kent Yao] update deps 05b13fb [Kent Yao] fix dep issues babc739 [Kent Yao] Add Basic Metrics Service Platform acc716a [Kent Yao] Add Basic Metrics Service Platform Authored-by: Kent Yao <yao@apache.org> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
Kent Yao
|
354f55a0a5
|
[KYUUBI #416] Detected Dependency List Change for better dependency management
 [](https://github.com/yaooqinn/kyuubi/pull/416)     [❨?❩](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info)<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT --> <!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> Detect Dependency List Change, to let us and users know what actually release with Kyuubi ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request Closes #416 from yaooqinn/dep. 5938423 [Kent Yao] update doc 037c897 [Kent Yao] nit 2b06bf9 [Kent Yao] Detected Dependency List Change for better dependency management Authored-by: Kent Yao <yao@apache.org> Signed-off-by: Kent Yao <yao@apache.org> |
