### _Why are the changes needed?_
This PR aims to migrate the vanilla Zookeeper and Curator to the Kyuubi Shaded Zookeeper. It's the first step to adapting JDK 17.
There is a known issue [ZOOKEEPER-3779](https://issues.apache.org/jira/browse/ZOOKEEPER-3779) that Zookeeper 3.4 client can not run on JDK 14 and above, in https://github.com/apache/kyuubi-shaded/pull/5, we fixed this issue by a surgical.
With the above fixing, zk-3.4 and zk-3.6 clients both work well on JDK 17, we just randomly pick some cases to make sure zk-3.6 is tested
zk-3.4 client supports zk-3.4+ server, but zk-3.6 client only supports zk-3.5+ server; in the meanwhile, zk-3.4 is adopted widely, (CDH 5/6, HDP, EMR created before 2023).
We are sticky to zk-3.4 to ensure that Kyuubi can be out-of-box in the most existing Hadoop cluster but also provide zk-3.6 as an alternative(simply replace the kyuubi-shaded-zk-3.4 jar w/ kyuubi-shaded-zk-3.6, or build w/ -Pzookeeper-3.6) for users who concerns that zk-3.4 is EOL.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4852 from pan3793/shaded-zk.
Closes#4852
d960cc945 [Cheng Pan] remove staging repo
1b3622080 [Cheng Pan] Switch to Kyuubi Shaded Zookeeper
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
This PR aims to remove Hudi integration tests from the Kyuubi project.
Actually, there is no obvious benefit to running Hudi tests w/ Kyuubi, since the real work happens on the compute engine and Hudi integration. Besides, Hudi's horrible dependency management brings significant maintenance efforts to the Kyuubi community.
This change only affects tests, does not affect any functionality.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4744 from pan3793/remove-hudi.
Closes#4744
ea99f747e [Cheng Pan] Remove Hudi integration tests
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- to consolidate styles in markdown files from manual written or auto-generated
- apply markdown formatting rules with flexmark from [spotless-maven-plugin](https://github.com/diffplug/spotless/tree/main/plugin-maven#markdown) to *.md files in `/docs`
- use `flexmark` to format markdown generation in `TestUtils` of common module used by `AllKyuubiConfiguration` and `KyuubiDefinedFunctionSuite`, as the same way in `FlexmarkFormatterFunc ` of `spotless-maven-plugin` using with `COMMONMARK` as `FORMATTER_EMULATION_PROFILE` (https://github.com/diffplug/spotless/blob/maven/2.30.0/lib/src/flexmark/java/com/diffplug/spotless/glue/markdown/FlexmarkFormatterFunc.java)
- using `flexmark` of` 0.62.2`, as the last version requiring Java 8+ (checked from pom file and bytecode version)
```
<markdown>
<includes>
<include>docs/**/*.md</include>
</includes>
<flexmark></flexmark>
</markdown>
```
- Changes applied to markdown doc files,
- no style change or breakings in built docs by `make html`
- removal all the first blank in licences and comments to conform markdown style rules
- tables regenerated by flexmark following as in [GitHub Flavored Markdown](https://help.github.com/articles/organizing-information-with-tables/) (https://github.com/vsch/flexmark-java/wiki/Extensions#tables)
### _How was this patch tested?_
- [x] regenerate docs using `make html` successfully and check all the markdown pages available
- [x] regenerate `settings.md` and `functions.md` by `AllKyuubiConfiguration` and `KyuubiDefinedFunctionSuite`, and pass the checks by both themselves and spotless check via `dev/reformat`
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4200 from bowenliang123/markdown-formatting.
Closes#4200
1eeafce4 [liangbowen] revert minor changes in AllKyuubiConfiguration
4f892857 [liangbowen] use flexmark in markdown doc generation
8c978abd [liangbowen] changes on markdown files
a9190556 [liangbowen] apply markdown formatting rules with `spotless-maven-plugin` to markdown files with in `/docs`
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
This pr aims to suppress warnning of the maven-antrun-plugin echo;
before:
```log
[WARNING] [echo] unpacking netty jar
[WARNING] [echo] renaming netty native libraries
[WARNING] [echo] deleting META-INF/native-image folder
[WARNING] [echo] repackaging netty jar
```
after:
```log
[INFO] [echo] unpacking netty jar
[INFO] [echo] renaming netty native libraries
[INFO] [echo] deleting META-INF/native-image folder
[INFO] [echo] repackaging netty jar
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4050 from Yikf/echo.
Closes#4050
2729f7988 [Yikf] Suppress warnning of the maven-antrun-plugin echo
Authored-by: Yikf <yikaifei1@gmail.com>
Signed-off-by: fwang12 <fwang12@ebay.com>
### _Why are the changes needed?_
1. `mvn antrun:runbuild-info -pl kyuubi-common`
2. debug engine suite
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3847 from cxzl25/profile_jdbc_debug.
Closes#3847
e1c32067 [Cheng Pan] Merge branch 'master' into profile_jdbc_debug
f41d3cd9 [sychen] use jdbc-shaded
25b54b18 [sychen] use jdbc-shaded
dd23f864 [sychen] use kyuubi-hive-jdbc as default
79b0ffcb [sychen] add jdbc profile
Lead-authored-by: sychen <sychen@ctrip.com>
Co-authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Introduce code style check support for Maven's pom.xml with sortPom in spotless maven plugin.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3843 from bowenliang123/spotless-pom.
Closes#3842
3c654597 [liangbowen] apply to pom.xml
fd1536f7 [liangbowen] set expandEmptyElements to true
e498423f [liangbowen] apply spotless:apply to all pom.xml
e46bcfec [liangbowen] add pom style check support in spotless
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [X] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3714 from hddong/use-ServicesResourceTransformer.
Closes#3714
d4c0ed27 [hongdongdong] Use ServicesResourceTransformer to concatenating service entries
Authored-by: hongdongdong <hongdd@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Close https://github.com/apache/incubator-kyuubi/issues/3566.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3570 from Yikf/shade-spark-engine.
Closes#3566
0290cbc9 [Yikf] shade zookeeper in spark-engine
Authored-by: Yikf <yikaifei1@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Preparing v1.7.0-SNAPSHOT with branch-1.6 cut
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3264 from SteNicholas/prepare-1.7.0-snapshot.
Closes#3264
374d56bf [SteNicholas] preparing v1.7.0-SNAPSHOT with branch-1.6 cut
Authored-by: SteNicholas <programgeek@163.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
https://github.com/apache/iceberg/releases/tag/apache-iceberg-0.14.0
Notable changes:
- Support Spark 3.3
- Unified package name
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3050 from pan3793/iceberg.
Closes#3050
3e700a11 [Cheng Pan] Fix ut caused by SPARK-37931
039c4e2d [Cheng Pan] Enable Iceberg test w/ Spark 3.3
53cac396 [Cheng Pan] Bump Apache Iceberg 0.14.0
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Add etcd discovery client for HA
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2767 from hddong/etcd-support.
Closes#2644
6fa37578 [hongdongdong] Rebase and remove from rat-excludes
fc23a2b4 [hongdongdong] upgrade grpc-1.47.0
61c77d27 [hongdongdong] Fix tests
28abd13e [hongdongdong] fix comments
9e81a6a3 [hongdongdong] Rename HA_ZK_ENGINE_REF_ID to HA_ENGINE_REF_ID
aa2b4260 [hongdongdong] [KYUUBI #2644][WIP] Add etcd discovery client for HA
Authored-by: hongdongdong <hongdongdong@cmss.chinamobile.com>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
Handles configuring the JUL -> SLF4J bridge, otherwise, the components which use JUL like Jersey will be not controlled by log4j2 configurations
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3037 from pan3793/jul.
Closes#3037
fda05ff6 [Cheng Pan] update license
31f579fd [Cheng Pan] nit
7da0f9fb [Cheng Pan] Fix dependencyList
3179e6bd [Cheng Pan] Handles configuring the JUL -> SLF4J bridge
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- Resolve WARNING prompt for IntelliJ IDEA Build Panel
- Re-adjusted the order of several hudi moudles to make it consistent in the order of server and engine
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2659 from cxzl25/KYUUBI-2576-FOLLOWUP.
Closes#2576
e6c5abf5 [sychen] avoid warning & reformat
Authored-by: sychen <sychen@ctrip.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
https://hudi.apache.org/releases/release-0.11.0
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2576 from cxzl25/bump_hudi_11_version.
Closes#2576
ed643c69 [sychen] exclude extra dependency
39ee155a [sychen] trigger test
1d65e401 [sychen] comment
da252538 [sychen] Revert "try use hudi spark bundle"
038bb45b [sychen] Revert "constant"
77fd0f29 [sychen] Revert "add commons-codec test scope"
0aa0fc82 [sychen] Revert "use spark version judge"
1cdeeb9e [sychen] use spark version judge
0a8fb139 [sychen] add commons-codec test scope
c6b3c9a3 [sychen] constant
2b889e28 [sychen] try use hudi spark bundle
0df19c78 [sychen] remove hadoop-common dependency
1290cf90 [sychen] bump hudi version
Authored-by: sychen <sychen@ctrip.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Separate events to a submodule - kyuubi-event
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2392 from zhaomin1423/event-module.
Closes#2323
7efa923f [Min Zhao] fix unit test
07126f43 [Min Zhao] [KYUUBI #2323] Separate events to a submodule - kyuubi-event
cccbdf11 [Min Zhao] [KYUUBI #2323] Separate events to a submodule - kyuubi-event
194aca82 [Min Zhao] add shade and log4j to test/resources
f75891a1 [Min Zhao] [KYUUBI #2323] Separate events to a submodule - kyuubi-event
357395fa [Min Zhao] [KYUUBI #2323] Separate events to a submodule - kyuubi-event
b6dfd186 [Min Zhao] [KYUUBI #2323] Separate events to a submodule - kyuubi-event
fc876688 [Min Zhao] [KYUUBI #2323] Separate events to a submodule - kyuubi-event
1b86d41b [Min Zhao] seperate event
Authored-by: Min Zhao <zhaomin1423@163.com>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
The spark sql engine should use the same hive-service-rpc version with that of kyuubi server to prevent conflicts with spark native hive service rpc library.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2221 from turboFei/same_service_rpc.
Closes#2221
66a35305 [Fei Wang] address comments
c9d8d9b1 [Fei Wang] add notice
4b46d258 [Fei Wang] The spark sql engine should use the same hive-service-rpc with kyuubi server
Authored-by: Fei Wang <fwang12@ebay.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
### _Why are the changes needed?_
This PR reverts "[KYUUBI #1948] Upgrade thrift version to 0.16.0" in branch-1.5 and some subsequent commits due to issues caused by breaking changes in Thrift, e.g. #2101, #2093
We need more time to evaluate it and add more test cases to ensure stability before the next release.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2103 from pan3793/revert.
Closes#2103Closes#1948
575dfe8d [Cheng Pan] properties order
4c94481e [Cheng Pan] fix deps
bfbdeb89 [Cheng Pan] Add log4j deps for kyuubi-hive-jdbc test
bad4e90e [Cheng Pan] surefire
f2ae8b9b [Cheng Pan] Revert "[KYUUBI #1948] Upgrade thrift version to 0.16.0"
4501f743 [Cheng Pan] Revert "[KYUUBI #1948][FOLLOWUP] Remove TFramedTransport in extension-spark-3-1 and extension-spark-3-2"
cc04df49 [Cheng Pan] Revert "[KYUUBI #1996] Clean up log4j2 conf and enhance surefire plugin conf"
52a20e17 [Cheng Pan] Revert "[KYUUBI #1948][FOLLOWUP] Relocate fb303 classes"
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
We should relocate fb303 classes to avoid conflict, fix#2093.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2096 from SteNicholas/KYUUBI-2093.
Closes#1948
2095f835 [SteNicholas] [KYUUBI #2093] kyuubi beeline can not connect the kyuubi server due to exception
Authored-by: SteNicholas <programgeek@163.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Preparing v1.6.0-SNAPSHOT with branch-1.5 cut
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2045 from yaooqinn/pv.
Closes#2045
6a506a8f [Kent Yao] Preparing v1.6.0-SNAPSHOT
Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
Upgrade libthrift to 0.16.0 due to [CVE-2020-13949](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-13949) and the coming upstream change of Spark https://github.com/apache/spark/pull/34362
### _What changed in this PR?_
- Upgrade libthrift to 0.16.0
- Shade and relocate `thrift` and `hive-service-rpc` classes in `kyuubi-spark-engine`, it's necessary to avoide conflicting with old thrift libs bundled in Spark binary releases.
- Due to thrift change the method signature, the subclasses those interfaces in Kyuubi also need to modify to pass compile.
We rely on Hive 2.3.9 jars in some components, e.g. `kyuubi-hive-jdbc`, `LocalMetaServer` in `kyuubi-server` test classes.
Some classes in Hive jars compiled against old thrift interfaces which are not compatible with thrift 0.16.0, it causes runtime link error, we found the following classes which breaks the test and copied them with necessary modification to make it work with thrift 0.16.0.
- `TFramedTransport`
- `TFilterTransport`
- `TUGIAssumingTransport`
- `TUGIContainingTransport`
- Next Steps, I think it's worth to do in separated PRs.
- Recover the `HiveDelegationTokenProviderSuite`, one approach is use an isolate classloader to load HMS classes and thrift 0.9.3 classes from Maven, this approach can also be used for the planed Zoopkeeper upgrading to help us verficating the compatibility of Zookeeper Server 3.4.x.
- Rewrite `kyuubi-hive-jdbc` to make it decouple with Hive jars, because there maybe other places which may not work with thrift 0.16.0 but the UTs does not cover.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1953 from SteNicholas/KYUUBI-1948.
Closes#1948
de5d1ea2 [SteNicholas] [KYUUBI #1948] Upgrade thrift version to 0.16.0
898effcd [SteNicholas] [KYUUBI #1948] Upgrade thrift version to 0.16.0
803e270c [SteNicholas] [KYUUBI #1948] Upgrade thrift version to 0.16.0
Authored-by: SteNicholas <programgeek@163.com>
Signed-off-by: Kent Yao <yao@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
Take spark as an example.
[link](https://mvnrepository.com/artifact/org.apache.spark/spark-core_2.13/3.2.1)
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1949 from yaooqinn/url.
Closes#1949
f88c0598 [Kent Yao] [INFA] Add homepage for every module in maven cental
684f588f [Kent Yao] [INFA] Add homepage for every module in maven cental
Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1441 from turboFei/bump_1.5.0.
Closes#1441
0c03c75d [fwang12] [BUILD] Bump 1.5.0-SNAPSHOT
Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: fwang12 <fwang12@ebay.com>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
This partial revert [KYUUBI #1203], recover codecov
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1238 from pan3793/mvn.
Closes#1238
b3ba87b2 [Cheng Pan] remove unnecessary changes
c124340a [Cheng Pan] nit
39c436be [Cheng Pan] pom
28546cbd [Cheng Pan] CI
f22e9c91 [Cheng Pan] nit
72e9542d [Cheng Pan] Rebase master
7cd1cba3 [Cheng Pan] [BUILD] Add back Scala stuff to root POM
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
Remove dead module dependency.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1227 from timothy65535/master.
Closes#1227
13dec05d [timothy65535] Remove kyuubi-spark-monitor dependency from kyuubi engine module
Authored-by: timothy65535 <timothy65535@163.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
clean dead code
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1223 from yaooqinn/mon.
Closes#1223
3b7a9a57 [Kent Yao] Remove dead module kyuubi-spark-monitor
b1e2239e [Kent Yao] Remove dead module kyuubi-spark-monitor
Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
Sub task of #1131
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1152 from pan3793/ut-kyuubi-hive-jdbc.
Closes#1152
2f508356 [Cheng Pan] Avoid hardcode Hive Jdbc class
5032d23d [Cheng Pan] [KYUUBI #1131][TEST] Replace hive-jdbc by kyuubi-hive-jdbc in UT
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
#1131
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1203 from pan3793/pom.
Closes#1203
8e6b8f9b [Cheng Pan] fix
d31fd198 [Cheng Pan] fix
9da82843 [Cheng Pan] fix
6d150de9 [Cheng Pan] fix
9faf77e5 [Cheng Pan] Remove Scala stuff from kyuubi-hive-jdbc pom
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
It's pre-step of #1152, otherwise will cause circular dependence.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1151 from pan3793/move-jdbc-test.
Closes#1131
e65f9979 [Cheng Pan] nit
f2e66535 [Cheng Pan] [KYUUBI #1131][TEST] Move KyuubiHiveDriverSuite to kyuubi-spark-sql-engine module
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
HBase deps is optional for creating hudi table and query, but required for writing data to hudi table, as currently we only test create table and query for data lake formats, it's reasonable to remove the hbase deps to reduce the deps pulled from hudi.
This PR also change all data lake test suites in spark engine from `hive` catalog to `in-memory` catalog.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1028 from pan3793/hudi.
Closes#703
56c41a30 [Cheng Pan] Use in-memory catalog for DataLake test
b582a865 [Cheng Pan] Revert "Default use in-memory in Spark engine test"
e62c403a [Cheng Pan] Default use in-memory in Spark engine test
e2b8e0fd [Cheng Pan] Remove unused hudi spark conf
069fbacc [Cheng Pan] Revert "HFileBootstrapIndex"
ca1b20e1 [Cheng Pan] HFileBootstrapIndex
da37b0fc [Cheng Pan] Remove log4j
41ce8526 [Cheng Pan] Align jackson version
f56f5e89 [Cheng Pan] Remove HBase dependencies
c2b76b54 [Cheng Pan] Exclude hudi-timeline-service
56641762 [Cheng Pan] nit
e588c12f [Cheng Pan] [DEPS][TEST] Reduce Hudi dependencies
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Hudi has add Spark Sql Support in version 0.9.0.
This PR add Hudi-0.9.0 dependency and Unit Testing #703
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#994 from simon824/master.
Closes#703
a04b2e42 [simon] exclude jdk.tools dep
50e1ab01 [simon] fix jdk11 dep
f33ebdf1 [simon] fix hudi dep conflicts
ff2585c2 [simon] fix yarn dep conflicts
32dd1ea8 [simon] fix scala version conflicts
b8a37401 [simon] add spark.sql.catalogImplementation
82f5422f [simon] fix spark3 dependency
2a6c497d [simon] fix spark3 dependency
4dbec8be [Simon] Merge branch 'apache:master' into master
3f180157 [simon] fix maven
73e48d0a [simon] add spark3 support maven dependency
3def658b [simon] fix missing spark-sql-engine hudi maven dependency
524132d3 [simon] bugfix
e98998a6 [simon] fix pom error
59fc6669 [Simon] Merge branch 'apache:master' into master
d9e17ebf [simon] fix conflict
2466ece3 [simon] add sparkSQL test on hudi-0.9.0
Lead-authored-by: simon <zhangshiming@cvte.com>
Co-authored-by: Simon <3656562@qq.com>
Signed-off-by: Kent Yao <yao@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#919 from pan3793/asf-misc.
Closes#919
336a4ea0 [Cheng Pan] Update
30ba74fb [Cheng Pan] Update developers
db60eac4 [Cheng Pan] Update developers
fcc9adc9 [Cheng Pan] Cleanup pom
70b6596a [Cheng Pan] Add DISCLAIMER
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
### 1. Describe the feature
Kyuubi use `TestingServer` as the EmbeddedZookeeper service in codebase, and require many args to init the service. These `electionPort`, `quorumPort`, `serverId` parameters only work in cluster mode, these ports are not bound to the host.
From hadoop and bookkeeper project, they use `ZooKeeperServer` as a built-in zk service, especially the bookeeper project.
- https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-registry/src/main/java/org/apache/hadoop/registry/server/services/MicroZookeeperService.java
- https://github.com/apache/bookkeeper/blob/master/bookkeeper-server/src/main/java/org/apache/bookkeeper/shims/zk/ZooKeeperServerShimImpl.java
### 2. Describe the solution
Improve
- Remove `electionPort`, `quorumPort`, `serverId` which are nerver be used, hadoop and bookkeer do the same
- Replace `TestingServer` with `ZooKeeperServer`
- Remove `curator-test` dependency
### 3. `deleteDataDirectoryOnClose`
The name style for `deleteDataDirectoryOnClose` option, refer to hadoop `core-default.xml`
https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/core-default.xml
```
hadoop.security.dns.log-slow-lookups.enabled
hadoop.security.dns.log-slow-lookups.threshold.ms
dfs.client.block.write.replace-datanode-on-failure.policy
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request
Closes#907 from timothy65535/ky-894.
Closes#894
7d7fde9f [timothy65535] remove useless config
ce06a0fd [timothy65535] limit travis build only master branch
ff825c3f [timothy65535] revert deleteDataDirectoryOnClose config
a336544a [timothy65535] remove javassist
3da0bd55 [timothy65535] remove javassist
0abb44df [timothy65535] [KYUUBI #894] Enhance kyuubi-zookeeper module
Authored-by: timothy65535 <timothy65535@163.com>
Signed-off-by: ulysses-you <ulyssesyou18@gmail.com>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
FSDataOutputStream is used for this
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request
Closes#892 from yaooqinn/2021080417.
Closes#892
e291a73c [Kent Yao] nit
e6ebef1a [Kent Yao] Make JsonEventLogger compatible with HDFS-like FileSystems
Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
- Add scala version suffix on scala modules to match the name pattern.
- Enable `kyuubi-extension-spark-3-1` in daily publish snapshot
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request
Closes#888 from pan3793/scala.
Closes#888
571eb9c3 [Cheng Pan] comments
16cb263a [Cheng Pan] remove hardcode scala version in pom
6090c798 [Cheng Pan] Fix shade
5178847f [Cheng Pan] Fixup
a1547753 [Cheng Pan] Fix engine jar name
2cf4793e [Cheng Pan] fix
cac96531 [Cheng Pan] Enable kyuubi-extension-spark-3-1 in daily publish snapshot
1ca31de5 [Cheng Pan] Add scala version suffix on scala modules
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: ulysses-you <ulyssesyou18@gmail.com>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request
Closes#885 from yaooqinn/engineevent.
Closes#885
6a7ac341 [Kent Yao] refine
41baf0e7 [Kent Yao] refine
2c141a06 [Kent Yao] test history server
d1b0823a [Kent Yao] test history server
3aae8417 [Kent Yao] dump the final engine status
257e50f5 [Kent Yao] Add Event for Engine
f1f1f0e8 [Kent Yao] Add Event for Engine
85addc6e [Kent Yao] Add Event for Engine
cece5888 [Kent Yao] Add Event for Engine
Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
https://infra.apache.org/publishing-maven-artifacts.htmlhttps://issues.apache.org/jira/browse/INFRA-22114
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request
Closes#849 from pan3793/asf.
Closes#849
c609f6ba [Cheng Pan] update pom
31f1f2e1 [Cheng Pan] rename root project name to kyuubi-parent
9c5616d8 [Cheng Pan] [INFRA] Update POM for Apache release
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request
Closes#823 from pan3793/deps.
Closes#823
51cdcb5d [Cheng Pan] restore thrift deps from spark-hive
148038a3 [Cheng Pan] [BUILD] [DEPS] Leverage maven to resolve transitive dependencies except hive
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
Generate KStatement info. This object includes the following elements:
1. statement
2. statementId
3. appId
4. sessionId
5. executionId
6. physicalPlan
7. stateTime: contains each state and the time occurrence
Those data was packaged in KStatement and we store those object in mem.
You can get some summary data from this object.
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/NetEase/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
Aiming to make a better supported client for Kyuubi and Spark
- Add catalog to getTables meta function for DataLakes (DONE)
- Deploy to maven central (PENDING)
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request
Closes#762 from yaooqinn/kyuubi-jdbc.
Closes#762
5a881c7b [Kent Yao] address comments
6f01f8e0 [Kent Yao] Add Kyuubi Hive JDBC Module
Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/NetEase/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request
Closes#613 from pan3793/delta.
Closes#613
ccadedb [Cheng Pan] [TEST] [DELTA] Add test for deltalake
Authored-by: Cheng Pan <379377944@qq.com>
Signed-off-by: Kent Yao <yao@apache.org>
