### _Why are the changes needed?_
Support to set result max rows for spark engine.
And it will only take affect if incremental mode is not enabled.
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2225 from turboFei/max_res.
Closes#2225
bb2acd8b [Fei Wang] nit
5d827d11 [Fei Wang] address comments
3193d22e [Fei Wang] address comments
51983c35 [Fei Wang] address comments
2904b4ce [Fei Wang] address comments
f2a8aa62 [Fei Wang] fix docs
232f975c [Fei Wang] add ut
508b3560 [Fei Wang] Add engine spark max rows
Authored-by: Fei Wang <fwang12@ebay.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
To close#2102
Support to retry all the internal thrift request calls(except RenewDelegationToken now), and fast fail if the remote engine is not stable or not alive.
In this PR, it supports engine liveness probe.
If it is enabled, a companion thrift client will be created and open a liveness probe session when opening remote engine session.
It will send some simple thrift request(GetInfo) to check whether the remote engine is alive, and fast fail before retry if remote engine is not connectable.
#### Why not use the same thrift client to check engine liveness before retry?
I tried that, but met `out of resp sequence` error.
For example:
1. send getOperationStatus request
2. read time out
3. send GetInfoType request
4. receive getOperationStatus response (out of resp sequence)
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2122 from turboFei/retry_rpc.
Closes#2102
3926ba04 [Fei Wang] adress comments
ade4ede6 [Fei Wang] add timeout
1b7a64f9 [Fei Wang] Only check remote engine alive before retry
98e03f8e [Fei Wang] refactor
fac388cf [Fei Wang] remove unused import
9c6d8737 [Fei Wang] add ut
9b595650 [Fei Wang] Support to retry the thrift request and engine alive probe
Authored-by: Fei Wang <fwang12@ebay.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
### _Why are the changes needed?_
Based on this https://github.com/apache/incubator-kyuubi/pull/2162
Add several configuration items to control the output of the progress bar.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2168 from cxzl25/KYUUBI-2119-FOLLOWUP.
Closes#2119
36db8c5f [sychen] Add configuration progress bar
Authored-by: sychen <sychen@trip.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
Bump Flink 1.14.4 version.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2173 from deadwind4/flink-1.14.4.
Closes#2172
1be8ed91 [Ada Wong] [KYUUBI #2172][BUILD] Bump Flink 1.14.4 version
Authored-by: Ada Wong <rsl4@foxmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Add ProcBuilder for HiveEngine.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2146 from zhaomin1423/hive_process_builder.
Closes#2024
1baf1397 [Min Zhao] [KYUUBI #2024] Hive Backend Engine - ProcBuilder for HiveEngine
23fdc3b7 [Min Zhao] [KYUUBI #2024] Hive Backend Engine - ProcBuilder for HiveEngine
Authored-by: Min Zhao <zhaomin1423@163.com>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
fix#2157 cofiguration format error
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2165 from jiaoqingbo/2156-followup.
Closes#2156
9862ddb2 [jiaoqingbo] [KYUUBI #2156][FOLLOWUP] Fix configuration format in document
47157c5c [jiaoqingbo] Merge branch 'master' into 2156-followup
9c89c081 [jiaoqingbo] [KYUUBI #2156][FOLLOWUP]fix cofiguration format
46084328 [jiaoqingbo] Merge branch 'master' into 2156
96a22e59 [jiaoqingbo] add link to doc
dc11a214 [jiaoqingbo] [KYUUBI #2156] Change log to reflect exactly why getting token failed
Authored-by: jiaoqingbo <1178404354@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
fix#2156
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2157 from jiaoqingbo/2156.
Closes#2156
96a22e59 [jiaoqingbo] add link to doc
dc11a214 [jiaoqingbo] [KYUUBI #2156] Change log to reflect exactly why getting token failed
Authored-by: jiaoqingbo <1178404354@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Add a `dev/reformat` usage in `developer.md`.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2149 from deadwind4/reformat-doc.
Closes#2148
0457d879 [Ada Wong] fix typo
67af7e4b [Ada Wong] [KYUUBI #2148][DOCS] Add dev/reformat usage
Authored-by: Ada Wong <rsl4@foxmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
helm安装kyuubi服务完成后会提示：
```script
NAME: kyuubi-helm
LAST DEPLOYED: Wed Mar 16 13:14:32 2022
NAMESPACE: kyuubi
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
#
# Licensed to the Apache Software Foundation (ASF) under one or more
......
Get kyuubi expose URL by running these commands:
**export NODE_PORT=$(kubectl get --namespace kyuubi -o jsonpath="{.spec.ports[0].nodePort}" services kyuubi-helm-nodeport)**
export NODE_IP=$(kubectl get nodes --namespace kyuubi -o jsonpath="{.items[0].status.addresses[0].address}")
echo $NODE_IP:$NODE_PORT
```
使用命令
```
export NODE_PORT=$(kubectl get --namespace kyuubi -o jsonpath="{.spec.ports[0].nodePort}" services kyuubi-helm-nodeport)
```
会得到一个报错：
```
Error from server (NotFound): services "kyuubi-helm-nodeport" not found
```
使用`kubectl get svc -A`，查询到namespace：kyuubi下只有一个service："kyuubi-svc"，运行
```
export NODE_PORT=$(kubectl get --namespace kyuubi -o jsonpath="{.spec.ports[0].nodePort}" services kyuubi-svc)
```
成功。so这里给的serviceName是否是错的？
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2150 from HYBG-1126/HYBG-1126.
Closes#2150
ea152f33 [liuzy04] helm安装kyuubi服务完成后会提示： NAME: kyuubi-helm LAST DEPLOYED: Wed Mar 16 13:14:32 2022 NAMESPACE: kyuubi STATUS: deployed REVISION: 1 TEST SUITE: None NOTES: # # Licensed to the Apache Software Foundation (ASF) under one or more ......
Authored-by: liuzy04 <liuzy04@mail.jj.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Add IDEA setup guide for developers.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2143 from deadwind4/KYUUBI-2142.
Closes#2143
f0450c05 [Ada Wong] Update formatter for Java
be77cfe5 [Ada Wong] [KYUBBI #2142][DOCS] Add IDEA setup guide
Authored-by: Ada Wong <rsl4@foxmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Follow up #1936
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2092 from lightning-L/kyuubi-1936.
Closes#1936
2bb2c10d [Tianlin Liao] [KYUUBI #1936][FOLLOWUP] Send credentials when opening session and wait for completion
Authored-by: Tianlin Liao <tiliao@ebay.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
fix#2084
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2091 from jiaoqingbo/2084.
Closes#2084
624216fa [Cheng Pan] permission
c45a71ce [jiaoqingbo] delete empty line
29987bf1 [jiaoqingbo] fix ut failed
2e7f0acc [jiaoqingbo] Merge branch 'master' into 2084
04c0f8e1 [jiaoqingbo] Merge branch 'master' into 2084
1267d195 [jiaoqingbo] Merge branch '2084' of https://github.com/jiaoqingbo/incubator-kyuubi into 2084
1bb2b6d6 [jiaoqingbo] Merge branch 'master' into 2084
db4a4aac [jiaoqingbo] Merge branch 'master' into 2084
82097388 [jiaoqingbo] ignore java/sun prefix
d4640ec3 [jiaoqingbo] add log
d5af758f [jiaoqingbo] modify ut
0f3af6c6 [jiaoqingbo] modify ut case
9ffb4177 [jiaoqingbo] Merge branch 'master' into 2084
52dd5090 [jiaoqingbo] modify ut
3f881748 [jiaoqingbo] modify test conf
705f94af [jiaoqingbo] modify test conf
096fba1d [jiaoqingbo] modify test conf
83f1aba9 [jiaoqingbo] add more example configuration
d2581534 [jiaoqingbo] fix compile problem
65c3198a [jiaoqingbo] [KYUUBI #2084] Support arbitrary parameters for KyuubiConf
Lead-authored-by: jiaoqingbo <1178404354@qq.com>
Co-authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
Add `Deploy Kyuubi Flink engine on Yarn`.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2131 from SteNicholas/KYUUBI-1866.
Closes#1866
ba639f15 [SteNicholas] [KYUUBI #1866][FOLLOWUP] Add Deploy Kyuubi Flink engine on Yarn
cc6f4d44 [SteNicholas] [KYUUBI #1866][FOLLOWUP] Add Deploy Kyuubi Flink engine on Yarn
Authored-by: SteNicholas <programgeek@163.com>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
This PR reverts "[KYUUBI #1948] Upgrade thrift version to 0.16.0" in branch-1.5 and some subsequent commits due to issues caused by breaking changes in Thrift, e.g. #2101, #2093
We need more time to evaluate it and add more test cases to ensure stability before the next release.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2103 from pan3793/revert.
Closes#2103Closes#1948
575dfe8d [Cheng Pan] properties order
4c94481e [Cheng Pan] fix deps
bfbdeb89 [Cheng Pan] Add log4j deps for kyuubi-hive-jdbc test
bad4e90e [Cheng Pan] surefire
f2ae8b9b [Cheng Pan] Revert "[KYUUBI #1948] Upgrade thrift version to 0.16.0"
4501f743 [Cheng Pan] Revert "[KYUUBI #1948][FOLLOWUP] Remove TFramedTransport in extension-spark-3-1 and extension-spark-3-2"
cc04df49 [Cheng Pan] Revert "[KYUUBI #1996] Clean up log4j2 conf and enhance surefire plugin conf"
52a20e17 [Cheng Pan] Revert "[KYUUBI #1948][FOLLOWUP] Relocate fb303 classes"
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Add logging of Flink SQL Engine in `logging.md`.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2111 from SteNicholas/KYUUBI-1866.
Closes#1866
7a0bdbb8 [SteNicholas] [KYUUBI #1866] Add logging of Flink SQL Engine
Authored-by: SteNicholas <programgeek@163.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Add quick start documents of the Flink SQL Engine.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2106 from deadwind4/KYUUBI-1866-quickstart.
Closes#1866
2533aafd [Ada Wong] remove Yarn section
6aa4db8a [Ada Wong] compress png
ff6bff72 [Ada Wong] [KYUUBI #1866][DOCS] Add flink sql engine quick start
Authored-by: Ada Wong <rsl4@foxmail.com>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
Add description about trino in the config of engine.type.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2109 from zhaomin1423/fix_engine_type_doc.
Closes#2108
adf898c0 [Min Zhao] [Kyuubi #2108] Add description about trino in the config of engine.type
8f11514e [Min Zhao] [Kyuubi #2108] Add description about trino in the config of engine.type
Authored-by: Min Zhao <zhaomin1423@163.com>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
Add debugging instructions for Flink engine.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2098 from SteNicholas/KYUUBI-2089.
Closes#2089
08189b85 [SteNicholas] [KYUUBI #2089] Add debugging instructions for flink engine
Authored-by: SteNicholas <programgeek@163.com>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
YARN_CONF_DIR shall be added to kyuubi server classpath as HADOOP_CONF_DIR.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2082 from SteNicholas/KYUUBI-2081.
Closes#2081
c6f76c5c [SteNicholas] [KYUUBI #2081] YARN_CONF_DIR shall be added to kyuubi server classpath as HADOOP_CONF_DIR
fee5c21f [SteNicholas] [KYUUBI #2081] YARN_CONF_DIR shall be added to kyuubi server classpath as HADOOP_CONF_DIR
Authored-by: SteNicholas <programgeek@163.com>
Signed-off-by: Kent Yao <yao@apache.org>
… engine
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2080 from yanghua/KYUUBI-2079.
Closes#2079
2bf98720 [yanghua] [KYUUBI #2079] Update kyuubi layer source file to add flink and trino engine
Authored-by: yanghua <yanghua1127@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
cretate ns kyuubi ---> create ns kyuubi
### _Why are the changes needed?_
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1999 from HYBG-1126/patch-1.
Closes#1999
f68f73e9 [HYBG] Update quick_start_with_helm.md
Lead-authored-by: HYBG-1126 <hybg1126@outlook.com>
Co-authored-by: HYBG <44338121+HYBG-1126@users.noreply.github.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
Add CONTRIBUTING.md to list COMMUNITY resources that I contributor can use
delete contributions and improvement-proposal in main repo
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2004 from yaooqinn/CONTRIBUTING.
Closes#2004
ddbcd521 [Kent Yao] CONTRIBUTING
f81c14b3 [Kent Yao] Merge branch 'master' of github.com:apache/incubator-kyuubi into CONTRIBUTING
3bdbdc9b [Kent Yao] CONTRIBUTING
574ceff2 [Kent Yao] CONTRIBUTING
f0a6b4c4 [Kent Yao] CONTRIBUTING
c55c6a02 [Kent Yao] CONTRIBUTING
Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
[THRIFT-4805](https://issues.apache.org/jira/browse/THRIFT-4805) has been fixed in thrift 0.13.0, we can remove the workaround of log4j2 configurations because we use thrift 0.16.0 now, the change also removes the following warnings.
```
2022-03-01 23:18:16,530 main ERROR Filters contains invalid attributes "onMatch", "onMismatch"
2022-03-01 23:18:16,543 main ERROR Filters contains invalid attributes "onMatch", "onMismatch"
```
`maven-surefire-plugin` is only used in `kyuubi-hive-jdbc`, we should disable it in other modules.
```
[INFO] --- maven-surefire-plugin:2.22.0:test (default-test) kyuubi-ctl_2.12 ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
2022-03-01 23:18:16,530 main ERROR Filters contains invalid attributes "onMatch", "onMismatch"
2022-03-01 23:18:16,543 main ERROR Filters contains invalid attributes "onMatch", "onMismatch"
[INFO] Running org.apache.kyuubi.ctl.ServiceControlCliSuite
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.031 s - in org.apache.kyuubi.ctl.ServiceControlCliSuite
[INFO] Running org.apache.kyuubi.ctl.ServiceControlCliArgumentsSuite
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s - in org.apache.kyuubi.ctl.ServiceControlCliArgumentsSuite
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1996 from pan3793/log.
Closes#1996
291441c0 [Cheng Pan] nit
f2c94246 [Cheng Pan] Tune surefire plugin in conf
e1ee6402 [Cheng Pan] Clean up log4j2 conf
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Make plan only mode skippable for configuable plans.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1967 from SteNicholas/KYUUBI-1963.
Closes#1963
e253aec6 [SteNicholas] [KYUUBI #1963] Make plan only mode skippable for configuable plans
b5cff405 [SteNicholas] [KYUUBI #1963] Make plan only mode skippable for configuable plans
Authored-by: SteNicholas <programgeek@163.com>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
Replace `mirror-cn` profile with Apache officially suggested dlcdn.
mirror-cn is only suite for CN, while dlcdn is much wiser.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1970 from yaooqinn/cdn.
Closes#1970
a325f828 [Kent Yao] Repalce mirror-cn profile with Apache officially suggested dlcdn
a0756b4f [Kent Yao] Repalce mirror-cn profile with Apache officially suggested dlcdn
Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
Upgrade libthrift to 0.16.0 due to [CVE-2020-13949](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-13949) and the coming upstream change of Spark https://github.com/apache/spark/pull/34362
### _What changed in this PR?_
- Upgrade libthrift to 0.16.0
- Shade and relocate `thrift` and `hive-service-rpc` classes in `kyuubi-spark-engine`, it's necessary to avoide conflicting with old thrift libs bundled in Spark binary releases.
- Due to thrift change the method signature, the subclasses those interfaces in Kyuubi also need to modify to pass compile.
We rely on Hive 2.3.9 jars in some components, e.g. `kyuubi-hive-jdbc`, `LocalMetaServer` in `kyuubi-server` test classes.
Some classes in Hive jars compiled against old thrift interfaces which are not compatible with thrift 0.16.0, it causes runtime link error, we found the following classes which breaks the test and copied them with necessary modification to make it work with thrift 0.16.0.
- `TFramedTransport`
- `TFilterTransport`
- `TUGIAssumingTransport`
- `TUGIContainingTransport`
- Next Steps, I think it's worth to do in separated PRs.
- Recover the `HiveDelegationTokenProviderSuite`, one approach is use an isolate classloader to load HMS classes and thrift 0.9.3 classes from Maven, this approach can also be used for the planed Zoopkeeper upgrading to help us verficating the compatibility of Zookeeper Server 3.4.x.
- Rewrite `kyuubi-hive-jdbc` to make it decouple with Hive jars, because there maybe other places which may not work with thrift 0.16.0 but the UTs does not cover.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1953 from SteNicholas/KYUUBI-1948.
Closes#1948
de5d1ea2 [SteNicholas] [KYUUBI #1948] Upgrade thrift version to 0.16.0
898effcd [SteNicholas] [KYUUBI #1948] Upgrade thrift version to 0.16.0
803e270c [SteNicholas] [KYUUBI #1948] Upgrade thrift version to 0.16.0
Authored-by: SteNicholas <programgeek@163.com>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
Currently, Flink engine would pull all result rows into memory before returning it to the client. This would be problematic for large result sets and infinite result sets.
This is a sub-task of KPIP-2 https://github.com/apache/incubator-kyuubi/issues/1322.
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1938 from link3280/feature/FLINK-1883.
Closes#1883
80020cee [Paul Lin] Update externals/kyuubi-flink-sql-engine/src/test/scala/org/apache/kyuubi/engine/flink/operation/FlinkOperationSuite.scala
1b958221 [Paul Lin] [KYUUBI #1883] Avoid allocating too much buffer space
5be7535c [Paul Lin] [KYUUBI #1883] Support max result rows for Flink queries
Authored-by: Paul Lin <paullin3280@gmail.com>
Signed-off-by: Kent Yao <yao@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
Destroy the process after the engine starts. #1796
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [X] Add screenshots for manual tests if appropriate
add conf: `kyuubi.session.engine.startup.waitCompletion false`
- [X] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1815 from wForget/KYUUBI-1796.
Closes#1796
12aea19a [Wang Zhen] fix
27cb602a [Wang Zhen] fix
e70e7071 [Wang Zhen] add log
ecaf828c [Wang Zhen] regenerate settings.md
a5f6c5de [Wang Zhen] fix desc
09812beb [Wang Zhen] Merge remote-tracking branch 'origin/master' into KYUUBI-1796
27fab5eb [Wang Zhen] [KYUUBI-1796] Destroy the process after the engine starts
Authored-by: Wang Zhen <wangzhen07@qiyi.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
Add more enum values for `OperationModes`.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1933 from SteNicholas/KYUUBI-1919.
Closes#1919
adcc7f5e [SteNicholas] [KYUUBI #1919] Add more enum values for OperationModes
Authored-by: SteNicholas <programgeek@163.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
Update metric doc when metric change.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1916 from zhenjiaguo/update_metric_doc.
Closes#1914
e8ea7f6c [zhenjiaguo] update metric doc
a5d730ce [zhenjiaguo] update metric doc
Authored-by: zhenjiaguo <zhenjiaguo@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
Set Spark 3.2.1 as default.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1913 from SteNicholas/KYUUBI-1876.
Closes#1876
1a6e4c6c [SteNicholas] [KYUUBI #1876] Set Spark 3.2.1 as default
Authored-by: SteNicholas <programgeek@163.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
closes https://github.com/apache/incubator-kyuubi/issues/1536
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1892 from ulysses-you/KYUUBI-1536.
Closes#1536
f0c450a9 [ulysses-you] codecov
35fac4f6 [ulysses-you] fix
f34cfbf4 [ulysses-you] comment
78a54bc1 [ulysses-you] address comment
5cb574ba [ulysses-you] fix test
1033909b [ulysses-you] fix
3c925e56 [ulysses-you] address comment
a74988cd [ulysses-you] style
649f02d0 [ulysses-you] Add session conf advisor
debd3adb [ulysses-you] Add session conf advisor
Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
… generate mail content automatically
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1900 from yanghua/KYUUBI-1898.
Closes#1898
782ff20b [yanghua] [KYUUBI-1898] Enhance release guide add more information about how to generate mail content automatically
Authored-by: yanghua <yanghua1127@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
https://kyuubi.apache.org/docs/latest/deployment/settings.html
The configuration parameters in the document are too long and a newline is inserted(`<br/>`), which makes it difficult to search.
And if the user copies the configuration parameters, it may become several lines.
Use `<code>` to replace `<br/>`.
https://github.com/apache/incubator-kyuubi/issues/1889
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1890 from cxzl25/KYUUBI-1889.
Closes#1889
765b0622 [sychen] format metrics.md
abeb0601 [sychen] add newline
28865ad7 [sychen] increase max width
2f9c7e05 [sychen] use <code>
Authored-by: sychen <sychen@trip.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1864 from yanghua/KYUUBI-1844.
Closes#1844
47d09c39 [yanghua] Address review comments
b5a8bbc3 [yanghua] [KYUUBI #1844] Add step for generating release notes
Authored-by: yanghua <yanghua1127@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
Revert the dependency changes for kyuubi-hive-jdbc and kyuubi-hive-jdbc-shaded modules from #1842
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1854 from turboFei/slf4j-api.
Closes#1769
dffe6c5c [Fei Wang] reserve for kyuubi-hive-beeline
20012434 [Fei Wang] [KYUUBI #1769][FOLLOWUP] Revert the dependency change for kyuubi-hive-beeline and kyuubi-hive-jdbc(shaded) modules
Authored-by: Fei Wang <fwang12@ebay.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
Enable the internal secure access between Kyuubi server and engine.
This pr use secret and cipher to encrypt a dynamic token identifier for authentication.
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1813 from turboFei/engine_secure.
Closes#1813
aa7ad0de [Fei Wang] run a simple query for engien security
34354d97 [Fei Wang] requre zk node must be defined for zk secret provider
f84ce7c9 [Fei Wang] Require zk secret node non empty
b545281d [Fei Wang] refactor
d8d6dfd2 [Fei Wang] address comments
0e9273e1 [Fei Wang] refactor provider conf
d3263c08 [Fei Wang] place holder
e1266e32 [Fei Wang] reformat
9e252fea [Fei Wang] enhance prompt
dbc9100c [Fei Wang] fix conf
ff6b6dea [Fei Wang] refactor
4ea020ed [Fei Wang] refactor
20915902 [Fei Wang] refactor
a7b4e41e [Fei Wang] add more ut:
1e1ceb27 [Fei Wang] use orignal cipher
8d8387b2 [Fei Wang] save
ad777cb4 [Fei Wang] refactor
b8997313 [Fei Wang] save
b997095d [Fei Wang] refactor
e05b47c5 [Fei Wang] move
a322b784 [Fei Wang] refactor
b14f7cbf [Fei Wang] Enable the internal secure authentication
Authored-by: Fei Wang <fwang12@ebay.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
This patch proposes to migrate from log4j1 to log4j2 in Kyuubi.
Refer the spark patch: https://github.com/apache/spark/pull/34895
### Does this PR introduce any user-facing change?
Yes. Users may need to rewrite their log4j properties file for log4j2. As of version 2.4, log4j now supports configuration via properties files. Note that the property syntax is not the same as the syntax used in log4j 1, but during the migration I found the syntax is pretty close so the migration should be straightforward.
### _How was this patch tested?_
Passed all existing tests.
Closes#1769 from turboFei/log4j2.
Closes#1769
8613779c [Fei Wang] remove log4j dependencies from spark-sql-engine
b2fe6dba [Fei Wang] Use String to present default log level
762e2d03 [Fei Wang] only remove org.apache.logging.log4j:log4j-slf4j-impl
8a912086 [Fei Wang] remove dependencies from spark-sql-engine
7e3a4980 [Fei Wang] address commments
051f49f5 [Fei Wang] address comments
85316a0b [Fei Wang] Keep compatible with log4j12
01d1a84e [Fei Wang] for log4j1
b9e17e1b [Fei Wang] refactor
e24391ed [Fei Wang] revert log count
38803002 [Fei Wang] add log4j2.properties.template
4f0b22fc [Fei Wang] save
7ce84119 [Fei Wang] modify log level
1ea5ca53 [Fei Wang] add log4j to engine
c4a86d4d [Fei Wang] use AbstractFilter
27b08b6a [Fei Wang] remove more
8cc15ae7 [Fei Wang] reformat
c13ec29e [Fei Wang] save temporarily
33a38e2e [Fei Wang] exclude log4j12 from spark-sql
9129a64a [Fei Wang] refactor
5362b43d [Fei Wang] make it run at first
7f27f519 [Fei Wang] more
56f4f1ff [Fei Wang] fix logging
a74b6d37 [Fei Wang] start appender
dea964aa [Fei Wang] fix build erorr at first
e20b7500 [Fei Wang] address comments
2ec02b4d [Fei Wang] fix LogDivertAppender
dded1290 [Fei Wang] more
c63e0008 [Fei Wang] add log4j2.properties
Authored-by: Fei Wang <fwang12@ebay.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1837 from yanghua/KYUUBI-1836.
Closes#1836
913221fc [yanghua] [KYUUBI #1836] [RELEASE] Enhance release guide
Authored-by: yanghua <yanghua1127@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
Fix the jdbc connection url format doc.
In fact, the kyuubiConfs part and kyuubiVars part should have the same effect.
But it is better to split the engine initialization conf and runtime vars to two parts, so here I do not modify these two parts.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1831 from turboFei/jdbc_docs.
Closes#1831
40288282 [Fei Wang] Update the docs for jdbc url
Authored-by: Fei Wang <fwang12@ebay.com>
Signed-off-by: Kent Yao <yao@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
This PR mainly focus on refactoring
1. Use Apache Thrift Naming style for thrift-based classes which is `TXyzAbc`, e.g. `TFrontendService`
2. Created `TFrontendService` and move the `TCLIService.Iface` APIs and other common fields in `TFrontendService`, later we will use it to implement the `KyuubiTHttpFrontendService` in the server module
3. introduce a new conf also for server FEs to be able to pick IP first
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1761 from yaooqinn/http1.
Closes#1761
98b4afdd [Kent Yao] address comment
3138af29 [Kent Yao] [WIP] Thrift Http protocol support 1
a06026af [Kent Yao] [WIP] Thrift Http protocol support 1
efa899fa [Kent Yao] [WIP] Thrift Http protocol support 1
d1403e30 [Kent Yao] [WIP] Thrift Http protocol support 1
d114efd4 [Kent Yao] [WIP] Thrift Http protocol support 1
86e30c47 [Kent Yao] [WIP] Thrift Http protocol support 1
8117d12d [Kent Yao] [WIP] Thrift Http protocol support 1
62194323 [Kent Yao] [WIP] Thrift Http protocol support 1
Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
Add TRINO engine type and TrinoProcessBuilder
### _How was this patch tested?_
- [X] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1695 from hddong/add-trino-process-builder.
Closes#1510
394b853e [hongdongdong] fix
044ca8cc [hongdongdong] fix
568b1b8e [hongdongdong] use java options to pass args
fbe7e60d [hongdongdong] [KYUUBI #1510] Add TRINO engine type and TrinoProcessBuilder
Authored-by: hongdongdong <hongdongdong@cmss.chinamobile.com>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
This PR covers #1619.
Overall, this PR contains the following changs,
1. change `build/dist` script to support flink sql engine
2. enhance `externals/flink-sql-engine/pom.xml` to support create a shaded jar
3. simplify `externals/kyuubi-flink-sql-engine/bin/flink-sql-engine.sh`
4. introduce `FlinkSQLEngine`(flink sql engine entrypoint) and `FlinkProcessBuilder`(kyuubi server launcher)
5. add ut in kyuubi server side
After this PR, we can run the basic query e.g. `select now()` from beeline and get result, and Kyuubi Server can auto launch flink engine if there is no proper one. The Flink engine also supports other engine share levels defined in Kyuubi.
The implementation based on Flink 1.14 codebase.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1629 from pan3793/flink-backend.
Closes#1629
b7e5f0e3 [Cheng Pan] revert tgz name change
a4496c37 [Cheng Pan] Fix reflection
3b4e86a1 [Cheng Pan] deps
68efa42c [Cheng Pan] log
8a9e37f3 [Cheng Pan] nit
10fb2bc3 [Cheng Pan] CI
c1560fdb [Cheng Pan] nit
303e2f1e [Cheng Pan] Restore log conf
d84720b1 [Cheng Pan] SessionContext
b258d81a [Cheng Pan] cleanup
16edd528 [Cheng Pan] Cleanup
9ae54557 [Cheng Pan] Fix CI
25b6b57d [Cheng Pan] hadoop-client-api
c12b5ca4 [Cheng Pan] Server UT pass
502d3f08 [Cheng Pan] pass
dac4323b [yanghua] Laungh local flink engine container successfully
Lead-authored-by: Cheng Pan <chengpan@apache.org>
Co-authored-by: yanghua <yanghua1127@gmail.com>
Signed-off-by: Kent Yao <yao@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
Add operation and backend service metrics doc.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1607 from zhenjiaguo/add_be_metric_doc.
Closes#1607
e10bf146 [zhenjiaguo] Comment
0062f3d8 [zhenjiaguo] Metric format adjustment
af84534c [zhenjiaguo] Add metrics version compatibility description
c6f9eb26 [zhenjiaguo] Add operation and backend service method metric info doc
Authored-by: zhenjiaguo <zhenjia_guo@163.com>
Signed-off-by: Kent Yao <yao@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
Like the `hive.exec.drop.ignorenonexistent` configuration in hive, do not report an error if DROP DATABASE/TABLE/VIEW/Function/Partition specifies a non-existent database/table/view/function/partition. For details: #1577.
### _How was this patch tested?_
- [X] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1583 from wForget/KYUUBI-1577.
Closes#1577
63382660 [Wang Zhen] delete useless comment
adadf424 [Wang Zhen] add DropIgnoreNonexistent rule using injectPostHocResolutionRule
860b15d3 [Wang Zhen] add DropIgnoreNonexistent rule for spark 3.2
aad43cdb [Wang Zhen] adjust code
ca1b9f11 [Wang Zhen] make some adjustments and update rules.md
804bf40e [Wang Zhen] [KYUUBI-1577] Add DropIgnoreNonexistent Rule.
Authored-by: Wang Zhen <wangzhen07@qiyi.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
Many kyuubi users are not familiar with kerberos authentication mechanism and encountered troubles connecting to kerberized Kyuubi.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
Configurations are verified in #1432
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1552 from zhouyifan279/1550.
Closes#1550
58e324a8 [zhouyifan279] [KYUUBI #1550] Add kerberos authentication flow
3b030a91 [zhouyifan279] [KYUUBI #1550] Add kerberos authentication flow
4449b689 [zhouyifan279] [KYUUBI #1550] Improve doc style
3aa8e20c [zhouyifan279] [KYUUBI #1550] Add Windows configuration steps
7e67cdc5 [zhouyifan279] [KYUUBI #1550] Capitalize title and emphasize the relation between KinitAuxiliaryService and Kerberos authentication
424e56b3 [zhouyifan279] [KYUUBI #1550] Rename client/kerberos.md to client/kerberized_kyuubi.md
d8977cea [zhouyifan279] [KYUUBI #1550] Provide a specific user guide about connecting to kerberized kyuubi
Authored-by: zhouyifan279 <zhouyifan279@gmail.com>
Signed-off-by: Kent Yao <yao@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
Add kyuubi.engine.pool.name configuration. #1554
### _How was this patch tested?_
- [X] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [X] Add screenshots for manual tests if appropriate
- [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1572 from wForget/KYUUBI-1554.
Closes#1554
40850f5a [Wang Zhen] [KYUUBI-1554] Add kyuubi.engine.pool.name configuration
Authored-by: Wang Zhen <wangzhen07@qiyi.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
