### _Why are the changes needed?_
After previous efforts, we are close to the goal. This is the latest step to to allow Kyuubi run on JDK 17 by adding `--add-open` in bootstrap scripts and test the plugin's parameter.
This PR also enables JDK 17 test in CI, but only for Kyuubi server and Spark(3.3+) engine moudles, because
- Spark supports JDK 17 since 3.3, see SPARK-33772
- Flink does not support JDK 17 until the latest 1.17
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
- Run Kyuubi server on JDK 17
- Run simple Spark (3.3.2) query on JDK 17
- Run `kyuubi-zk-cli` on JDK 17
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4849 from pan3793/jdk-17.
Closes#4849
165e3558c [Cheng Pan] Enable JDK 17 in CI
f5778e7bd [Cheng Pan] Open modules in bootstrap scripts
76b507ffd [Cheng Pan] Open modules in UT
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
This PR aims to migrate the vanilla Zookeeper and Curator to the Kyuubi Shaded Zookeeper. It's the first step to adapting JDK 17.
There is a known issue [ZOOKEEPER-3779](https://issues.apache.org/jira/browse/ZOOKEEPER-3779) that Zookeeper 3.4 client can not run on JDK 14 and above, in https://github.com/apache/kyuubi-shaded/pull/5, we fixed this issue by a surgical.
With the above fixing, zk-3.4 and zk-3.6 clients both work well on JDK 17, we just randomly pick some cases to make sure zk-3.6 is tested
zk-3.4 client supports zk-3.4+ server, but zk-3.6 client only supports zk-3.5+ server; in the meanwhile, zk-3.4 is adopted widely, (CDH 5/6, HDP, EMR created before 2023).
We are sticky to zk-3.4 to ensure that Kyuubi can be out-of-box in the most existing Hadoop cluster but also provide zk-3.6 as an alternative(simply replace the kyuubi-shaded-zk-3.4 jar w/ kyuubi-shaded-zk-3.6, or build w/ -Pzookeeper-3.6) for users who concerns that zk-3.4 is EOL.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4852 from pan3793/shaded-zk.
Closes#4852
d960cc945 [Cheng Pan] remove staging repo
1b3622080 [Cheng Pan] Switch to Kyuubi Shaded Zookeeper
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
The difference between w/ and w/o `exec` in shell script
https://www.baeldung.com/linux/exec-command-in-shell-script
> When using exec, however, the command following exec replaces the current shell. This means no subshell is created and the current process is replaced with this new command.
We must do this change to make `bin/kyuubi` suitable for Cloudera Manager.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4494 from wangmiao1002/master.
Closes#4494
ed9320447 [miaowang] Modify script run_kyuubi mode
Authored-by: miaowang <miaowang@gaojihealth.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
<!--
fix-#4057 info: modify the shellcheck errors file in ./bin
1. "$" is a array, we want use string to compare. so modify "$" => "$*"
2. `tty` mean execute the command, we can use $(tty) replace it
3. param $# is a number, compare number should use -gt/-lt,not >/<
4. not sure the /bin/kyuubi line 63 'exit -1' need modify? so the directory bin only have a shellcheck note in /bin/kyuubi
-->
- fix shellcheck violations in scripts of /bin folder
- enable shellcheck rule checks
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4162 from davidyuan1223/master.
Closes#4078
c48ad38c7 [yuanfuyuan] remove the used blank lines
55a0a43c5 [xiaoyuandajian] Merge pull request #10 from xiaoyuandajian/fix-#4057
cb1193576 [yuan] Merge remote-tracking branch 'origin/fix-#4057' into fix-#4057
86e4e1ce0 [yuan] fix-#4057 info: modify the shellcheck errors file in ./bin 1. "$@" is a array, we want use string to compare. so update "$@" => "$*" 2. `tty` mean execute the command, we can use $(tty) replace it 3. param $# is a number, compare number should use -gt/-lt,not >/< 4. not sure the /bin/kyuubi line 63 'exit -1' need modify? so the directory bin only have a shellcheck note in /bin/kyuubi
dd39efdeb [袁福元] fix-#4057 info: 1. "$@" is a array, we want use string to compare. so update "$@" => "$*" 2. `tty` mean execute the command, we can use $(tty) replace it 3. param $# is a number, compare number should use -gt/-lt,not >/<
Lead-authored-by: yuan <yuanfuyuan@mafengwo.com>
Co-authored-by: 袁福元 <yuanfuyuan@mafengwo.com>
Co-authored-by: xiaoyuandajian <51512358+xiaoyuandajian@users.noreply.github.com>
Co-authored-by: yuanfuyuan <1406957364@qq.com>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
```
Usage:
+----------------------------------------------------------------------------------------------+
| ./build/dist [--name <custom_name>] [--tgz] [--web-ui] [--flink-provided] [--hive-provided] |
| [--spark-provided] [--mvn <maven_executable>] <maven build options> |
+----------------------------------------------------------------------------------------------+
name: - custom binary name, using project version if undefined
tgz: - whether to make a whole bundled package
web-ui: - whether to include web ui
flink-provided: - whether to make a package without Flink binary
hive-provided: - whether to make a package without Hive binary
spark-provided: - whether to make a package without Spark binary
mvn: - external maven executable location
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
Create binary artifacts using `build/dist --tgz --web-ui` and run, then open `http://0.0.0.0:10099/ui`
<img width="1401" alt="image" src="https://user-images.githubusercontent.com/26535726/220753103-ce801f12-f394-4ece-92a4-1902e93c62c7.png">
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4397 from pan3793/webui-build.
Closes#4397
97901d63e [Cheng Pan] doc
37d5e2ad3 [Cheng Pan] mirror-cdn
c5751dd5b [Cheng Pan] remove unused dep
d308defb7 [Cheng Pan] nit
9abca4705 [Cheng Pan] nit
c1d184afd [Cheng Pan] nit
7091d5bf5 [Cheng Pan] regex
f0ac16b3c [Cheng Pan] [BUILD] `build/dist` support --web-ui
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
See more in #4184
This subtask remove build kyuubi docker image from dev mode(source code).
In follow subtask, will add feature build kyuubi docker image from source code.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4217 from zwangsheng/tool/docker-image-tool.
Closes#4184
34c866025 [zwangsheng] Add +x premission for docker-image-tool
97b404db7 [zwangsheng] Copy download spark binary
6f267e221 [zwangsheng] Refactor docker-image-tool.sh
Authored-by: zwangsheng <2213335496@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Fix#2062
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4078 from xiaoyuandajian/fix-#2062.
Closes#4078
a5a01945 [Cheng Pan] Update .github/workflows/style.yml
3c08e692 [袁福元] fix the shellcheck found error file
a33cbb22 [袁福元] fix the shellcheck found error file
da195782 [袁福元] fix the shellcheck found error file
b24e6adf [袁福元] fix the shellcheck found error file
90ef0712 [袁福元] fix the shellcheck found error file
42acb31f [袁福元] fixup! test1
31c1783d [袁福元] fixup! test
c40b5f38 [袁福元] Revert "test"
d97801fa [袁福元] Merge remote-tracking branch 'origin/fix-#2062' into fix-#2062
39a9ff31 [袁福元] test
fc7198da [xiaoyuandajian] Update style.yml
62e80726 [袁福元] test
0abdef36 [xiaoyuandajian] Delete main.yml
1e252e2a [xiaoyuandajian] Create main.yml
c19728ec [xiaoyuandajian] Delete main.yml
5db61e5a [xiaoyuandajian] Delete main1.yml
9e27a363 [xiaoyuandajian] Create main1.yml
69540652 [xiaoyuandajian] Create main.yml
Lead-authored-by: 袁福元 <yuanfuyuan@mafengwo.com>
Co-authored-by: xiaoyuandajian <51512358+xiaoyuandajian@users.noreply.github.com>
Co-authored-by: Cheng Pan <pan3793@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Close#3679
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3680 from lightning-L/admin-ctl.
Closes#3679
9c748f073 [Tianlin Liao] [KYUUBI #3679] Admin command line supports delete/list engine operation
Authored-by: Tianlin Liao <tiliao@ebay.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
### _Why are the changes needed?_
Fix#3598
Hive Beeline has the same issue: https://issues.apache.org/jira/browse/HIVE-6758 and was fixed by append jline option "-Djline.termial=jline.UnsupportedTerminal".
We do the same in this PR.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
<img width="1043" alt="image" src="https://user-images.githubusercontent.com/88070094/194760564-04258a46-646f-4f76-ab19-b3e305b0ef69.png">
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3599 from zhouyifan279/3598.
Closes#3598
3c11d27c [zhouyifan279] improve indent
6d39b3cc [zhouyifan279] Beeline doesn't work with -e option when started in background
Authored-by: zhouyifan279 <zhouyifan279@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
1、Check and wait until process disappeared before remove pid file.
2、If pid file was still not be removed，we should remind user to kill and remove manually .
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3546 from asagjj/pr-3544.
Closes#3544
e72e42fd [asagjj] Update bin/kyuubi
3c789be8 [asagjj] fix bin/kyuubi stop
Lead-authored-by: asagjj <776172439@qq.com>
Co-authored-by: asagjj <shenyinjie@cmss.chinamobile.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
1. The version options should be in zkOptions.
2. Add docs for kyuubi-adminctl.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3066 from turboFei/admin_ctl_docs.
Closes#3045
983d133f [Fei Wang] comments
65675ad7 [Fei Wang] add more
b5b772d5 [Fei Wang] comments
222b409c [Fei Wang] add docs
20c50c00 [Fei Wang] kyuubi-adminctl -> kyuubi-admin
078044e0 [Fei Wang] refactor
4e89b52d [Fei Wang] remove unused
c2d1dc70 [Fei Wang] add admin ctl docs
Authored-by: Fei Wang <fwang12@ebay.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
### _Why are the changes needed?_
Close#3045
Introduce kyuubi-adminctl for admin control.
```
Usage: kyuubi-adminctl [refresh] [options]
-v, --version <value> Using the compiled KYUUBI_VERSION default, change it if the active service is running in another.
-b, --verbose Print additional debug output.
--hostUrl <value> Host url for rest api.
--authSchema <value> Auth schema for rest api, valid values are basic, spnego.
--username <value> Username for basic authentication.
--password <value> Password for basic authentication.
--spnegoHost <value> Spnego host for spnego authentication.
--conf <value> Kyuubi config property pair, formatted key=value.
Command: refresh [config] <args>...
Refresh the resource.
Command: refresh config [<configType>]
Refresh the config with specified type.
<configType> The valid config type can be one of the following: hadoopConf.
-h, --help Show help message and exit.
```
Now support `kyuubi-adminctl refresh config hadoopConf` to reload hadoop conf.
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3057 from turboFei/kyuubi_admin_ctl.
Closes#3045
b6b46f3a [Fei Wang] refactor the info render
d3b600f2 [Fei Wang] add toString
234ae84c [Fei Wang] fix ut
7db818d0 [Fei Wang] save
e9e50a0f [Fei Wang] add ut
849edad8 [Fei Wang] add ut
c4d9b784 [Fei Wang] fix ut
1c8d7c9b [Fei Wang] add kyuubi-adminctl
8b1921ae [Fei Wang] save
155f57af [Fei Wang] saev
2ca8b665 [Fei Wang] save
5ad40cea [Fei Wang] save
bb2e1178 [Fei Wang] add refresh config
Authored-by: Fei Wang <fwang12@ebay.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
Signed-off-by: zhouyifan279 <zhouyifan279gmail.com>
### _Why are the changes needed?_
Fix#3033
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3034 from zhouyifan279/3033.
Closes#3033
646e0dbc [zhouyifan279] [KYUUBI #3033][Bug] Kyuubi failed to start due to PID directory not exists
Authored-by: zhouyifan279 <zhouyifan279@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
We can make the logo printed by Kyuubi on the terminal colorful and vibrant.
### Why are the changes needed?
- When TTY is available, print the colorfully logo, else print default
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2744 from wolfboys/master.
Closes#2743
2205e79c [benjobs] [KYUUBI #2743] colorfully kyuubi logo support
Authored-by: benjobs <benjobs@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
In scala code mode implementation, we need to call `repl.addUrlsToClassPath` to set the classpath of the current thread to repl before entering `repl`
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2475 from iodone/dev-1.
Closes#2471
21b92cc4 [odone] [KYUUBI #2471] add dynamic jar generator
3542786e [odone] [KYUUBI #2471] Fixed
3db2a7df [odone] [KYUUBI #2471] Fixed
65303683 [odone] [KYUUBI #2471] Fixed
Authored-by: odone <odone.zhang@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
`docker-image-tools.sh` should use the version where the script was added as the trial version.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2410 from zwangsheng/bug/fix_docker_image_build_usage.
Closes#2410
f32ef8e2 [zwangsheng] fix version
Authored-by: zwangsheng <2213335496@qq.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
…consistency
### _Why are the changes needed?_
fix#2244
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2245 from jiaoqingbo/2244.
Closes#2244
f9d68c7f [jiaoqingbo] [KYUUBI #2244] load-kyuubi-env.sh should print SPARK_ENGINE_HOME for consistency
Authored-by: jiaoqingbo <1178404354@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Add ProcBuilder for HiveEngine.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2146 from zhaomin1423/hive_process_builder.
Closes#2024
1baf1397 [Min Zhao] [KYUUBI #2024] Hive Backend Engine - ProcBuilder for HiveEngine
23fdc3b7 [Min Zhao] [KYUUBI #2024] Hive Backend Engine - ProcBuilder for HiveEngine
Authored-by: Min Zhao <zhaomin1423@163.com>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
YARN_CONF_DIR shall be added to kyuubi server classpath as HADOOP_CONF_DIR.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2082 from SteNicholas/KYUUBI-2081.
Closes#2081
c6f76c5c [SteNicholas] [KYUUBI #2081] YARN_CONF_DIR shall be added to kyuubi server classpath as HADOOP_CONF_DIR
fee5c21f [SteNicholas] [KYUUBI #2081] YARN_CONF_DIR shall be added to kyuubi server classpath as HADOOP_CONF_DIR
Authored-by: SteNicholas <programgeek@163.com>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
Users who use Flink or Trino need not set SPARK_HOME, so I remove SPARK_HOME predication in bin/kyuubi script.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1994 from deadwind4/KYUUBI-1992.
Closes#1992
5f11df57 [Ada Wong] [KYUUBI #1992] Remove SPARK_HOME predication.
Authored-by: Ada Wong <rsl4@foxmail.com>
Signed-off-by: Kent Yao <yao@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
The original idea of SPARK_HADOOP_VERSION is used to concat spark release names only, now we need to remove it as
- SPARK_HADOOP_VERSION is misunderstood by developers and misused somewhere like the one of kyuubi compiled
- multi-engine support now
- the release names of spark(or something else) are very easy to get through code with different environments, prod/test/dev
- A `mvn` job is bundled with `bin/load-kyuubi-env.sh` which is truly worrisome
- SPARK_HADOOP_VERSION on spark side hass broken already for spark 3.2 which actually bundled with hadoop 3.3, see https://github.com/apache/spark-website/pull/361#discussion_r730716668
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1950 from yaooqinn/hadoop.
Closes#1950
b47be7c6 [Kent Yao] Remove ambiguous SPARK_HADOOP_VERSION
3b33ee56 [Kent Yao] Remove ambiguous SPARK_HADOOP_VERSION
Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [X] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1932 from hddong/add-trino-engine-home.
Closes#1930
26ed8ce2 [root] [KYUUBI #1930] Export TRINO_ENGINE_HOME in load-kyuubi-env script
Lead-authored-by: hongdongdong <hongdongdong@cmss.chinamobile.com>
Co-authored-by: root <root@DESKTOP-30EIBMO.localdomain>
Signed-off-by: Cheng Pan <chengpan@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
Set Spark 3.2.1 as default.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1913 from SteNicholas/KYUUBI-1876.
Closes#1876
1a6e4c6c [SteNicholas] [KYUUBI #1876] Set Spark 3.2.1 as default
Authored-by: SteNicholas <programgeek@163.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1795 from yanghua/KYUUBI-1787.
Closes#1787
2d01d627 [yanghua] [KYUUBI #1787] Export FLINK_ENGINE_HOME in load-kyuubi-env script
Authored-by: yanghua <yanghua1127@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1793 from yanghua/KYUUBI-1774.
Closes#1774
178a6289 [yanghua] minor fix
ba7cc202 [yanghua] reorder code
7f15ffbb [yanghua] refactor code
85aa1856 [yanghua] [KYUUBI #1774] Export FLINK_HOME in load-kyuubi-env script
Authored-by: yanghua <yanghua1127@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
`bin/kyuubi-zk-cli` should has execute permission
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1748 from pan3793/execute.
Closes#1748
cba11d02 [Cheng Pan] bin/kyuubi-zk-cli should has execute permission
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
When HADOOP_HOME is not set, the stop-application.sh script cannot kill the yarn task even if the yarn command exists. #1649
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [X] Add screenshots for manual tests if appropriate
- [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1650 from wForget/KYUUBI-1649.
Closes#1649
aa8c08d8 [wForget] change error msg
4ec523be [wForget] [KYUUBI-1649] Determine the yarn command in stop-application.sh.
Authored-by: wForget <643348094@qq.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
correct verion of
- Spark Hadoop
- Kyuubi Hadoop
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1618 from ulysses-you/correct-version.
Closes#1618
64c8a5a8 [ulysses-you] grep -v
86d48df9 [ulysses-you] fix
eb39f636 [ulysses-you] fix
Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: Kent Yao <yao@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
1. fix when using `-t` may cause build fail, because tag invalid reference format
2. add new option `-S`; Declare SPARK_HOME in Docker Image. When you configured -S, you need to provide an image
with Spark as BASE_IMAGE.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1495 from zwangsheng/KYUUBI/1480.
Closes#1480
0b43985a [zwangsheng] replace -d & add usage
7bc5d946 [zwangsheng] use SPARK_HOME_IN_DOCKER
7b68a8ba [zwangsheng] remove spark_binary
765dc284 [zwangsheng] rename
387590d8 [zwangsheng] use as build
8f2420c2 [zwangsheng] Merge branch 'KYUUBI/1480' of https://github.com/zwangsheng/incubator-kyuubi into KYUUBI/1480
e3407060 [zwangsheng] replace
1486b1ee [zwangsheng] replace
a775006c [zwangsheng] add comments
f98a7c23 [zwangsheng] 1480
Authored-by: zwangsheng <2213335496@qq.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
When you use docker-image-tool.sh to help build docker image, you can use a non-existent path as SPARK_HOME.You won't be aware of the problem until Dockerfile does the COPY SPARK_HOME operation.
See #1478
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1479 from zwangsheng/KYUUBI/1478.
Closes#1478
cac129eb [zwangsheng] 1478
Authored-by: zwangsheng <2213335496@qq.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
At the beginning, you just want to add a script to help the user build the image.In the process, the Dockerfile of the existing master branch was discovered.
Although the multi-layer build method can help to build quickly and easily, it is only limited to source code.
In other words, when the user uses the binary package officially released, the Dockerfile is deactivated.
So, In this PR, we move it(the multi-layer Dockerfile) to dev. And use the previous Dockerfile.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1435 from zwangsheng/add-docker-image-tools.
Closes#1396
83d593af [zwangsheng] fix note
30772c5e [zwangsheng] fix
499b3969 [zwangsheng] fix
e64ffb7d [zwangsheng] fix
27b54113 [zwangsheng] fix
fcdfc55e [zwangsheng] dix
3aea0a0b [zwangsheng] fix SPARK_HOME print
67a97083 [zwangsheng] remove help code
74efb245 [zwangsheng] move and modify workflow
f7649e86 [zwangsheng] move and modify workflow
4672a09e [zwangsheng] move and modify workflow
7447ce28 [zwangsheng] move
567478d3 [zwangsheng] move
a224ac0b [zwangsheng] move
cadeef29 [zwangsheng] workflow use dev/Dockerfile
354bc14c [zwangsheng] fix Dockerfile
83561d8b [zwangsheng] build spark into docker image
7d4323c3 [zwangsheng] add kyuubi verison help build tag
4748d09b [zwangsheng] use offical image
4f107d8a [zwangsheng] use offical iamhge
f8c8ba9d [zwangsheng] for test
cecd0485 [zwangsheng] modify
09917320 [zwangsheng] add docker-image
Authored-by: zwangsheng <2213335496@qq.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
Add KYUUBI_CTL_JAVA_OPTS to configure java opts of kyuubi-ctl. For details: #1439.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1440 from wForget/KYUUBI-1439.
Closes#1439
bbbf1f23 [wForget] [KYUUBI-1439] Add KYUUBI_CTL_JAVA_OPTS to configure java opts of kyuubi-ctl. update settings.md
da7779a7 [wForget] [KYUUBI-1439] Add KYUUBI_CTL_JAVA_OPTS to configure java opts of kyuubi-ctl.
Authored-by: wForget <643348094@qq.com>
Signed-off-by: Kent Yao <yao@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
Add kyuubi-hive-beeline module to support KyuubiConnection and KyuubiStatement.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
<img width="1410" alt="Screen Shot 2021-11-19 at 1 56 21 PM" src="https://user-images.githubusercontent.com/6757692/142572735-409afe25-b96a-49da-8191-e9a8ced3947b.png">
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1414 from turboFei/rebase-kyuubi_beeline.
Closes#1414
4694bbca [fwang12] return engine launch handle anyway
20ca1400 [fwang12] address comments
2999c30a [fwang12] fix ut
4d3abf4f [fwang12] jars.target.dir to target
4659f1f6 [fwang12] add readme
7e74c5c8 [fwang12] integrate beeline and server
969af045 [fwang12] fix beeline shell
90395ade [fwang12] BeeLine version By Apache Kyuubi
4e63695e [fwang12] fix ut
026e41ef [fwang12] refactor properties
a49677af [fwang12] refactor build and beeline script
ca68eba1 [fwang12] use beeline mode property instead of static method
4a1932e7 [fwang12] cli -> beeline
624281e3 [fwang12] nit
d7d27a09 [fwang12] refactor kyuubi cli assembly
9d9b25a9 [fwang12] simplify the waitLaunchEngineToComplete
cc025ecf [fwang12] Rebase PR_1407
Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
This change enabled the JLine support so that the `kyuubi-zookeeper-shell` command can be easier to use with history, prompt and completion, etc. Note that the `jline` version `0.9.94` is dependent on `zookeeper` version `3.4.14` which is a lower version. BTW, users can consider upgrading the `zookeeper` version for the sake of performance improvements and new features.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1413 from murong00/branch-1413.
Closes#1413
2d1ec084 [sunfangbin] Rename shell script to zk-cli
d09f64dc [sunfangbin] Fixes failed ci
ce88357a [sunfangbin] Improve usage of zookeeper cli
Authored-by: sunfangbin <sunfangbin@cmss.chinamobile.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
It is useful to support a zookeeper shell so that we can view some metadata if necessary. For example:
```
$ bin/kyuubi-zookeeper-shell -server localhost:2181
Connecting to localhost:2181
Welcome to ZooKeeper!
...
ls /
[zookeeper, kyuubi]
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1388 from murong00/branch-1387.
Closes#1388
7698e79c [sunfangbin] Added support a zookeeper shell
Authored-by: sunfangbin <sunfangbin@cmss.chinamobile.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Currently, the enviroment variables are also printed to console when using `bin/kyuubi status` or `bin/kyuubi stop` like below:
```
$ bin/kyuubi status
Using kyuubi environment file /data01/apache-kyuubi-1.4.0-SNAPSHOT-bin-incubating/conf/kyuubi-env.sh to initialize...
JAVA_HOME: /usr/jdk1.8.0_172
KYUUBI_HOME: /data01/apache-kyuubi-1.4.0-SNAPSHOT-bin-incubating
KYUUBI_CONF_DIR: /data01/apache-kyuubi-1.4.0-SNAPSHOT-bin-incubating/conf
KYUUBI_LOG_DIR: /data01/apache-kyuubi-1.4.0-SNAPSHOT-bin-incubating/logs
KYUUBI_PID_DIR: /data01/apache-kyuubi-1.4.0-SNAPSHOT-bin-incubating/pid
KYUUBI_WORK_DIR_ROOT: /data01/apache-kyuubi-1.4.0-SNAPSHOT-bin-incubating/work
SPARK_HOME: /data01/apache-kyuubi-1.4.0-SNAPSHOT-bin-incubating/externals/spark-3.1.2-bin-hadoop3.2
SPARK_CONF_DIR: /data01/apache-kyuubi-1.4.0-SNAPSHOT-bin-incubating/externals/spark-3.1.2-bin-hadoop3.2/conf
HADOOP_CONF_DIR:
Kyuubi is not running
```
In general, these variables are only needed to display when starting a cluster to keep clean style, something like below:
```
$ bin/kyuubi status
Kyuubi is running (pid: 95472)
$ bin/kyuubi stop
Stopping org.apache.kyuubi.server.KyuubiServer
...
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1365 from murong00/branch-1364.
Closes#1365
3856494f [sunfangbin] add checking run command
92686fe1 [sunfangbin] Refine print environment variables to console
Authored-by: sunfangbin <sunfangbin@cmss.chinamobile.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
When kyuubi stops, it will print a confuse message `Starting Kyuubi Server ...`.
When kyuubi starts, KYUUBI_HOME will be included in kyuubi environment output print.
So I think this 'echo' is redundant.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1343 from zhenjiaguo/kyuubi-script-fix.
Closes#1336
3b4cc75c [AnybodyHome] Remove confusing output when kyuubi stop
Authored-by: AnybodyHome <zhenjia_guo@163.com>
Signed-off-by: Kent Yao <yao@apache.org>
kill yarn job when engine initialize timeout and yarnApplicationState is ACCEPTED
#1039
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1119 from simon824/killyarnjob.
Closes#1039
7face4db [simon] get KYUUBI_HOME
da8c3ec4 [Simon] Merge branch 'apache:master' into killyarnjob
95ae8f59 [simon] add appId
8a6ddcd1 [simon] kill application by script
e0cd2af8 [simon] kill application by script
d15d38c5 [simon] kill application by script
1fd1373b [simon] mv sparklauncher to sparkProcessBuilder
5e806426 [simon] fix codestyle
a9cc4505 [simon] fix option No value Exception
9c7ca2f8 [simon] fix option No value Exception
bbfe8e25 [simon] add set sparkHome
04f23c8a [simon] fix codestyle
75599233 [simon] add spark-launcher dep
ef4b2706 [simon] implement by sparkLauncher
59c25b7c [simon] kill yarn application by restful api
49921a48 [simon] fix ut
a31d8f6a [simon] fix ut
1abc6665 [simon] rename killApplication
43a95c1d [simon] Merge branch 'master' into killyarnjob
3d9c12d3 [simon] Merge remote-tracking branch 'upstream/master'
9eaeb16d [simon] fix ut
64ee1b11 [simon] code style
5b905dbd [simon] Merge remote-tracking branch 'upstream/master'
139f3b79 [Simon] Merge branch 'apache:master' into killyarnjob
1a52401d [simon] #1039
Lead-authored-by: simon <zhangshiming@cvte.com>
Co-authored-by: Simon <3656562@qq.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
To make the error message more clear when `SPARK_HOME` is invalid.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1013 from pan3793/sh.
Closes#1013
c9a573b1 [Cheng Pan] nit
e1628640 [Cheng Pan] indent with 2 spaces
68a767f5 [Cheng Pan] Address comments
59d4f076 [Cheng Pan] Enhance load-kyuubi-env.sh
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
When start the server, If not specified `SPARK_CONF_DIR`, it should output default.
```
KYUUBI_WORK_DIR_ROOT: /root/tmp/kyuubi/work
SPARK_HOME: /root/package/spark-3.1.2-bin-hadoop2.7
SPARK_CONF_DIR:
HADOOP_CONF_DIR: /root/package/hadoop-3.2.0/etc/hadoop
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request
Closes#896 from timothy65535/ky-895.
Closes#895
9a936a77 [timothy65535] [KYUUBI #895] Enhance output uses the default spark conf
Authored-by: timothy65535 <timothy65535@163.com>
Signed-off-by: ulysses-you <ulyssesyou18@gmail.com>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/NetEase/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
Add license header on kyuubi ASCII Logo.
### _How was this patch tested?_
Existing UT.
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request
Closes#532 from alex601049502/KYUUBI#517.
Closes#517
38b0d29 [peng.zhao] KYUUBI#517 remove exclude kyuubi-logo
c8102be [peng.zhao] KYUUBI#517 refactor cat logo
90bf6dd [peng.zhao] KYUUBI#517 add new line
8b02f56 [peng.zhao] KYUUBI#529 refactor function name
516894e [peng.zhao] echo logo
Authored-by: peng.zhao <peng.zhao@idiaoyan.com>
Signed-off-by: fwang12 <fwang12@ebay.com>
 [](https://github.com/yaooqinn/kyuubi/pull/359)    [❨?❩](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info)<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
the default versions may be missing if 'MVN' is absent,
```
SPARK_HOME: /Users/kentyao/kyuubi/externals/kyuubi-download/target/spark--bin-hadoop
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request
```
SPARK_HOME: /Users/kentyao/kyuubi/externals/kyuubi-download/target/spark-3.0.2-bin-hadoop2.7
SPARK_CONF_DIR:
HADOOP_CONF_DIR:
Candidate Kyuubi lib /Users/kentyao/kyuubi/jars doesn't exist, searching development environment...
Starting org.apache.kyuubi.server.KyuubiServer, logging to /Users/kentyao/kyuubi/logs/kyuubi-kentyao-org.apache.kyuubi.server.KyuubiServer-hulk.local.out
Welcome to
__ __ __
/\ \/\ \ /\ \ __
\ \ \/'/' __ __ __ __ __ __\ \ \____/\_\
\ \ , < /\ \/\ \/\ \/\ \/\ \/\ \\ \ '__`\/\ \
\ \ \\`\\ \ \_\ \ \ \_\ \ \ \_\ \\ \ \L\ \ \ \
\ \_\ \_\/`____ \ \____/\ \____/ \ \_,__/\ \_\
\/_/\/_/`/___/> \/___/ \/___/ \/___/ \/_/
/\___/
\/__/
```
Closes#359 from yaooqinn/356.
2014862 [Kent Yao] [KYUUBI #356] Fix missing default versions
Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
