Commit Graph

42 Commits

Author SHA1 Message Date
hongdd
164bd03799
[KYUUBI #684]Improve kyuubi ctl verbose output (#685)
* [KYUUBI #684]Improve kyuubi ctl verbose output

* fix
2021-06-28 14:09:29 +08:00
peng.zhao
53ba63cd61 [KYUUBI #517][LICENSE] Add license header on kyuubi ASCII Logo
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/NetEase/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

Add license header on kyuubi ASCII Logo.

### _How was this patch tested?_
Existing UT.

- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request

Closes #532 from alex601049502/KYUUBI#517.

Closes #517

38b0d29 [peng.zhao] KYUUBI#517 remove exclude kyuubi-logo
c8102be [peng.zhao] KYUUBI#517 refactor cat logo
90bf6dd [peng.zhao] KYUUBI#517 add new line
8b02f56 [peng.zhao] KYUUBI#529 refactor function name
516894e [peng.zhao] echo logo

Authored-by: peng.zhao <peng.zhao@idiaoyan.com>
Signed-off-by: fwang12 <fwang12@ebay.com>
2021-04-15 18:38:03 +08:00
fwang12
091a9bec8f [KYUUBI #304] Add kyuubi-ctl shell script
![turboFei](https://badgen.net/badge/Hello/turboFei/green) [![Closes #512](https://badgen.net/badge/Preview/Closes%20%23512/blue)](https://github.com/yaooqinn/kyuubi/pull/512) ![46](https://badgen.net/badge/%2B/46/red) ![0](https://badgen.net/badge/-/0/green) ![1](https://badgen.net/badge/commits/1/yellow) [<img width="16" alt="Powered by Pull Request Badge" src="https://user-images.githubusercontent.com/1393946/111216524-d2bb8e00-85d4-11eb-821b-ed4c00989c02.png">](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info)<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->

<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
Add shell script `kyuubi-ctl` to launch kyuubi service control client.

### _How was this patch tested?_
Run test locally.
![image](https://user-images.githubusercontent.com/6757692/114349132-14bafe00-9b9a-11eb-8e29-53eaa5be4c03.png)

- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request

Closes #512 from turboFei/kyuubi_ctl_shell.

Closes #304

b2a24c0 [fwang12] Add kyuubi-ctl shell script

Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: Cheng Pan <379377944@qq.com>
2021-04-12 15:50:09 +08:00
watermen
fc437a41a4
[KYUUBI #455] Fix Hadoop Version for Dev start stript
![watermen](https://badgen.net/badge/Hello/watermen/green) [![Closes #455](https://badgen.net/badge/Preview/Closes%20%23455/blue)](https://github.com/yaooqinn/kyuubi/pull/455) ![2](https://badgen.net/badge/%2B/2/red) ![2](https://badgen.net/badge/-/2/green) ![1](https://badgen.net/badge/commits/1/yellow) ![Target Issue](https://badgen.net/badge/Missing/Target%20Issue/ff0000) ![Test Plan](https://badgen.net/badge/Missing/Test%20Plan/ff0000) [<img width="16" alt="Powered by Pull Request Badge" src="https://user-images.githubusercontent.com/1393946/111216524-d2bb8e00-85d4-11eb-821b-ed4c00989c02.png">](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info)<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->

Closes #455 from watermen/hadoopversion.

9e2f259 [watermen] Fix Hadoop Version for Dev start stript

Authored-by: watermen <qiyadong2010@gmail.com>
Signed-off-by: Kent Yao <yao@apache.org>
2021-03-26 11:57:36 +08:00
Kent Yao
28db93cc75
[KYUUBI #361] Remove Hive Suffix for built in
![yaooqinn](https://badgen.net/badge/Hello/yaooqinn/green) [![Closes #361](https://badgen.net/badge/Preview/Closes%20%23361/blue)](https://github.com/yaooqinn/kyuubi/pull/361) ![5](https://badgen.net/badge/%2B/5/red) ![11](https://badgen.net/badge/-/11/green) ![1](https://badgen.net/badge/commits/1/yellow) ![Target Issue](https://badgen.net/badge/Missing/Target%20Issue/ff0000) [&#10088;?&#10089;](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info)<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->

<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

the builtin hive1.2 is no need for builtin anymore

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request

Closes #361 from yaooqinn/hivesuffix.

99d49fc [Kent Yao] Remove Hive Suffix for built in

Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
2021-02-24 00:13:20 +08:00
Kent Yao
54c306b967
[KYUUBI #356] Fix missing default versions
![yaooqinn](https://badgen.net/badge/Hello/yaooqinn/green) [![Closes #359](https://badgen.net/badge/Preview/Closes%20%23359/blue)](https://github.com/yaooqinn/kyuubi/pull/359) ![3](https://badgen.net/badge/%2B/3/red) ![2](https://badgen.net/badge/-/2/green) ![1](https://badgen.net/badge/commits/1/yellow) [&#10088;?&#10089;](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info)<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->

<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

the default versions may be missing if 'MVN' is absent,

```
SPARK_HOME: /Users/kentyao/kyuubi/externals/kyuubi-download/target/spark--bin-hadoop
```

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request

```
SPARK_HOME: /Users/kentyao/kyuubi/externals/kyuubi-download/target/spark-3.0.2-bin-hadoop2.7
SPARK_CONF_DIR:
HADOOP_CONF_DIR:

Candidate Kyuubi lib /Users/kentyao/kyuubi/jars doesn't exist, searching development environment...
Starting org.apache.kyuubi.server.KyuubiServer, logging to /Users/kentyao/kyuubi/logs/kyuubi-kentyao-org.apache.kyuubi.server.KyuubiServer-hulk.local.out
Welcome to
  __  __                           __
 /\ \/\ \                         /\ \      __
 \ \ \/'/'  __  __  __  __  __  __\ \ \____/\_\
  \ \ , <  /\ \/\ \/\ \/\ \/\ \/\ \\ \ '__`\/\ \
   \ \ \\`\\ \ \_\ \ \ \_\ \ \ \_\ \\ \ \L\ \ \ \
    \ \_\ \_\/`____ \ \____/\ \____/ \ \_,__/\ \_\
     \/_/\/_/`/___/> \/___/  \/___/   \/___/  \/_/
                /\___/
                \/__/
```

Closes #359 from yaooqinn/356.

2014862 [Kent Yao] [KYUUBI #356] Fix missing default versions

Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
2021-02-23 16:15:49 +08:00
ulysses-you
bacff6c59a [KYUUBI #356] Update built-in Spark version to 3.0.2
![ulysses-you](https://badgen.net/badge/Hello/ulysses-you/green) [![Closes #356](https://badgen.net/badge/Preview/Closes%20%23356/blue)](https://github.com/yaooqinn/kyuubi/pull/356) ![11](https://badgen.net/badge/%2B/11/red) ![3](https://badgen.net/badge/-/3/green) ![2](https://badgen.net/badge/commits/2/yellow) ![Target Issue](https://badgen.net/badge/Missing/Target%20Issue/ff0000) ![Test Plan](https://badgen.net/badge/Missing/Test%20Plan/ff0000) [&#10088;?&#10089;](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info)<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->

<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
Spark 3.0.2 has been released now.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request

Closes #356 from ulysses-you/spark-3.0.2.

c791e19 [ulysses-you] fix
ad6c564 [ulysses-you] 3.0.2

Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou18@gmail.com>
2021-02-20 18:26:26 +08:00
Cheng Pan
600b014070
[KYUUBI #312] Support run kyuubi in foreground
![pan3793](https://badgen.net/badge/Hello/pan3793/green) [![Closes #352](https://badgen.net/badge/Preview/Closes%20%23352/blue)](https://github.com/yaooqinn/kyuubi/pull/352) ![26](https://badgen.net/badge/%2B/26/red) ![3](https://badgen.net/badge/-/3/green) ![3](https://badgen.net/badge/commits/3/yellow) ![Feature](https://badgen.net/badge/Label/Feature/) [&#10088;?&#10089;](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info)<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->

<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
close #312

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [x] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request

![image](https://user-images.githubusercontent.com/26535726/107225226-81f3cb00-6a53-11eb-8c1e-1f0d5f8275b4.png)

Closes #352 from pan3793/KYUUBI-312.

1a48504 [Cheng Pan] update help
08f589d [Cheng Pan] update doc and help
563f39e [Cheng Pan] [KYUUBI-312] support run kyuubi in foreground

Authored-by: Cheng Pan <379377944@qq.com>
Signed-off-by: Kent Yao <yao@apache.org>
2021-02-08 23:53:06 +08:00
Kent Yao
91d0ce302e add HADOOP_CONF_DIR to classpath 2020-11-19 00:30:25 +08:00
Kent Yao
64b83a48c5 Show possible error even timeout 2020-11-16 22:56:25 +08:00
Kent Yao
9bd4139453 Remove Kyuubi useless deps 2020-11-16 20:52:45 +08:00
Kent Yao
91f622f94f Remove Kyuubi 0.8 and before 2020-11-16 20:12:08 +08:00
Kent Yao
e92071a67c Suppport making distribution 2020-09-25 15:40:41 +08:00
Kent Yao
bae3ac071c Fix log4j 2020-09-21 18:08:54 +08:00
Kent Yao
470d7ca90d Make man service work 2020-09-21 14:05:34 +08:00
Kent Yao
01d50d3ec7 add tests for meta operations 2020-09-09 16:15:45 +08:00
Kent Yao
06178e52fe KyuubiServer Part 1 2020-09-08 14:45:40 +08:00
Kent Yao
86e1a7aac3 SparkProcessBuilder 2020-08-18 17:22:08 +08:00
Kent Yao
7fb0b1a32f Enable EmbeddedZkServer for KyuubiServer 2020-06-17 22:42:19 +08:00
Kent Yao
b9a50d62ee new scripts for new framework 2020-05-22 19:16:04 +08:00
Guillem Nieto
a131ce0535 Squashed commit of the following:
commit 065238d830e1bf237cf5b5d74f987827cbe3ba61
Author: Guillem Nieto <gnieto@scopely.com>
Date:   Thu May 9 17:01:37 2019 +0200

    Cast to int wc jar count

    While using Kyuubi on my local environment (OSX) I'm having issues with
    `wc` returning the result with a preffixed tab character.

    Due to this issue, it fails to compare "	1" with "1" and script
    is aborted due to duplicated JARs found.

    To solve that, I'm casting `wc` result to integer and perform integer
    comparisions, which seems to solve the issue.
2019-05-10 12:01:35 +08:00
Kent Yao
13c68b5550
[KYUUBI-167]Handling ApplicationMaster token expiry to support long caching sparkcontext with dynamic executor allocation (#168)
* [KYUUBI-167]Handling ApplicationMaster token expiry to support long caching sparkcontext with dynamic executor allocation - fix #167

* fix ut

* add ut

* fix ut

* add doc

* fix https://github.com/apache/spark/pull/24120

* logger

* typo

* comment/dist jar

* fix ut

* typo
2019-03-19 13:51:08 +08:00
Kent Yao
128928b2fc typo 2019-02-21 17:46:19 +08:00
Kent Yao
07a744d486 add logofor kyuubi 2018-12-29 11:38:30 +08:00
Kent Yao
6676ebfc01 init commit for making kyuubi server a sub-project fix #90 2018-06-22 14:20:04 +08:00
Kent Yao
2d386fff98 typo mistake 2018-05-22 16:26:00 +08:00
Kent Yao
e113a7cf52 bin dir 2018-05-21 19:23:37 +08:00
Kent Yao
57721c0e6a export kyuubi home in kyuubi env 2018-05-21 19:19:28 +08:00
Kent Yao
f1486322ee suitable stop script 2018-05-21 19:12:12 +08:00
Kent Yao
f9d6188f39 add kyuubi submit 2018-05-20 21:42:25 +08:00
Kent Yao
756e9b7f21 .sh 2018-05-20 00:08:04 +08:00
Kent Yao
92b9afebdb disable nice 2018-05-20 00:07:13 +08:00
Kent Yao
f1f96ea82f export kyuubi home 2018-05-20 00:04:35 +08:00
Kent Yao
cfe8f27f71 add permit, export kyuubi home 2018-05-19 23:59:10 +08:00
Kent Yao
4ea8d4d4b2 inline start scripts 2018-05-19 23:52:20 +08:00
Kent Yao
828585e6ea fixes #55 add script for making distribution 2018-04-09 14:59:06 +08:00
Kent Yao
1f7d7993dc mv mvn to build 2018-01-20 00:01:36 +08:00
Kent Yao
58a9a3f998 add stop script 2018-01-19 23:43:24 +08:00
Kent Yao
6fb2cb27e1 use spark-daemon.sh instead of spark-submit 2018-01-19 23:27:08 +08:00
Kent Yao
145bf9e79f add mvn version 2018-01-19 16:30:40 +08:00
Kent Yao
aa90968973 rename start script 2018-01-19 16:08:01 +08:00
Kent Yao
78b44f5d27 1. add build script
2. add start stop script
2018-01-19 15:55:08 +08:00