Cheng Pan
2f8df2355d
[KYUUBI #377 ] build/dist support --spark-provided
...
 [](https://github.com/yaooqinn/kyuubi/pull/378 )    [❨?❩](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info )<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues , add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
close #377
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests ) locally before make a pull request
- [x] Manual test
`build/dist --spark-provided`
```
(kyuubi) ➜ kyuubi git:(master) ✗ tree dist
dist
├── LICENSE
├── RELEASE
├── bin
│ ├── kyuubi
│ ├── kyuubi-logo
│ └── load-kyuubi-env.sh
├── conf
│ ├── kyuubi-defaults.conf.template
│ ├── kyuubi-env.sh.template
│ └── log4j.properties.template
├── externals
│ └── engines
│ └── spark
│ └── kyuubi-spark-sql-engine-1.1.0-SNAPSHOT.jar
├── jars
│ ├── apacheds-i18n-2.0.0-M15.jar
│ ├── apacheds-kerberos-codec-2.0.0-M15.jar
│ ├── api-asn1-api-1.0.0-M20.jar
│ ├── api-util-1.0.0-M20.jar
│ ├── audience-annotations-0.5.0.jar
│ ├── commons-beanutils-1.7.0.jar
│ ├── commons-beanutils-core-1.8.0.jar
│ ├── commons-codec-1.4.jar
│ ├── commons-collections-3.2.2.jar
│ ├── commons-configuration-1.6.jar
│ ├── commons-digester-1.8.jar
│ ├── commons-io-2.4.jar
│ ├── commons-lang-2.6.jar
│ ├── commons-lang3-3.10.jar
│ ├── commons-math-2.2.jar
│ ├── curator-client-2.7.1.jar
│ ├── curator-framework-2.7.1.jar
│ ├── curator-recipes-2.7.1.jar
│ ├── curator-test-2.7.1.jar
│ ├── guava-11.0.2.jar
│ ├── hadoop-annotations-2.7.4.jar
│ ├── hadoop-auth-2.7.4.jar
│ ├── hadoop-common-2.7.4.jar
│ ├── hive-service-rpc-2.3.7.jar
│ ├── httpclient-4.5.6.jar
│ ├── httpcore-4.4.12.jar
│ ├── javassist-3.18.1-GA.jar
│ ├── jcl-over-slf4j-1.7.30.jar
│ ├── jsr305-3.0.2.jar
│ ├── kyuubi-common-1.1.0-SNAPSHOT.jar
│ ├── kyuubi-ha-1.1.0-SNAPSHOT.jar
│ ├── kyuubi-main-1.1.0-SNAPSHOT.jar
│ ├── libfb303-0.9.3.jar
│ ├── libthrift-0.9.3.jar
│ ├── log4j-1.2.17.jar
│ ├── netty-3.7.0.Final.jar
│ ├── scala-library-2.12.10.jar
│ ├── slf4j-api-1.7.30.jar
│ ├── slf4j-log4j12-1.7.30.jar
│ ├── spotbugs-annotations-3.1.9.jar
│ └── zookeeper-3.4.14.jar
├── logs
├── pid
└── work
```
Bundle size
```
(kyuubi) ➜ kyuubi git:(KYUUBI-377) ll | grep tar.gz
-rw-r--r-- 1 chengpan staff 230M Feb 26 12:59 kyuubi-1.1.0-SNAPSHOT-bin-spark-3.0.2.tar.gz
-rw-r--r-- 1 chengpan staff 20M Feb 26 12:56 kyuubi-1.1.0-SNAPSHOT-bin-without-spark.tar.gz
```
Closes #378 from pan3793/KYUUBI-377.
442569e [Cheng Pan] add profile spark-provided for skip download spark binary
0d8da24 [Cheng Pan] update doc
5b7c41f [Cheng Pan] [KYUUBI #377 ] build/dist support --spark-provided
Authored-by: Cheng Pan <379377944@qq.com>
Signed-off-by: ulysses-you <ulyssesyou18@gmail.com>
2021-02-26 15:00:42 +08:00
Cheng Pan
da8d2d4dd3
[KYUUBI #374 ] [DOC] Add Collaborators page
...
 [](https://github.com/yaooqinn/kyuubi/pull/374 )      [❨?❩](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info )<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues , add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests ) locally before make a pull request
Closes #374 from pan3793/doc.
da57a03 [Cheng Pan] remove unused import js script
7e791ee [Cheng Pan] add Collaborators page
Authored-by: Cheng Pan <379377944@qq.com>
Signed-off-by: Kent Yao <yao@apache.org>
2021-02-25 21:47:56 +08:00
Kent Yao
4b13f1c551
[KYUUBI #367 ] Engine terminating checker should start after exposed
...
 [](https://github.com/yaooqinn/kyuubi/pull/367 )      [❨?❩](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info )<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues , add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
Currently, the engine starts to check its lifecycle during the session manager's startup without any initial delay. Before the engine gets ready finally, it might trigger self-terminating.
In this PR, we move the checker to the end where the engine is exposed to the servers.
Avoiding errors like:
```
org.apache.hive.service.cli.HiveSQLException: Error operating GET_CATALOGS: java.net.SocketException: Connection reset
at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:267)
at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:258)
at org.apache.hive.jdbc.HiveDatabaseMetaData.getCatalogs(HiveDatabaseMetaData.java:142)
at org.apache.kyuubi.operation.BasicIcebergJDBCTests.$anonfun$$init$$2(BasicIcebergJDBCTests.scala:50)
at org.apache.kyuubi.operation.BasicIcebergJDBCTests.$anonfun$$init$$2$adapted(BasicIcebergJDBCTests.scala:48)
at org.apache.kyuubi.operation.JDBCTestUtils.$anonfun$withMultipleConnectionJdbcStatement$3(JDBCTestUtils.scala:43)
at org.apache.kyuubi.operation.JDBCTestUtils.$anonfun$withMultipleConnectionJdbcStatement$3$adapted(JDBCTestUtils.scala:43)
at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests ) locally before make a pull request
Closes #367 from yaooqinn/engine.
f6a979e [Kent Yao] fix test
9cc561c [Kent Yao] address comment
3d15ab2 [Kent Yao] nit
a85d2e0 [Kent Yao] Engine terminating checker should start after exposed
Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
2021-02-25 15:10:43 +08:00
ulysses-you
bacff6c59a
[KYUUBI #356 ] Update built-in Spark version to 3.0.2
...
 [](https://github.com/yaooqinn/kyuubi/pull/356 )      [❨?❩](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info )<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues , add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
Spark 3.0.2 has been released now.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests ) locally before make a pull request
Closes #356 from ulysses-you/spark-3.0.2.
c791e19 [ulysses-you] fix
ad6c564 [ulysses-you] 3.0.2
Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou18@gmail.com>
2021-02-20 18:26:26 +08:00
Cheng Pan
600b014070
[KYUUBI #312 ] Support run kyuubi in foreground
...
 [](https://github.com/yaooqinn/kyuubi/pull/352 )     [❨?❩](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info )<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues , add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
close #312
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests ) locally before make a pull request

Closes #352 from pan3793/KYUUBI-312.
1a48504 [Cheng Pan] update help
08f589d [Cheng Pan] update doc and help
563f39e [Cheng Pan] [KYUUBI-312] support run kyuubi in foreground
Authored-by: Cheng Pan <379377944@qq.com>
Signed-off-by: Kent Yao <yao@apache.org>
2021-02-08 23:53:06 +08:00
Cheng Pan
9dc24d6650
[KYUUBI #344 ] [MINOR] update kyuubi version in quick_start
...
 [](https://github.com/yaooqinn/kyuubi/pull/344 )      [❨?❩](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info )<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues , add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests ) locally before make a pull request
Closes #344 from pan3793/docs.
eaa4eb1 [Cheng Pan] [MINOR] update kyuubi version in quick_start
Authored-by: Cheng Pan <379377944@qq.com>
Signed-off-by: Kent Yao <yao@apache.org>
2021-02-03 19:30:45 +08:00
ulysses-you
6a4697019c
[KYUUBI #339 ] Add JDK8 bug troubleshootint
...
 [](https://github.com/yaooqinn/kyuubi/pull/339 )      [❨?❩](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info )<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues , add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
Help user to find the resolution.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests ) locally before make a pull request
Closes #339 from ulysses-you/jdk8-bug.
b25a629 [ulysses-you] init
Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: Kent Yao <yao@apache.org>
2021-02-02 18:05:37 +08:00
jhx1008
ed6775856f
[KYUUBI #327 ] Update contribution guide
...
 [](https://github.com/yaooqinn/kyuubi/pull/327 )      [❨?❩](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info )<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues , add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
keep contribution guideline fresh
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests ) locally before make a pull request
Closes #327 from yaooqinn/updatedoc.
861d434 [jhx1008] Update contribution guide
Authored-by: jhx1008 <jhx1008@gmail.com>
Signed-off-by: Kent Yao <yao@apache.org>
2021-01-30 15:11:49 +08:00
jhx1008
e5dbfe9e8e
[KYUUBI #326 ] Remove outdated deployment guide
...
 [](https://github.com/yaooqinn/kyuubi/pull/326 )      [❨?❩](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info )<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues , add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
the deployment guide is outdated
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests ) locally before make a pull request
Closes #326 from yaooqinn/rmdeploydoc.
7d5393a [Kent Yao] Remove outdated deployment guide
Lead-authored-by: jhx1008 <jhx1008@gmail.com>
Co-authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
2021-01-30 14:57:38 +08:00
jhx1008
ecd1c0e1a6
[KYUUBI #323 ] Fix Kyuubi conf template
...
 [](https://github.com/yaooqinn/kyuubi/pull/323 )     [❨?❩](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info )<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines:
https://kyuubi.readthedocs.io/en/latest/community/contributions.html
-->
### _Which issue are you going to fix?_
<!--
Replace ${ID} below with the actual issue id from
https://github.com/yaooqinn/kyuubi/issues ,
so that the issue will be linked and automatically closed after merging
-->
Fixes #${ID}
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the user case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
the Hadoop conf can not be recognized in `kyuubi-defaulfs.conf`, use HADOOP_CONF_DIR instead
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests ) locally before make a pull request
Closes #323 from yaooqinn/conf.
a5f8a66 [Kent Yao] nit
7ef0046 [jhx1008] Fix Kyuubi conf template
Lead-authored-by: jhx1008 <jhx1008@gmail.com>
Co-authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
2021-01-30 13:36:45 +08:00
jhx1008
97c0ff8395
[KYUUBI #321 ] Add Terminologies Documentation
...
 [](https://github.com/yaooqinn/kyuubi/pull/321 )    [❨?❩](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info )<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines:
https://kyuubi.readthedocs.io/en/latest/community/contributions.html
-->
### _Which issue are you going to fix?_
<!--
Replace ${ID} below with the actual issue id from
https://github.com/yaooqinn/kyuubi/issues ,
so that the issue will be linked and automatically closed after merging
-->
Fixes #${ID}
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the user case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
Add terminology documentation for better understanding both for users and devlopers
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests ) locally before make a pull request
Closes #321 from yaooqinn/terminology.
2a6c7e7 [jhx1008] Add Terminologies Documentation
Authored-by: jhx1008 <jhx1008@gmail.com>
Signed-off-by: Kent Yao <yao@apache.org>
2021-01-30 12:28:42 +08:00
cz1019sniper
a8b970133e
[KYUUBI #319 ] Update Kyuubi Doc Version
...
 [](https://github.com/yaooqinn/kyuubi/pull/319 )     [❨?❩](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info )<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines:
https://kyuubi.readthedocs.io/en/latest/community/contributions.html
-->
### _Which issue are you going to fix?_
<!--
Replace ${ID} below with the actual issue id from
https://github.com/yaooqinn/kyuubi/issues ,
so that the issue will be linked and automatically closed after merging
-->
Fixes #${ID}
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the user case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
the doc version is out of date
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests ) locally before make a pull request
Closes #319 from yaooqinn/versiondoc.
c972479 [cz1019sniper] Update Kyuubi Doc Version
Authored-by: cz1019sniper <tianlove1019@163.com>
Signed-off-by: Kent Yao <yao@apache.org>
2021-01-29 22:59:35 +08:00
Kent Yao
086bfab8b3
[KYUUBI #291 ] Add a quick start guide for DBeaver ( #291 )
...
 [](https://github.com/yaooqinn/kyuubi/pull/291 ) [❨?❩](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info )<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->
<!--
Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
-->
### Please add issue ID here?
<!-- replace ${issue ID} with the actual issue id -->
fixes #291
### Why are the changes needed?
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the user case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
add documentation for dbeaver database tool
### Test Plan:
- Add some test cases that check the changes thoroughly including negative and positive cases if possible
- Add screenshots for manual tests if appropriate

- [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests ) locally before make a pull request
* dbeaver
* dbeaver
* nit
2021-01-16 16:04:15 +08:00
ulysses-you
6dd78652d8
[KYUUBI #284 ] Sesson submit log should be overwritten
...
fixes #284
We will create and write a log during submit a new session (like spark-submit) but we don't delete or overwrite it. This PR aims to add a time config to decide if we need to overwrite a session submit log.
The log path template is `KYUUBI_WORD_DIR/user/$engine-log.$index`. We will increase the `index` automatically if a user create some session concurrently.
After this PR. The log path template is `KYUUBI_WORD_DIR/user/$engine.log.$index`.
Squashed commit of the following:
commit 7794406098df0a3129ccd2ab04a445d2155bef8e
Author: ulysses-you <ulyssesyou18@gmail.com>
Date: Thu Jan 14 19:52:10 2021 +0800
fix
commit 13fd3bbb577c8bdad0ef73851262344d2870fd4d
Author: ulysses-you <ulyssesyou18@gmail.com>
Date: Thu Jan 14 19:02:36 2021 +0800
fix docs
commit 82c192bc6bccf749f7d7232ed4f03f3c0a56c65f
Author: ulysses-you <ulyssesyou18@gmail.com>
Date: Thu Jan 14 18:04:50 2021 +0800
fix
commit 134ea4389b0b9fddc777c79b1a9f43fc59ec3f7e
Author: ulysses-you <ulyssesyou18@gmail.com>
Date: Thu Jan 14 17:20:55 2021 +0800
docs
commit 921ac11b27c028c508e834b88f9f9d9e17eb104a
Author: ulysses-you <ulyssesyou18@gmail.com>
Date: Thu Jan 14 17:16:09 2021 +0800
fix config
commit bfcbd9633a18e9a4cf0b5b2308855ad8af7f7efd
Author: ulysses-you <ulyssesyou18@gmail.com>
Date: Thu Jan 14 16:36:00 2021 +0800
prefix
commit bceb8c59cbd7d7757b2c30058f845fa1e3c7ee9d
Author: ulysses-you <ulyssesyou18@gmail.com>
Date: Thu Jan 14 16:14:20 2021 +0800
doc
commit 4dd556333c7e6e8181f595293bb3877473c00ee7
Author: ulysses-you <ulyssesyou18@gmail.com>
Date: Thu Jan 14 16:11:00 2021 +0800
init
2021-01-14 20:42:11 +08:00
Kent Yao
c6bf6ebf9e
PR ISSUE and Contribution Guide ( #287 )
2021-01-14 20:28:30 +08:00
Kent Yao
776994471b
[DOC] Add v1.0.2 Documentation
2021-01-13 17:50:16 +08:00
Cheng Pan
57ec15d09e
[MINOR] fix typo
...
fixes #277
Squashed commit of the following:
commit 291cb2c9baaf4072689231a9534c41a5eafd9a7e
Author: Cheng Pan <379377944@qq.com>
Date: Tue Jan 12 23:42:47 2021 +0800
update doc
commit 9f706c49f6c59d67c64d93c197cc963217636444
Author: Cheng Pan <379377944@qq.com>
Date: Tue Jan 12 22:04:30 2021 +0800
[MINOR] fix typo
2021-01-13 11:25:19 +08:00
Kent Yao
37b8abfa62
FrontendService tests fix
2021-01-04 15:43:13 +08:00
Kent Yao
aaaeddeb48
Refine documentaion architecture
2021-01-01 02:43:05 +08:00
Kent Yao
d70aec651d
Spark Conf temp
2020-12-31 15:02:21 +08:00
Kent Yao
a1bfcc1273
Kyuubi Architecture documentation
2020-12-30 13:21:13 +08:00
Kent Yao
5d0ec40ac1
Kyuubi Architecture documentation
2020-12-29 16:27:50 +08:00
Kent Yao
6925712798
Kyuubi Architecture documentation
2020-12-29 16:17:56 +08:00
Kent Yao
c6ef275146
Kyuubi Architecture documentation
2020-12-29 16:13:22 +08:00
Kent Yao
ad9c0562ad
Kyuubi Architecture documentation
2020-12-29 16:13:07 +08:00
Kent Yao
3ec0db6a40
Improve Session Share Level
2020-12-27 00:05:07 +08:00
Kent Yao
b11a9d4172
Support _HOST
2020-12-14 20:47:11 +08:00
Kent Yao
d357203b0e
Add authentication guide
2020-11-30 17:45:20 +08:00
Kent Yao
87e243c596
support query log for async exec
2020-11-27 16:40:07 +08:00
Kent Yao
575e9c9599
Add support for user defaults
2020-11-25 21:37:25 +08:00
Kent Yao
dd4fb44d40
Kinit Auxiliary Service should be scheduled w/o delay during starting
2020-11-25 14:38:21 +08:00
Kent Yao
f8ef7c8997
Support ansynchronous exec for Kyuubi Server
2020-11-24 10:55:29 +08:00
Kent Yao
9ca23bcf20
Support ansynchronous exec for Spark SQL engine
2020-11-20 19:52:57 +08:00
Kent Yao
f5900b052f
reorder doc
2020-11-19 11:32:29 +08:00
Kent Yao
91d0ce302e
add HADOOP_CONF_DIR to classpath
2020-11-19 00:30:25 +08:00
Kent Yao
ea1431b162
[DOC] Integration w/ Hive Metastore
2020-11-18 16:37:21 +08:00
Kent Yao
64b83a48c5
Show possible error even timeout
2020-11-16 22:56:25 +08:00
Kent Yao
15820a2b09
fix
2020-11-16 19:42:06 +08:00
Kent Yao
b3d996917d
fix static and core conf via hiveconf or hivevar
2020-11-13 16:08:47 +08:00
Kent Yao
a69fb2553d
nit
2020-11-13 14:48:05 +08:00
Kent Yao
d58484c4ec
pics
2020-11-13 14:18:45 +08:00
Kent Yao
4396e59abe
Add doc for overview
2020-11-13 14:17:42 +08:00
Kent Yao
fb4bace6a5
Ignore static and other unmodifiable settting in session
2020-11-11 11:15:09 +08:00
Kent Yao
574c119712
JDBCTests added and some related bugs fixed
2020-11-10 21:24:02 +08:00
Kent Yao
a67d485951
Missing version in readMessageBegin, old client
...
2020-11-10 10:15:42,001 [61208289] - ERROR [FrontendServiceHandler-Pool: Thread-449:TThreadPoolServer$WorkerProcess@315] - Thrift error occurred during processing of message.
org.apache.thrift.protocol.TProtocolException: Missing version in readMessageBegin, old client?
at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:234)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:27)
at org.apache.kyuubi.service.authentication.TSetIpAddressProcessor.process(TSetIpAddressProcessor.scala:36)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:310)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2020-11-10 17:21:17 +08:00
Kent Yao
f8dc5a9ca3
Using ha.zookeeper.acl.enabled to control ZK acl
2020-11-09 17:34:55 +08:00
Kent Yao
c8bdb5649d
Fix NullPointerException in org.apache.kyuubi.session.KyuubiSessionImpl.close
2020-11-09 14:57:17 +08:00
Kent Yao
56ead65496
Add Trouble Shooting with a HADOOP_CONF_DIR case
2020-11-06 19:40:23 +08:00
Kent Yao
b0bb38fa9d
Improve OpenSession error Msg
2020-11-06 19:00:00 +08:00
Kent Yao
92e3d6d443
FIX org.apache.kyuubi.config.KyuubiConf#toSparkPrefixedConf
...
/**
* This method is used to convert kyuubi configs to configs that Spark could identify.
* - If the key is start with `spark.`, keep it AS IS as it is a Spark Conf
* - If the key is start with `hadoop.`, it will be prefixed with `spark.hadoop.`
* - Otherwise, the key will be added a `spark.` prefix
* @return a map with spark specified configs
*/
2020-11-06 14:37:40 +08:00
Kent Yao
7e1f6ef7ec
Add Trouble Shooting with a UnsupportedClassVersionError case
2020-11-06 11:40:25 +08:00
Kent Yao
39e5da584d
add gitter
2020-11-05 20:49:22 +08:00
Kent Yao
7157370ef6
Add Building Doc
2020-11-05 19:05:26 +08:00
Kent Yao
bd319f4216
remove most tests in kyuubi-server module
2020-11-05 14:14:11 +08:00
Kent Yao
cdb46e90db
Add Configurations Guide
2020-11-04 22:40:02 +08:00
Kent Yao
c888a5b6e0
add logo 2
2020-11-03 21:07:10 +08:00
Kent Yao
2331d9c490
add logo 2
2020-11-03 21:07:04 +08:00
Kent Yao
43bfbed473
add logo
2020-11-03 20:46:58 +08:00
Kent Yao
9aea755b5e
add readthedocs badge
2020-11-03 15:44:06 +08:00
Kent Yao
a8281fc34c
readthedocs
2020-11-03 14:59:32 +08:00
Kent Yao
67858e8c15
[DOCS] Add qucik start documentation
2020-11-02 21:07:11 +08:00
Kent Yao
ba9bff1487
https://stackoverflow.com/questions/56336234/build-fail-sphinx-error-contents-rst-not-found
...
FIX SphinxError: master file /home/docs/checkouts/readthedocs.org/user_builds/kyuubi/checkouts/latest/docs/contents.rst not found
2020-10-28 16:01:56 +08:00
Kent Yao
7fcad51d55
doc init 2
2020-10-28 15:55:21 +08:00
Kent Yao
0abf85dcb7
doc init
2020-10-28 15:28:24 +08:00
Kent Yao
470d7ca90d
Make man service work
2020-09-21 14:05:34 +08:00
Kent Yao
6621d361af
add user info and statement id in executing log ( #212 )
...
* add user info and statement id in executing log
* add ut
* update conf
* long max
* modify log
* modify log
* unpersit
* typo
* add ut
* fix ut
* typo
* code cov
* add log
* ex order
* rm some ex
* none get
* Revert "none get"
This reverts commit 6792e33db11619ec4fa7c075706da1500fb229e3.
* rm debug
2019-08-16 16:51:12 +08:00
Kent Yao
5edf961a44
[KYUUBI-204]Apache Kudu Integration Guide fix #204 ( #205 )
...
* [KYUUBI-204]Apache Kudu Integration Guide fix #204
* references
2019-07-09 16:47:20 +08:00
Kent Yao
63245dd691
[KYUUBI-196]Refactoring SparkSessionWithUGI‘s get and create SparkContext logic ( #197 )
...
* fxit 194 Refactoring SparkSessionWithUGI‘s get and create sc logic
* upgrade scala-maven-plugin to 3.4.6
* fix travis
* fix travis again
* fix travis open jdk8
* fix travis open jdk8
* fix travis open jdk8
* fix travis open jdk8
* fix travis open jdk8
* add state to get yarn applications
* promise complete
* fix #69 capture spark exceptions
* typo
* fix #196 add ut and log
* typo
* typo
* add ut
2019-06-17 11:47:13 +08:00
Kent Yao
e295995959
[KYUUBI-194]Add a server side metrics system ( #195 )
...
* fix #194 add server side metrics system
* add spark session cache size
* add ut
* add ut
* add doc
2019-06-12 11:12:18 +08:00
Kent Yao
505e01d7e2
[KYUUBI-186]Add a max cache time to clean SparkSessions that may have token expiry issue
...
---
---
fix #186 fix #187
---
Squashed commit of the following:
commit 4b8297f8d76b40f22e5419c72f0ddf1eadb3c7d9
Author: Kent Yao <yaooqinn@hotmail.com>
Date: Mon May 20 19:36:34 2019 +0800
add log
commit 7b811f8a0ccb65fb0d0832ba1313e3c099f74dad
Author: Kent Yao <yaooqinn@hotmail.com>
Date: Thu May 16 16:46:04 2019 +0800
add ut
commit 19f67fa8909c4a19f49fbe3b30a75d09c0736350
Author: Kent Yao <yaooqinn@hotmail.com>
Date: Thu May 16 10:57:24 2019 +0800
fix ut
commit 7ad7c20f6536d3bbc9d789c71af238331a3e03c5
Author: Kent Yao <yaooqinn@hotmail.com>
Date: Thu May 16 10:40:42 2019 +0800
fix ut
commit 9a114abb8be3083160ea01dab188333f675677e4
Author: Kent Yao <yaooqinn@hotmail.com>
Date: Thu May 16 00:10:57 2019 +0800
mv cache validating in its own class
commit 0afba5a5ab7a005948acde545b4a68c8d26bfce9
Author: Kent Yao <yaooqinn@hotmail.com>
Date: Wed May 15 11:47:42 2019 +0800
fix ut
commit eff3f41f91b4bfc2bfc84efc6af29fa4365d9dae
Author: Kent Yao <yaooqinn@hotmail.com>
Date: Wed May 15 11:12:51 2019 +0800
add ut
commit 9bbbea78bcb8f5d409819628974628f0bf59f69c
Author: Kent Yao <yaooqinn@hotmail.com>
Date: Tue May 14 23:17:00 2019 +0800
add ut
commit 0e0e59eedc055ddb6788a9dd6f07d4308772145d
Author: Kent Yao <yaooqinn@hotmail.com>
Date: Tue May 14 19:39:09 2019 +0800
mv init time to spark session cache
commit a4a1c69e29ec7d5d51a0e5026cf8f3c2aaf6b2ef
Author: Kent Yao <yaooqinn@hotmail.com>
Date: Tue May 14 18:36:06 2019 +0800
Add a max cache time to clean SparkSessions that may have token expiry issue fix #186
2019-05-21 22:07:18 +08:00
Kent Yao
1f0bc742e4
[KYUUBI-167][FOLLOWUP]populate tokens via spark session cache mgr ( #183 )
...
* Prepare releasing v0.6.0
* fix #KYUUBI-167 populate token via spark session cache mgr
* typo
* fix ut
* code cov
2019-05-07 10:32:04 +08:00
Kent Yao
13c68b5550
[KYUUBI-167]Handling ApplicationMaster token expiry to support long caching sparkcontext with dynamic executor allocation ( #168 )
...
* [KYUUBI-167]Handling ApplicationMaster token expiry to support long caching sparkcontext with dynamic executor allocation - fix #167
* fix ut
* add ut
* fix ut
* add doc
* fix https://github.com/apache/spark/pull/24120
* logger
* typo
* comment/dist jar
* fix ut
* typo
2019-03-19 13:51:08 +08:00
Kent Yao
51e7a5d480
Prepare for developing v0.6.0 ( #154 )
2019-02-27 13:40:29 +08:00
Kent Yao
ae85c245d2
Prepare releasing v0.5.0 ( #152 )
...
* Prepare releasing 0.5.0
* update documentations
2019-02-25 12:33:48 +08:00
Kent Yao
72e664fced
[KYUUBI-116][Experimental] Support long cache spark session in kerberized cluster ( #117 )
...
* fixes @116 Support long caching SparkSession/SparkContext for secured hadoop cluster
* handle sub classes of coarse grained scheduler backend
* fix ut for spark 2.2
* updating doc
2019-02-23 01:10:05 +08:00
Kent Yao
7171f77d7a
fix #140 Add a server side configuration to limit the query result size
2019-01-11 15:37:17 +08:00
Kent Yao
b591f05e4f
modify project version
2018-09-29 16:22:36 +08:00
Kent Yao
5d5b3ec74a
add documentation
2018-09-29 16:05:52 +08:00
Kent Yao
60a9de2791
add documentation
2018-09-29 15:55:06 +08:00
Kent Yao
1c402ccb44
typo
2018-07-30 19:30:02 +08:00
Kent Yao
eb6cde5d52
add pic
2018-07-30 19:20:25 +08:00
Kent Yao
602b985312
add documentation for kyuubi deployment
2018-07-30 19:19:46 +08:00
Kent Yao
6676ebfc01
init commit for making kyuubi server a sub-project fix #90
2018-06-22 14:20:04 +08:00
Kent Yao
5e965a4c35
add doc
2018-06-21 17:26:58 +08:00
Kent Yao
da97da3bb9
add doc
2018-06-21 17:26:51 +08:00
Kent Yao
47deab7bfd
doc
2018-05-31 14:13:34 +08:00
Kent Yao
356009f40a
updating docs 3
2018-05-26 14:30:21 +08:00
Kent Yao
33a85988f0
updating docs
2018-05-26 12:55:03 +08:00
Kent Yao
f7d5e24839
operation configuration should be treated as runtime config
2018-05-25 14:20:37 +08:00
Kent Yao
fbb2687af4
uptate doc
2018-05-21 23:45:28 +08:00
Kent Yao
0deb1bf5b4
doc typo
2018-05-10 14:51:51 +08:00
Kent Yao
0dcbdc9376
add kyuubi architecture descriptions
2018-05-09 20:05:12 +08:00
Kent Yao
1c76bc4138
add sparksession idle timeout parameter
2018-04-20 10:39:08 +08:00
Kent Yao
c9d9cafa93
doc updated
2018-04-09 15:59:29 +08:00
Kent Yao
555fdd4d28
add security documentation
2018-04-08 14:16:07 +08:00
Kent Yao
2e8a92378c
fixes # 49 doc updated
2018-04-04 18:06:28 +08:00
Kent Yao
59cc84eeea
fixed #7 inline exception
2018-03-29 15:29:27 +08:00
Kent Yao
2a2cb51a40
add docs/issue_template.md
2018-03-14 11:06:28 +08:00
Kent Yao
dcbdd128de
doc style
2018-03-07 16:09:45 +08:00
Kent Yao
45442e3fe1
add hive configuration doc
2018-03-07 15:08:59 +08:00
Kent Yao
3c74836463
readme en / configuration doc kyuui part
2018-03-06 23:52:29 +08:00
Kent Yao
78b44f5d27
1. add build script
...
2. add start stop script
2018-01-19 15:55:08 +08:00
Kent Yao
79010ea2ce
init commit for kyuubi
2018-01-05 19:38:54 +08:00