### 1. Description
EngineRef use exists znode to determine whether engine started successfully or not(use the last znode), there are inconsistencies in this. So, take an idea to improve the detection mechanism by adding the started sessionid to the znone. `EngineRef` will check the sessionid in a loop after a engine started.
### 2. Here are two options
**Option I, add sesssionid to the data of znode**
**Option II, add sessionid to the path of znode**
### 3. Solution
In order to be consistent with the design of server znode, prefer to choose _Option II_ .
**Demo**
```
/kyuubi
- serviceUri=bigdata:10009;version=1.3.0-SNAPSHOT;sequence=0000000000
/kyuubi_USER/test
- serviceUri=bigdata:39869;version=1.3.0-SNAPSHOT;session=8178069f-0b22-4d06-b1c0-908094769397;sequence=0000000000]
```
Closes#935 from timothy65535/ky-933.
Closes#933
ab5d6d80 [timothy65535] add issue id
d21cb990 [timothy65535] compatible with back
568acf21 [timothy65535] fix error
3db8ba7f [timothy65535] move EngineRef get to Discovery
827f3780 [timothy65535] rename get to getEngineBySessionId
88229314 [timothy65535] update session id using internal
ab06adf8 [timothy65535] improve EngineRef get logic
6a1cc39a [timothy65535] update session version
c94b6e55 [timothy65535] rename sessionId to createSessionId
0cc6aae2 [timothy65535] update ha conf
d24a152f [timothy65535] update ha conf
1a67c864 [timothy65535] fix initialize sql suite
b3b502de [timothy65535] [KYUUBI #933] Enhance the detection mechanism for engine startup
Authored-by: timothy65535 <timothy65535@163.com>
Signed-off-by: Kent Yao <yao@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
### Description
`ServiceDiscovery` contains `discovery` and `zookeeper` part, `ServiceDiscovery` should only has one responsibility.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#931 from timothy65535/ky-930.
Closes#930
29d48921 [timothy65535] Add ZooKeeperClientProvider
17fbe5a2 [timothy65535] [KYUUBI #930] Extract the zookeeper part from ServiceDiscovery
Authored-by: timothy65535 <timothy65535@163.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/NetEase/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
Use [`scopt`](https://github.com/scopt/scopt) replace custom parser to parse arguments.
### _How was this patch tested?_
- [X] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request
Closes#754 from hddong/kyuubi-753.
Closes#753
3d2645d8 [hongdongdong] fix check style
fd181452 [hongdongdong] fix output
ef7c44e4 [hongdongdong] use scopt help
682b3a2f [hongdongdong] [KYUUBI#753] Use scopt to parse arguments
Authored-by: hongdongdong <hongdongdong@cmss.chinamobile.com>
Signed-off-by: Kent Yao <yao@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/NetEase/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
when use `bin/kyuubi-ctl list` throw:
```
Exception in thread "main" org.apache.kyuubi.KyuubiException: Zookeeper namespace is not specified and no default value to load
at org.apache.kyuubi.ctl.ServiceControlCliArguments.fail(ServiceControlCliArguments.scala:339)
at org.apache.kyuubi.ctl.ServiceControlCliArguments.validateZkArguments(ServiceControlCliArguments.scala:136)
```
we expect: list engine nodes with default namespace `kyuubi`
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request
Closes#679 from hddong/fix-ctl-default-value.
Closes#678
bf1e6f2 [hongdongdong] fix
1474aba [hongdongdong] fix
fd07926 [hongdongdong] [KYUUBI #678]Fix: kyuubi ctl use default property if missing
Authored-by: hongdongdong <hongdongdong@cmss.chinamobile.com>
Signed-off-by: Kent Yao <yao@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/NetEase/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request
Closes#669 from pan3793/cleanup.
Closes#669
2cdc4a3 [Cheng Pan] Cleanup code and fix typos
Authored-by: Cheng Pan <379377944@qq.com>
Signed-off-by: Kent Yao <yao@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/NetEase/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
As `zookeeper` mentioned, the result of getChildren is no guarantee order. So it is also no guarantee order that use exists children to create new node .
flaky test log:
```
- test expose zk service node to another namespace *** FAILED ***
"...ceUri=localhost:1000[1];version=1.2.0;seque..." did not equal "...ceUri=localhost:1000[0];version=1.2.0;seque..." (ServiceControlCliSuite.scala:212)
Analysis:
"...ceUri=localhost:1000[1];version=1.2.0;seque..." -> "...ceUri=localhost:1000[0];version=1.2.0;seque..."
```
zookeeper description:
```
/**
* For the given znode path return the stat and children list.
* <p>
* If the watch is true and the call is successful (no exception is thrown),
* a watch will be left on the node with the given path. The watch will be
* triggered by a successful operation that deletes the node of the given
* path or creates/delete a child under the node.
* <p>
* The list of children returned is not sorted and no guarantee is provided
* as to its natural or lexical order.
* <p>
* A KeeperException with error code KeeperException.NoNode will be thrown
* if no node with the given path exists.
*
* since 3.3.0
*
* param path
* param watch
* param stat stat of the znode designated by path
* return an unordered array of children of the node with the given path
* throws InterruptedException If the server transaction is interrupted.
* throws KeeperException If the server signals an error with a non-zero
* error code.
*/
public List<String> getChildren(String path, boolean watch, Stat stat)
throws KeeperException, InterruptedException {
return getChildren(path, watch ? watchManager.defaultWatcher : null,
stat);
}
```
### _How was this patch tested?_
Pass CI.
Closes#653 from ulysses-you/flaky-test-ctl.
Closes#653
0be8248a [ulysses-you] fix
Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: fwang12 <fwang12@ebay.com>
 [](https://github.com/yaooqinn/kyuubi/pull/465)     [<img width="16" alt="Powered by Pull Request Badge" src="https://user-images.githubusercontent.com/1393946/111216524-d2bb8e00-85d4-11eb-821b-ed4c00989c02.png">](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info)<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
```
bin/kyuubi-service <create|get|delete|list> <server|engine> --zkAddress ... --namespace ... --user ... --host ... --port ... --version
Operations:
- create - expose a service to a namespace, this case is rare but sometimes we may want one server to be reached in 2 or more namespaces by different user groups
- get - get the service node info
- delete - delete the specified serviceNode
- list - list all the service nodes for a particular domain
Role:
- server default
- engine
Args:
--zkAddress - one of the zk ensemble address, using kyuubi-defaults/conf if absent
--namespace - the namespace, using kyuubi-defaults/conf if absent
--user -
--host
--port
--version
```
### _How was this patch tested?_
UT
Closes#465 from turboFei/KYUUBI_304_CMD.
Closes#304
4bab34b [fwang12] retest pleaes
8083a12 [fwang12] to increase code converage
c7e51a2 [fwang12] complete
7249cd6 [fwang12] add ut
c809c27 [fwang12] enable set verbose at first
bb3cbb6 [fwang12] fix ut
604820d [fwang12] validate for each action
a01ac1f [fwang12] fix scala style issue
76c9b4c [fwang12] save
7139dd5 [fwang12] increase test converage
318ebce [fwang12] add more ut
72978a6 [fwang12] save
2931f93 [fwang12] address comments
10b855d [fwang12] save
420912a [fwang12] treat help as an action
b27d0a6 [fwang12] treat help as action
896e20d [fwang12] add ctl module into codecov
ea43d69 [fwang12] rename kyuubi-service to kyuubi-ctl
65a0e30 [fwang12] save
db718b0 [fwang12] refactor
41b503e [fwang12] Add kyuubi-ctl arguments parser
cb3f6a8 [fwang12] with log appender
Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: Kent Yao <yao@apache.org>
