### _Why are the changes needed?_
Hudi has add Spark Sql Support in version 0.9.0.
This PR add Hudi-0.9.0 dependency and Unit Testing #703
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#994 from simon824/master.
Closes#703
a04b2e42 [simon] exclude jdk.tools dep
50e1ab01 [simon] fix jdk11 dep
f33ebdf1 [simon] fix hudi dep conflicts
ff2585c2 [simon] fix yarn dep conflicts
32dd1ea8 [simon] fix scala version conflicts
b8a37401 [simon] add spark.sql.catalogImplementation
82f5422f [simon] fix spark3 dependency
2a6c497d [simon] fix spark3 dependency
4dbec8be [Simon] Merge branch 'apache:master' into master
3f180157 [simon] fix maven
73e48d0a [simon] add spark3 support maven dependency
3def658b [simon] fix missing spark-sql-engine hudi maven dependency
524132d3 [simon] bugfix
e98998a6 [simon] fix pom error
59fc6669 [Simon] Merge branch 'apache:master' into master
d9e17ebf [simon] fix conflict
2466ece3 [simon] add sparkSQL test on hudi-0.9.0
Lead-authored-by: simon <zhangshiming@cvte.com>
Co-authored-by: Simon <3656562@qq.com>
Signed-off-by: Kent Yao <yao@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1003 from yanghua/KYUUBI-999.
Closes#999
78fdf542 [yanghua] Changed rest frontend service port from 10009 to 10099
73929d5d [yanghua] [KYUUBI #999] Build the basic framework for rest frontend service
Authored-by: yanghua <yanghua1127@gmail.com>
Signed-off-by: Kent Yao <yao@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
This PR finishes sub-task #1007 under umbrella issue #913.
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1015 from zhouyifan279/KYUUBI#1007.
Closes#1007
94263804 [zhouyifan279] [KYUUBI #1007] Implement delegation token renewal framework
6cea96d0 [zhouyifan279] [KYUUBI #1007] Implement delegation token renewal framework
dbffcf15 [zhouyifan279] [KYUUBI #1007] Implement delegation token renewal framework
57100271 [zhouyifan279] [KYUUBI #1007] Implement delegation token renewal framework
Authored-by: zhouyifan279 <zhouyifan279@gmail.com>
Signed-off-by: Kent Yao <yao@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1016 from yanghua/KYUUBI-1011.
Closes#1011
78107b76 [yanghua] Addressed review suggestion
651b5989 [yanghua] [KYUUBI #1011] Fallback some frontend service config option about thrift
Authored-by: yanghua <yanghua1127@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou18@gmail.com>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
`subdomain` is a widely used term in computer science
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1010 from yaooqinn/subdomain.
Closes#1010
72ed19a3 [Kent Yao] address comments
4dcae414 [Kent Yao] fix test
c3bb788d [Kent Yao] Rename engine.share.level.sub.domain to engine.share.level.subdomain
b7762357 [Kent Yao] Rename engine.share.level.sub.domain to engine.share.level.subdomain
Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
Fix test
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1001 from ulysses-you/kyuubi-988.
Closes#988
378a75bb [ulysses-you] fix
Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou18@gmail.com>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
Use underscore instead hyphen in path, friendly to sql.
### _How was this patch tested?_
- [X] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1000 from hddong/fix-underscore.
Closes#1000
8b3459a5 [hongdongdong] Use underscore instead hyphen in path
Authored-by: hongdongdong <hongdongdong@cmss.chinamobile.com>
Signed-off-by: ulysses-you <ulyssesyou18@gmail.com>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
closes https://github.com/apache/incubator-kyuubi/issues/988
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#989 from ulysses-you/kyuubi-session-event.
Closes#988
baebf2eb [ulysses-you] address comment
283d574c [ulysses-you] fix
c5d1d5f7 [ulysses-you] adddress comment
44cfd06f [ulysses-you] docs
43639993 [ulysses-you] simplify session
2618b764 [ulysses-you] config name
a2d4608b [ulysses-you] docs
5adbe5c8 [ulysses-you] config
ea5c70e1 [ulysses-you] nit
eea73ed5 [ulysses-you] config
cda6d38b [ulysses-you] test
a754c76f [ulysses-you] init
Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou18@gmail.com>
…
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#984 from yanghua/KYUUBI-983.
Closes#983
db953b8a [yanghua] Rename KyuubiFrontendService to be KyuubiFrontendServices
8c3d7f2a [yanghua] [KYUUBI #983] Refactor the basic abstraction of frontend service
Authored-by: yanghua <yanghua1127@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
According [comments](https://github.com/apache/incubator-kyuubi/pull/991#issuecomment-906914618)
1. Improve error message.
2. add SPARK_HOME in Examples.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#993 from cfmcgrady/ky-991-followup.
Closes#991
e7c25be6 [Fu Chen] update docs
945eeb73 [Fu Chen] review
98555196 [Fu Chen] improve error msg and add SPARK_HOME in Examples
Authored-by: Fu Chen <cfmcgrady@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou18@gmail.com>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
For detail, please go to _Originally posted by yaooqinn in https://github.com/apache/incubator-kyuubi/issues/962#issuecomment-905116261_
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
```
./bin/beeline -u 'jdbc:hive2://localhost:10009/;?kyuubi.engine.pool.enabled=true;kyuubi.engine.pool.size=3;'
```
Closes#986 from timothy65535/ky-962-v2.
Closes#962
b79f8778 [timothy65535] fix test name
9a435607 [timothy65535] fix style
abc746b8 [timothy65535] update tests
71b6933d [timothy65535] revert changes
018dee8b [timothy65535] revert changes
e3492934 [timothy65535] revert changes
67fc8fd9 [timothy65535] improve threshold
fa52acb2 [timothy65535] update pool threshold
d8b5977d [timothy65535] remove dup get
7f39b8d7 [timothy65535] fix test error
4fdf8d76 [timothy65535] update thread pool size to 1 -> threshold
2477348a [timothy65535] rich engine pool tests
8eee427f [timothy65535] fix update error msg
104be5ea [timothy65535] fix error tests
8c2d1441 [timothy65535] fix doc
ddd0eda9 [timothy65535] improve doc & tests
83d7c5cf [timothy65535] update doc
4b6c7ee3 [timothy65535] improve subdomain
169d9d06 [timothy65535] update todo
1424e565 [timothy65535] increase pool size range
3523cd6c [timothy65535] update todo
2cbeafb2 [timothy65535] remove enable option
424b5364 [timothy65535] revert ServiceDiscovery
6fc1c510 [timothy65535] remove random policy
9feef8f3 [timothy65535] [KYUUBI #962] Support engine pool feature
Authored-by: timothy65535 <timothy65535@163.com>
Signed-off-by: Kent Yao <yao@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
As `File.listFiles` may return null when spark package not found in folder `${PROJECT_ROOT_DIR}/kyuubi-server/externals/kyuubi-download/target`, the NPE will throw. Improve the error message.
Before thie pr:
```
Caused by: java.lang.RuntimeException: org.apache.kyuubi.KyuubiSQLException:Error opening session SessionHandle [9749221f-5b30-457b-b8e5-25affba25061] for fchen due to null
at org.apache.kyuubi.KyuubiSQLException$.apply(KyuubiSQLException.scala:68)
at org.apache.kyuubi.session.KyuubiSessionManager.openSession(KyuubiSessionManager.scala:74)
at org.apache.kyuubi.service.AbstractBackendService.openSession(AbstractBackendService.scala:45)
at org.apache.kyuubi.service.ThriftFrontendService.getSessionHandle(ThriftFrontendService.scala:190)
at org.apache.kyuubi.service.ThriftFrontendService.OpenSession(ThriftFrontendService.scala:199)
at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1377)
at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1362)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at org.apache.kyuubi.service.authentication.TSetIpAddressProcessor.process(TSetIpAddressProcessor.scala:36)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException: null
at scala.collection.mutable.ArrayOps$ofRef$.length$extension(ArrayOps.scala:204)
at scala.collection.mutable.ArrayOps$ofRef.length(ArrayOps.scala:204)
at scala.collection.IndexedSeqOptimized.isEmpty(IndexedSeqOptimized.scala:30)
at scala.collection.IndexedSeqOptimized.isEmpty$(IndexedSeqOptimized.scala:30)
at scala.collection.mutable.ArrayOps$ofRef.isEmpty(ArrayOps.scala:198)
at scala.collection.TraversableLike.headOption(TraversableLike.scala:608)
at scala.collection.TraversableLike.headOption$(TraversableLike.scala:608)
at scala.collection.mutable.ArrayOps$ofRef.headOption(ArrayOps.scala:198)
at org.apache.kyuubi.engine.spark.SparkProcessBuilder.$anonfun$executable$1(SparkProcessBuilder.scala:50)
at scala.Option.orElse(Option.scala:447)
at org.apache.kyuubi.engine.spark.SparkProcessBuilder.<init>(SparkProcessBuilder.scala:41)
at org.apache.kyuubi.engine.EngineRef.$anonfun$create$1(EngineRef.scala:140)
at org.apache.kyuubi.engine.EngineRef.tryWithLock(EngineRef.scala:116)
at org.apache.kyuubi.engine.EngineRef.create(EngineRef.scala:128)
at org.apache.kyuubi.engine.EngineRef.$anonfun$getOrCreate$1(EngineRef.scala:182)
at scala.Option.getOrElse(Option.scala:189)
at org.apache.kyuubi.engine.EngineRef.getOrCreate(EngineRef.scala:182)
at org.apache.kyuubi.session.KyuubiSessionImpl.$anonfun$open$2(KyuubiSessionImpl.scala:63)
at org.apache.kyuubi.session.KyuubiSessionImpl.$anonfun$open$2$adapted(KyuubiSessionImpl.scala:62)
at org.apache.kyuubi.ha.client.ZooKeeperClientProvider$.withZkClient(ZooKeeperClientProvider.scala:74)
at org.apache.kyuubi.session.KyuubiSessionImpl.open(KyuubiSessionImpl.scala:62)
at org.apache.kyuubi.session.KyuubiSessionManager.openSession(KyuubiSessionManager.scala:58)
... 12 more
```
After this pr:
```
Caused by: java.lang.RuntimeException: org.apache.kyuubi.KyuubiSQLException:Error opening session SessionHandle [5ea6a8b3-1727-4e0d-b542-992f84afcde8] for fchen due to SPARK_HOME is not set!
at org.apache.kyuubi.KyuubiSQLException$.apply(KyuubiSQLException.scala:68)
at org.apache.kyuubi.session.KyuubiSessionManager.openSession(KyuubiSessionManager.scala:74)
at org.apache.kyuubi.service.AbstractBackendService.openSession(AbstractBackendService.scala:45)
at org.apache.kyuubi.service.ThriftFrontendService.getSessionHandle(ThriftFrontendService.scala:190)
at org.apache.kyuubi.service.ThriftFrontendService.OpenSession(ThriftFrontendService.scala:199)
at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1377)
at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1362)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at org.apache.kyuubi.service.authentication.TSetIpAddressProcessor.process(TSetIpAddressProcessor.scala:36)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: org.apache.kyuubi.KyuubiSQLException:SPARK_HOME is not set!
at org.apache.kyuubi.KyuubiSQLException$.apply(KyuubiSQLException.scala:68)
at org.apache.kyuubi.engine.spark.SparkProcessBuilder.$anonfun$executable$5(SparkProcessBuilder.scala:61)
at scala.Option.getOrElse(Option.scala:189)
at org.apache.kyuubi.engine.spark.SparkProcessBuilder.<init>(SparkProcessBuilder.scala:61)
at org.apache.kyuubi.engine.EngineRef.$anonfun$create$1(EngineRef.scala:140)
at org.apache.kyuubi.engine.EngineRef.tryWithLock(EngineRef.scala:116)
at org.apache.kyuubi.engine.EngineRef.create(EngineRef.scala:128)
at org.apache.kyuubi.engine.EngineRef.$anonfun$getOrCreate$1(EngineRef.scala:182)
at scala.Option.getOrElse(Option.scala:189)
at org.apache.kyuubi.engine.EngineRef.getOrCreate(EngineRef.scala:182)
at org.apache.kyuubi.session.KyuubiSessionImpl.$anonfun$open$2(KyuubiSessionImpl.scala:63)
at org.apache.kyuubi.session.KyuubiSessionImpl.$anonfun$open$2$adapted(KyuubiSessionImpl.scala:62)
at org.apache.kyuubi.ha.client.ZooKeeperClientProvider$.withZkClient(ZooKeeperClientProvider.scala:74)
at org.apache.kyuubi.session.KyuubiSessionImpl.open(KyuubiSessionImpl.scala:62)
at org.apache.kyuubi.session.KyuubiSessionManager.openSession(KyuubiSessionManager.scala:58)
... 12 more
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#991 from cfmcgrady/improve-error-msg.
Closes#991
1e8da683 [Fu Chen] imporove the error message
Authored-by: Fu Chen <cfmcgrady@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou18@gmail.com>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
`isInterrupted` is always false if some `InterruptedException` are throwed. So we should use a new flag to check if log capture thread is released.
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#985 from ulysses-you/flaky-test.
Closes#985
1fcc0a99 [ulysses-you] empty
463a996f [ulysses-you] empty
d1f2ea54 [ulysses-you] eventually
f9f37343 [ulysses-you] fix
Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
Track operation state change and support output to custom system.
### _How was this patch tested?_
- [X] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#968 from hddong/add-server-events.
Closes#922
8be8f268 [hongdongdong] Add prefix for StatementEvent
9fdcc504 [hongdongdong] fix comment
a0cdb35d [hongdongdong] fix test
4d340c67 [hongdongdong] add dependency
00920751 [hongdongdong] [KYUUBI#922]Add OperationEvent for server to track operation state
Authored-by: hongdongdong <hongdongdong@cmss.chinamobile.com>
Signed-off-by: ulysses-you <ulyssesyou18@gmail.com>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#980 from yanghua/KPIP-1.
Closes#979
e67e37a4 [yanghua] [KYUUBI 979] Introduce AbstractFrontendService and KyuubiFrontendService
Authored-by: yanghua <yanghua1127@gmail.com>
Signed-off-by: Kent Yao <yao@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#960 from yanghua/KYUUBI-957.
Closes#957
94cdddda [yanghua] [KYUUBI #957] Reopen KyuubiServerSuite
Authored-by: yanghua <yanghua1127@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### 1. Description
EngineRef use exists znode to determine whether engine started successfully or not(use the last znode), there are inconsistencies in this. So, take an idea to improve the detection mechanism by adding the started sessionid to the znone. `EngineRef` will check the sessionid in a loop after a engine started.
### 2. Here are two options
**Option I, add sesssionid to the data of znode**
**Option II, add sessionid to the path of znode**
### 3. Solution
In order to be consistent with the design of server znode, prefer to choose _Option II_ .
**Demo**
```
/kyuubi
- serviceUri=bigdata:10009;version=1.3.0-SNAPSHOT;sequence=0000000000
/kyuubi_USER/test
- serviceUri=bigdata:39869;version=1.3.0-SNAPSHOT;session=8178069f-0b22-4d06-b1c0-908094769397;sequence=0000000000]
```
Closes#935 from timothy65535/ky-933.
Closes#933
ab5d6d80 [timothy65535] add issue id
d21cb990 [timothy65535] compatible with back
568acf21 [timothy65535] fix error
3db8ba7f [timothy65535] move EngineRef get to Discovery
827f3780 [timothy65535] rename get to getEngineBySessionId
88229314 [timothy65535] update session id using internal
ab06adf8 [timothy65535] improve EngineRef get logic
6a1cc39a [timothy65535] update session version
c94b6e55 [timothy65535] rename sessionId to createSessionId
0cc6aae2 [timothy65535] update ha conf
d24a152f [timothy65535] update ha conf
1a67c864 [timothy65535] fix initialize sql suite
b3b502de [timothy65535] [KYUUBI #933] Enhance the detection mechanism for engine startup
Authored-by: timothy65535 <timothy65535@163.com>
Signed-off-by: Kent Yao <yao@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
Ensure changes like #935 do not break the rule for the semantics of engine sharing
If and only if we want engines to be pooling, this test can be changed
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#958 from yaooqinn/same.
Closes#958
e172cebf [Kent Yao] Ensure two connections in user mode share the same engine
Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#951 from pan3793/license.
Closes#951
4629eecd [Cheng Pan] Fix
c45a0784 [Cheng Pan] nit
b9a46b42 [Cheng Pan] pin license header at first line
80d1a71b [Cheng Pan] nit
b2a46e4c [Cheng Pan] Update
f6acaaf8 [Cheng Pan] minor
ef99183f [Cheng Pan] Add license header on all docs
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
### Description
`ConfigBuilder` currently only supports public configuration by default. In practice, there are also some configurations, which are internal configurations and do not need to be reserved for users.
refer to https://github.com/apache/incubator-kyuubi/pull/935#issuecomment-900117832
### Modification
- Add `internal` to ConfigBuilder
- Update `AllKyuubiConfiguration`
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#949 from timothy65535/ky-942.
Closes#942
5d4723b3 [timothy65535] update config
b41461bb [timothy65535] [KYUUBI #942] Support for adding internal attributes to ConfigBuilder
Authored-by: timothy65535 <timothy65535@163.com>
Signed-off-by: Kent Yao <yao@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
After several benchmarks in the offline environment, kyuubi consumes a lot of CPU.
```
[bigdata]$ jps
3188 Main
6036 Jps
4189 KyuubiServer
[bigdata]$ top
top - 09:34:21 up 23 min, 1 user, load average: 10.67, 7.85, 4.98
Tasks: 320 total, 1 running, 319 sleeping, 0 stopped, 0 zombie
%Cpu(s): 48.2 us, 23.9 sy, 0.0 ni, 27.4 id, 0.0 wa, 0.5 hi, 0.0 si, 0.0 st
MiB Mem : 15806.0 total, 4738.4 free, 6085.0 used, 4982.5 buff/cache
MiB Swap: 0.0 total, 0.0 free, 0.0 used. 9306.5 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
4189 bigdata 20 0 10.1g 311272 22484 S 633.3 1.9 48:12.21 java
```
```
Threads: 53 total, 6 running, 47 sleeping, 0 stopped, 0 zombie
%Cpu(s): 31.9 us, 22.8 sy, 0.0 ni, 44.8 id, 0.0 wa, 0.4 hi, 0.0 si, 0.0 st
MiB Mem : 15806.0 total, 4770.5 free, 6052.9 used, 4982.5 buff/cache
MiB Swap: 0.0 total, 0.0 free, 0.0 used. 9338.7 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
5222 bigdata 20 0 10.1g 311272 22484 R 99.3 1.9 10:00.91 process-logger-
5044 bigdata 20 0 10.1g 311272 22484 R 99.0 1.9 10:03.98 process-logger-
4730 bigdata 20 0 10.1g 311272 22484 R 98.3 1.9 10:13.24 process-logger-
4911 bigdata 20 0 10.1g 311272 22484 R 98.3 1.9 10:06.42 process-logger-
4523 bigdata 20 0 10.1g 311272 22484 R 98.0 1.9 10:41.55 process-logger-
5769 bigdata 20 0 10.1g 311272 22484 R 96.7 1.9 2:46.80 process-logger-
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#941 from timothy65535/ky-940.
Closes#940
331b9423 [timothy65535] [KYUUBI #940] Fix process-logger-capture thread consumes high usage CPU
Authored-by: timothy65535 <timothy65535@163.com>
Signed-off-by: ulysses-you <ulyssesyou18@gmail.com>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
### Description
`ServiceDiscovery` contains `discovery` and `zookeeper` part, `ServiceDiscovery` should only has one responsibility.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#931 from timothy65535/ky-930.
Closes#930
29d48921 [timothy65535] Add ZooKeeperClientProvider
17fbe5a2 [timothy65535] [KYUUBI #930] Extract the zookeeper part from ServiceDiscovery
Authored-by: timothy65535 <timothy65535@163.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
### 1. Describe the feature
Kyuubi use `TestingServer` as the EmbeddedZookeeper service in codebase, and require many args to init the service. These `electionPort`, `quorumPort`, `serverId` parameters only work in cluster mode, these ports are not bound to the host.
From hadoop and bookkeeper project, they use `ZooKeeperServer` as a built-in zk service, especially the bookeeper project.
- https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-registry/src/main/java/org/apache/hadoop/registry/server/services/MicroZookeeperService.java
- https://github.com/apache/bookkeeper/blob/master/bookkeeper-server/src/main/java/org/apache/bookkeeper/shims/zk/ZooKeeperServerShimImpl.java
### 2. Describe the solution
Improve
- Remove `electionPort`, `quorumPort`, `serverId` which are nerver be used, hadoop and bookkeer do the same
- Replace `TestingServer` with `ZooKeeperServer`
- Remove `curator-test` dependency
### 3. `deleteDataDirectoryOnClose`
The name style for `deleteDataDirectoryOnClose` option, refer to hadoop `core-default.xml`
https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/core-default.xml
```
hadoop.security.dns.log-slow-lookups.enabled
hadoop.security.dns.log-slow-lookups.threshold.ms
dfs.client.block.write.replace-datanode-on-failure.policy
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request
Closes#907 from timothy65535/ky-894.
Closes#894
7d7fde9f [timothy65535] remove useless config
ce06a0fd [timothy65535] limit travis build only master branch
ff825c3f [timothy65535] revert deleteDataDirectoryOnClose config
a336544a [timothy65535] remove javassist
3da0bd55 [timothy65535] remove javassist
0abb44df [timothy65535] [KYUUBI #894] Enhance kyuubi-zookeeper module
Authored-by: timothy65535 <timothy65535@163.com>
Signed-off-by: ulysses-you <ulyssesyou18@gmail.com>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
To clear deprecated handleToTSessionHandle in KyuubiOperationManager.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request
Closes#911 from BoyangYue/test-branch.
Closes#905
3da8add4 [Yue Boyang] [KYUUBI #905] [MINOR] clear handleToTSessionHandle in KyuubiOperationManager
Authored-by: Yue Boyang <boyang.yue@student.unimelb.edu.au>
Signed-off-by: Kent Yao <yao@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
Imporve perf of generate config docs.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request
Closes#901 from ulysses-you/enhance-comment.
Closes#901
a86a9aa6 [ulysses-you] skip-download
Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
- Add scala version suffix on scala modules to match the name pattern.
- Enable `kyuubi-extension-spark-3-1` in daily publish snapshot
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request
Closes#888 from pan3793/scala.
Closes#888
571eb9c3 [Cheng Pan] comments
16cb263a [Cheng Pan] remove hardcode scala version in pom
6090c798 [Cheng Pan] Fix shade
5178847f [Cheng Pan] Fixup
a1547753 [Cheng Pan] Fix engine jar name
2cf4793e [Cheng Pan] fix
cac96531 [Cheng Pan] Enable kyuubi-extension-spark-3-1 in daily publish snapshot
1ca31de5 [Cheng Pan] Add scala version suffix on scala modules
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: ulysses-you <ulyssesyou18@gmail.com>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
This PR adds the Kyuubi Query Engine tab on Spark UI, it gathers basic stats currently. We will put more details in the following PRs
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request
Closes#869 from yaooqinn/ui.
Closes#869
aaba5f02 [Kent Yao] Merge branch 'master' of github.com:apache/incubator-kyuubi into ui
0a80a352 [Kent Yao] Add Kyuubi Query Engine for Spark UI with basic stats
ff877fe6 [Kent Yao] Add Kyuubi Query Engine for Spark UI with basic stats
f2e2cabb [Kent Yao] Add Kyuubi Query Engine for Spark UI with basic stats
7f2b5931 [Kent Yao] Add Kyuubi Query Engine for Spark UI with basic stats
cff28784 [Kent Yao] Add Kyuubi Query Engine for Spark UI with basic stats
Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
As we will release soon, it's better to format the module name and artifactId.
The name of kyuubi-main sounds like a bootstrap of Kyuubi but not a server.
### _How was this patch tested?_
Pass CI
Closes#872 from ulysses-you/kyuubi-server.
Closes#871
1437f9b5 [ulysses-you] address comment
117f2c7c [ulysses-you] server
86986baa [ulysses-you] master
eba660a2 [ulysses-you] Merge branch 'master' of https://github.com/NetEase/kyuubi into kyuubi-server
e649012b [ulysses-you] change module name
Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
