apache/kyuubi
1
0
Fork 0
Code Issues Pull Requests Actions 5 Packages Projects Releases Wiki Activity
1,423 Commits 23 Branches 0 Tags 65 MiB
6045104092
Commit Graph

276 Commits

Author SHA1 Message Date
zwangsheng
2f46d4cc49
[KYUUBI #1533] Add documentation about Kyuubi docker-image-tool using Spark image as BASE_IMAGE 
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
Add documentation about Kyuubi docker-image-tool using Spark image as BASE_IMAGE
#1533
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1534 from zwangsheng/KYUUBI/1533.

Closes #1533

803afeed [zwangsheng] fix
9fbc82e0 [zwangsheng] add doc

Authored-by: zwangsheng <2213335496@qq.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
 2021-12-10 11:12:45 +08:00
Kent Yao
c30d580686
[KYUUBI #1527] [DOC] Improve High Availability Guide 
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

![image](https://user-images.githubusercontent.com/8326978/145179241-050e66aa-32d6-471d-82bd-a88972c3ca98.png)

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [x] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1527 from yaooqinn/hadoc.

Closes #1527

249701b6 [Kent Yao] [DOC] Improve High Availability Guide
03244332 [Kent Yao] [DOC] Improve High Availability Guide
54c1cd9d [Kent Yao] [DOC] Improve High Availability Guide
6d8a0991 [Kent Yao] [DOC] Improve High Availability Guide
5b4dff3d [Kent Yao] [DOC] Improve High Availability Guide

Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
 2021-12-09 13:58:00 +08:00
Kent Yao
078edcd231
[KYUUBI #1526] [DOC] Update kyuubi_layers.drawio to add driver layer 
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

![image](https://user-images.githubusercontent.com/8326978/145150050-08b6a13f-9809-44ab-ae41-9beb825f6ff7.png)

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [x] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1526 from yaooqinn/img_kyuubi_layers.

Closes #1526

b769930f [Kent Yao] address comments
2c7de469 [Kent Yao] [DOC] Update kyuubi_layers.drawio to add driver layer

Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
 2021-12-08 17:07:11 +08:00
Kent Yao
97c3835738
[KYUUBI #1490] Introduce the basic framework for running scala 
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

Introduce the basic framework for running scala, see #1490 for the detail
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [x] Add screenshots for manual tests if appropriate
```
Beeline version 1.5.0-SNAPSHOT by Apache Kyuubi (Incubating)
0: jdbc:hive2://10.242.189.214:10009/> spark.version;
2021-12-03 13:47:07.556 INFO operation.ExecuteStatement: Processing kent's query[08b8b6da-d434-4296-b613-2027e3518441]: INITIALIZED_STATE -> PENDING_STATE, statement: spark.version
2021-12-03 13:47:07.560 INFO operation.ExecuteStatement: Processing kent's query[08b8b6da-d434-4296-b613-2027e3518441]: PENDING_STATE -> RUNNING_STATE, statement: spark.version
2021-12-03 13:47:07.558 INFO operation.ExecuteStatement: Processing kent's query[321dc15d-68d0-4f91-9216-1e08f09842df]: INITIALIZED_STATE -> PENDING_STATE, statement: spark.version
2021-12-03 13:47:07.559 INFO operation.ExecuteStatement: Processing kent's query[321dc15d-68d0-4f91-9216-1e08f09842df]: PENDING_STATE -> RUNNING_STATE, statement: spark.version
2021-12-03 13:47:07.560 INFO operation.ExecuteStatement:
           Spark application name: kyuubi_USER_SPARK_SQL_kent_default_61cff9fb-7035-4435-b509-80c1730876ed
                 application ID: local-1638510289918
                 application web UI: http://10.242.189.214:65027
                 master: local[*]
                 deploy mode: client
                 version: 3.1.2
           Start time: 2021-12-03T13:44:49.313
           User: kent
2021-12-03 13:47:07.562 INFO scheduler.DAGScheduler: Asked to cancel job group 321dc15d-68d0-4f91-9216-1e08f09842df
2021-12-03 13:47:07.565 INFO operation.ExecuteStatement: Processing kent's query[321dc15d-68d0-4f91-9216-1e08f09842df]: RUNNING_STATE -> ERROR_STATE, statement: spark.version, time taken: 0.006 seconds
2021-12-03 13:47:07.565 ERROR operation.ExecuteStatement: Error operating EXECUTE_STATEMENT: org.apache.spark.sql.catalyst.parser.ParseException:
mismatched input 'spark' expecting {'(', 'ADD', 'ALTER', 'ANALYZE', 'CACHE', 'CLEAR', 'COMMENT', 'COMMIT', 'CREATE', 'DELETE', 'DESC', 'DESCRIBE', 'DFS', 'DROP', 'EXPLAIN', 'EXPORT', 'FROM', 'GRANT', 'IMPORT', 'INSERT', 'LIST', 'LOAD', 'LOCK', 'MAP', 'MERGE', 'MSCK', 'REDUCE', 'REFRESH', 'REPLACE', 'RESET', 'REVOKE', 'ROLLBACK', 'SELECT', 'SET', 'SHOW', 'START', 'TABLE', 'TRUNCATE', 'UNCACHE', 'UNLOCK', 'UPDATE', 'USE', 'VALUES', 'WITH'}(line 1, pos 0)

== SQL ==
spark.version
^^^

	at org.apache.spark.sql.catalyst.parser.ParseException.withCommand(ParseDriver.scala:255)
	at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parse(ParseDriver.scala:124)
	at org.apache.spark.sql.execution.SparkSqlParser.parse(SparkSqlParser.scala:49)
	at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parsePlan(ParseDriver.scala:75)
	at org.apache.spark.sql.SparkSession.$anonfun$sql$2(SparkSession.scala:616)
	at org.apache.spark.sql.catalyst.QueryPlanningTracker.measurePhase(QueryPlanningTracker.scala:111)
	at org.apache.spark.sql.SparkSession.$anonfun$sql$1(SparkSession.scala:616)
	at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:775)
	at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:613)
	at org.apache.kyuubi.engine.spark.operation.ExecuteStatement.$anonfun$executeStatement$1(ExecuteStatement.scala:98)
	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
	at org.apache.kyuubi.engine.spark.operation.ExecuteStatement.withLocalProperties(ExecuteStatement.scala:157)
	at org.apache.kyuubi.engine.spark.operation.ExecuteStatement.org$apache$kyuubi$engine$spark$operation$ExecuteStatement$$executeStatement(ExecuteStatement.scala:92)
	at org.apache.kyuubi.engine.spark.operation.ExecuteStatement$$anon$1.run(ExecuteStatement.scala:125)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

org.apache.spark.sql.catalyst.parser.ParseException:
mismatched input 'spark' expecting {'(', 'ADD', 'ALTER', 'ANALYZE', 'CACHE', 'CLEAR', 'COMMENT', 'COMMIT', 'CREATE', 'DELETE', 'DESC', 'DESCRIBE', 'DFS', 'DROP', 'EXPLAIN', 'EXPORT', 'FROM', 'GRANT', 'IMPORT', 'INSERT', 'LIST', 'LOAD', 'LOCK', 'MAP', 'MERGE', 'MSCK', 'REDUCE', 'REFRESH', 'REPLACE', 'RESET', 'REVOKE', 'ROLLBACK', 'SELECT', 'SET', 'SHOW', 'START', 'TABLE', 'TRUNCATE', 'UNCACHE', 'UNLOCK', 'UPDATE', 'USE', 'VALUES', 'WITH'}(line 1, pos 0)

== SQL ==
spark.version
^^^

	at org.apache.spark.sql.catalyst.parser.ParseException.withCommand(ParseDriver.scala:255)
	at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parse(ParseDriver.scala:124)
	at org.apache.spark.sql.execution.SparkSqlParser.parse(SparkSqlParser.scala:49)
	at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parsePlan(ParseDriver.scala:75)
	at org.apache.spark.sql.SparkSession.$anonfun$sql$2(SparkSession.scala:616)
	at org.apache.spark.sql.catalyst.QueryPlanningTracker.measurePhase(QueryPlanningTracker.scala:111)
	at org.apache.spark.sql.SparkSession.$anonfun$sql$1(SparkSession.scala:616)
	at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:775)
	at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:613)
	at org.apache.kyuubi.engine.spark.operation.ExecuteStatement.$anonfun$executeStatement$1(ExecuteStatement.scala:98)
	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
	at org.apache.kyuubi.engine.spark.operation.ExecuteStatement.withLocalProperties(ExecuteStatement.scala:157)
	at org.apache.kyuubi.engine.spark.operation.ExecuteStatement.org$apache$kyuubi$engine$spark$operation$ExecuteStatement$$executeStatement(ExecuteStatement.scala:92)
	at org.apache.kyuubi.engine.spark.operation.ExecuteStatement$$anon$1.run(ExecuteStatement.scala:125)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
2021-12-03 13:47:07.569 INFO operation.ExecuteStatement: Query[08b8b6da-d434-4296-b613-2027e3518441] in ERROR_STATE
2021-12-03 13:47:07.569 INFO operation.ExecuteStatement: Processing kent's query[08b8b6da-d434-4296-b613-2027e3518441]: RUNNING_STATE -> ERROR_STATE, statement: spark.version, time taken: 0.009 seconds
Error: Error operating EXECUTE_STATEMENT: org.apache.spark.sql.catalyst.parser.ParseException:
mismatched input 'spark' expecting {'(', 'ADD', 'ALTER', 'ANALYZE', 'CACHE', 'CLEAR', 'COMMENT', 'COMMIT', 'CREATE', 'DELETE', 'DESC', 'DESCRIBE', 'DFS', 'DROP', 'EXPLAIN', 'EXPORT', 'FROM', 'GRANT', 'IMPORT', 'INSERT', 'LIST', 'LOAD', 'LOCK', 'MAP', 'MERGE', 'MSCK', 'REDUCE', 'REFRESH', 'REPLACE', 'RESET', 'REVOKE', 'ROLLBACK', 'SELECT', 'SET', 'SHOW', 'START', 'TABLE', 'TRUNCATE', 'UNCACHE', 'UNLOCK', 'UPDATE', 'USE', 'VALUES', 'WITH'}(line 1, pos 0)

== SQL ==
spark.version
^^^

	at org.apache.spark.sql.catalyst.parser.ParseException.withCommand(ParseDriver.scala:255)
	at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parse(ParseDriver.scala:124)
	at org.apache.spark.sql.execution.SparkSqlParser.parse(SparkSqlParser.scala:49)
	at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parsePlan(ParseDriver.scala:75)
	at org.apache.spark.sql.SparkSession.$anonfun$sql$2(SparkSession.scala:616)
	at org.apache.spark.sql.catalyst.QueryPlanningTracker.measurePhase(QueryPlanningTracker.scala:111)
	at org.apache.spark.sql.SparkSession.$anonfun$sql$1(SparkSession.scala:616)
	at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:775)
	at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:613)
	at org.apache.kyuubi.engine.spark.operation.ExecuteStatement.$anonfun$executeStatement$1(ExecuteStatement.scala:98)
	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
	at org.apache.kyuubi.engine.spark.operation.ExecuteStatement.withLocalProperties(ExecuteStatement.scala:157)
	at org.apache.kyuubi.engine.spark.operation.ExecuteStatement.org$apache$kyuubi$engine$spark$operation$ExecuteStatement$$executeStatement(ExecuteStatement.scala:92)
	at org.apache.kyuubi.engine.spark.operation.ExecuteStatement$$anon$1.run(ExecuteStatement.scala:125)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748) (state=,code=0)
0: jdbc:hive2://10.242.189.214:10009/> set kyuubi.operation.language=scala;
2021-12-03 13:47:11.982 INFO operation.ExecuteStatement: Processing kent's query[e4e4dc55-4e05-460e-9d9d-86b2cf5c054b]: INITIALIZED_STATE -> PENDING_STATE, statement: set kyuubi.operation.language=scala
2021-12-03 13:47:11.985 INFO operation.ExecuteStatement: Processing kent's query[e4e4dc55-4e05-460e-9d9d-86b2cf5c054b]: PENDING_STATE -> RUNNING_STATE, statement: set kyuubi.operation.language=scala
2021-12-03 13:47:11.983 INFO operation.ExecuteStatement: Processing kent's query[2b6ec68c-2ae9-4d93-a35b-9e2319df4d4f]: INITIALIZED_STATE -> PENDING_STATE, statement: set kyuubi.operation.language=scala
2021-12-03 13:47:11.984 INFO operation.ExecuteStatement: Processing kent's query[2b6ec68c-2ae9-4d93-a35b-9e2319df4d4f]: PENDING_STATE -> RUNNING_STATE, statement: set kyuubi.operation.language=scala
2021-12-03 13:47:11.985 INFO operation.ExecuteStatement:
           Spark application name: kyuubi_USER_SPARK_SQL_kent_default_61cff9fb-7035-4435-b509-80c1730876ed
                 application ID: local-1638510289918
                 application web UI: http://10.242.189.214:65027
                 master: local[*]
                 deploy mode: client
                 version: 3.1.2
           Start time: 2021-12-03T13:44:49.313
           User: kent
2021-12-03 13:47:11.995 INFO operation.ExecuteStatement: Processing kent's query[2b6ec68c-2ae9-4d93-a35b-9e2319df4d4f]: RUNNING_STATE -> RUNNING_STATE, statement: set kyuubi.operation.language=scala
2021-12-03 13:47:11.995 INFO operation.ExecuteStatement: Execute in full collect mode
2021-12-03 13:47:12.006 INFO operation.ExecuteStatement: Processing kent's query[2b6ec68c-2ae9-4d93-a35b-9e2319df4d4f]: RUNNING_STATE -> FINISHED_STATE, statement: set kyuubi.operation.language=scala, time taken: 0.022 seconds
2021-12-03 13:47:12.007 INFO operation.ExecuteStatement: Query[e4e4dc55-4e05-460e-9d9d-86b2cf5c054b] in FINISHED_STATE
2021-12-03 13:47:12.007 INFO operation.ExecuteStatement: Processing kent's query[e4e4dc55-4e05-460e-9d9d-86b2cf5c054b]: RUNNING_STATE -> FINISHED_STATE, statement: set kyuubi.operation.language=scala, time taken: 0.022 seconds
+----------------------------+--------+
|            key             | value  |
+----------------------------+--------+
| kyuubi.operation.language  | scala  |
+----------------------------+--------+
1 row selected (0.052 seconds)
0: jdbc:hive2://10.242.189.214:10009/> spark.version;
2021-12-03 13:47:13.685 INFO operation.ExecuteStatement: Processing kent's query[178ff72d-b870-44d4-99d8-08a0fa0e8efa]: INITIALIZED_STATE -> PENDING_STATE, statement: spark.version
2021-12-03 13:47:15.541 INFO operation.ExecuteStatement: Processing kent's query[178ff72d-b870-44d4-99d8-08a0fa0e8efa]: PENDING_STATE -> RUNNING_STATE, statement: spark.version
2021-12-03 13:47:15.543 INFO operation.ExecuteStatement: Query[178ff72d-b870-44d4-99d8-08a0fa0e8efa] in FINISHED_STATE
2021-12-03 13:47:15.544 INFO operation.ExecuteStatement: Processing kent's query[178ff72d-b870-44d4-99d8-08a0fa0e8efa]: RUNNING_STATE -> FINISHED_STATE, statement: spark.version, time taken: 0.003 seconds
+-----------------------+
|        output         |
+-----------------------+
| res0: String = 3.1.2  |
+-----------------------+
1 row selected (1.871 seconds)
0: jdbc:hive2://10.242.189.214:10009/> spark.sql("select current_date()")
. . . . . . . . . . . . . . . . . . .> ;
2021-12-03 13:47:36.512 INFO operation.ExecuteStatement: Processing kent's query[602c2d88-7e8a-4175-9b53-e34adfff007a]: INITIALIZED_STATE -> PENDING_STATE, statement: spark.sql("select current_date()")
2021-12-03 13:47:36.689 INFO operation.ExecuteStatement: Processing kent's query[602c2d88-7e8a-4175-9b53-e34adfff007a]: PENDING_STATE -> RUNNING_STATE, statement: spark.sql("select current_date()")
2021-12-03 13:47:36.692 INFO operation.ExecuteStatement: Query[602c2d88-7e8a-4175-9b53-e34adfff007a] in FINISHED_STATE
2021-12-03 13:47:36.692 INFO operation.ExecuteStatement: Processing kent's query[602c2d88-7e8a-4175-9b53-e34adfff007a]: RUNNING_STATE -> FINISHED_STATE, statement: spark.sql("select current_date()"), time taken: 0.003 seconds
+----------------------------------------------------+
|                       output                       |
+----------------------------------------------------+
| res1: org.apache.spark.sql.DataFrame = [current_date(): date] |
+----------------------------------------------------+
1 row selected (0.187 seconds)
0: jdbc:hive2://10.242.189.214:10009/> results += spark.range(1, 5, 2, 3);
Error: Error operating EXECUTE_STATEMENT: org.apache.kyuubi.KyuubiSQLException: Interpret error:
results += spark.range(1, 5, 2, 3)
 <console>:26: error: type mismatch;
 found   : org.apache.spark.sql.Dataset[Long]
 required: org.apache.spark.sql.Dataset[org.apache.spark.sql.Row]
       results += spark.range(1, 5, 2, 3)
                             ^
	at org.apache.kyuubi.KyuubiSQLException$.apply(KyuubiSQLException.scala:69)
	at org.apache.kyuubi.engine.spark.operation.ExecuteScala.runInternal(ExecuteScala.scala:70)
	at org.apache.kyuubi.operation.AbstractOperation.run(AbstractOperation.scala:130)
	at org.apache.kyuubi.session.AbstractSession.runOperation(AbstractSession.scala:93)
	at org.apache.kyuubi.engine.spark.session.SparkSessionImpl.runOperation(SparkSessionImpl.scala:62)
	at org.apache.kyuubi.session.AbstractSession.$anonfun$executeStatement$1(AbstractSession.scala:121)
	at org.apache.kyuubi.session.AbstractSession.withAcquireRelease(AbstractSession.scala:75)
	at org.apache.kyuubi.session.AbstractSession.executeStatement(AbstractSession.scala:118)
	at org.apache.kyuubi.service.AbstractBackendService.executeStatement(AbstractBackendService.scala:61)
	at org.apache.kyuubi.service.ThriftBinaryFrontendService.ExecuteStatement(ThriftBinaryFrontendService.scala:265)
	at org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1557)
	at org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1542)
	at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
	at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
	at org.apache.kyuubi.service.authentication.TSetIpAddressProcessor.process(TSetIpAddressProcessor.scala:36)
	at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:310)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748) (state=,code=0)
0: jdbc:hive2://10.242.189.214:10009/> results += spark.range(1, 5, 2, 3).toDF;
2021-12-03 13:48:30.327 INFO operation.ExecuteStatement: Processing kent's query[daccd163-978a-4d66-af85-af466086d1db]: INITIALIZED_STATE -> PENDING_STATE, statement: results += spark.range(1, 5, 2, 3).toDF
2021-12-03 13:48:31.700 INFO operation.ExecuteStatement: Processing kent's query[daccd163-978a-4d66-af85-af466086d1db]: PENDING_STATE -> RUNNING_STATE, statement: results += spark.range(1, 5, 2, 3).toDF
2021-12-03 13:48:31.702 INFO operation.ExecuteStatement: Query[daccd163-978a-4d66-af85-af466086d1db] in FINISHED_STATE
2021-12-03 13:48:31.702 INFO operation.ExecuteStatement: Processing kent's query[daccd163-978a-4d66-af85-af466086d1db]: RUNNING_STATE -> FINISHED_STATE, statement: results += spark.range(1, 5, 2, 3).toDF, time taken: 0.003 seconds
+-----+
| id  |
+-----+
| 1   |
| 3   |
+-----+
2 rows selected (1.387 seconds)
```

** Session level isolated **

![image](https://user-images.githubusercontent.com/8326978/144553902-fc390c30-06de-453b-af3d-cf8def577aa9.png)

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1491 from yaooqinn/scala.

Closes #1490

af4d0a13 [Kent Yao] Merge branch 'master' into scala
2ebdf6a3 [Kent Yao] provided scala dep
d58b2bfe [Kent Yao] [KYUUBI #1490] Introduce the basic framework for running scala
d256bfde [Kent Yao] [KYUUBI #1490] Introduce the basic framework for running scala
52090434 [Kent Yao] init

Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
 2021-12-06 11:17:59 +08:00
Fei Wang
4648b63a2c [KYUUBI #1379] Make SESSION_ENGINE_LAUNCH_ASYNC true by default 
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1379 from turboFei/async_open_default.

Closes #1379

a7c4c6a1 [Fei Wang] fix ut
cc8d2730 [Fei Wang] Make SESSION_ENGINE_LAUNCH_ASYNC true by default

Authored-by: Fei Wang <fwang12@ebay.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
 2021-12-04 18:30:13 +08:00
zhichaoleo
7f3f8af902 [KYUUBI #1484] Add a dependency describe for kyuubi on azure delta-lake doc. 
### _Why are the changes needed?_
1. After test again in my local, I found I miss a dependency jar which need to be put into ./spark/jars/

### _How was this patch tested?_
I test thses doc by the steps provided by the doc : https://kyuubi.apache.org/docs/latest/develop_tools/build_document.html

Closes #1484 from zhichaoleo/defect/Fix_Doc_For_Delta.

Closes #1484

a1dff005 [zhichaoleo] Add a dependency describe for kyuubi on azure delta-lake doc.

Authored-by: zhichaoleo <zhichaoleo@163.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
 2021-12-02 17:01:31 +08:00
fwang12
a6d23238cb [KYUUBI #1382] Sync the flink-provided profile in docs and testing suite comments 
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
Sync the flink-provided profile in docs and testing suite comments.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1382 from turboFei/flink_provided.

Closes #1382

3ae96963 [Fei Wang] address nit
ef5bba7f [fwang12] reorder
e9c9c527 [fwang12] save

Lead-authored-by: fwang12 <fwang12@ebay.com>
Co-authored-by: Fei Wang <fwang12@ebay.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
 2021-12-01 20:33:47 +08:00
ulysses-you
712749050e
[KYUUBI #1481] Add z-order docs 
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.

2. If you fix a bug, you can clarify why it is a bug.
-->
introduce how to use z-order with Kyuubi

### _How was this patch tested?_

<img width="1135" alt="iShot2021-12-01 16 08 24" src="https://user-images.githubusercontent.com/12025282/144195844-51ac71a0-de55-4bfd-9db4-ce27422210dd.png">

Closes #1481 from ulysses-you/zorder-docs.

Closes #1481

98b3e5f4 [ulysses-you] comment
7adeacd6 [ulysses-you] comment
62483a88 [ulysses-you] docs

Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: Kent Yao <yao@apache.org>
 2021-12-01 18:50:29 +08:00
Wang Zhen
cc40c87743
[KYUUBI #1472] allow the engine not to self-terminate. 
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

Allow the engine not to self-terminate.  For details: #1472.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1475 from wForget/KYUUBI-1472.

Closes #1472

97412e54 [Wang Zhen] Merge remote-tracking branch 'remotes/origin/master'
8ff0cf72 [Wang Zhen] [KYUUBI-1472] allow the engine not to self-terminate. Use zero or negative to represent no limit.
7cfa03c1 [Wang Zhen] [KYUUBI-1472] allow the engine not to self-terminate.

Authored-by: Wang Zhen <wangzhen07@qiyi.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
 2021-12-01 10:53:37 +08:00
I521853
5022bc1da0
[KYUUBI #1467] Add The Doc of Intergrate Azure Blob Storage Based Delta Lake with Kyuubi 
### _Why are the changes needed?_
I can't find any documents to set up related configurations on kyuubi offical doc site or google search to setup kyuubi with delta-lake (Azure blob storage). So I try to build a doc for this.

### _How was this patch tested?_
I check the doc by following the steps provided by [https://kyuubi.apache.org/docs/latest/develop_tools/build_document.html](https://kyuubi.apache.org/docs/latest/develop_tools/build_document.html).

Closes #1467 from zhichaoleo/feature/azure-based-delta-lake-doc.

Closes #1467

f3ab1dcf [I521853] Split delta lake doc into 2 docs
c7fdb70d [I521853] Layout optimization
878cfd55 [I521853] Add doc for delta lake

Authored-by: I521853 <alex.liu05@sap.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
 2021-11-30 10:24:37 +08:00
zwangsheng
96963ab293
[KYUUBI #1462] [DOC] Use relative paths instead of absolute paths 
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
1. Use relative paths instead of absolute paths, avoid jumping to the latest document page.
2. fix `setting` to `settings`

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1462 from zwangsheng/doc/replace_html_to_md.

Closes #1462

09c97004 [zwangsheng] remove
70379580 [zwangsheng] use uri replace url & fix

Authored-by: zwangsheng <2213335496@qq.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
 2021-11-30 09:32:34 +08:00
zwangsheng
3968832815
[KYUUBI #1454] Add Doc About Deploy Kyuubi On Kubernetes 
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
The Kyuubi documentation is missing about deploying Kyuubi on Kubernetes.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1457 from zwangsheng/doc/kyuubi_on_k8s.

Closes #1454

f916866f [zwangsheng] modify .html to .md
707368ce [zwangsheng] fix url & add todo
1a509781 [zwangsheng] Fix spelling
a624a602 [zwangsheng] fix
4ca74164 [zwangsheng] rename & add

Authored-by: zwangsheng <2213335496@qq.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
 2021-11-29 14:15:19 +08:00
Fei Wang
69b00ae342
[KYUUBI #1461] [DOC] Remove deprecated dist repo directory for deprecated release candidates 
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1461 from turboFei/remove_old_staging.

Closes #1461

a6670bef [Fei Wang] [DOC] Remove deprecated dist repo directory for deprecated release candidates

Authored-by: Fei Wang <fwang12@ebay.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
 2021-11-29 14:10:50 +08:00
Fu Chen
78c9cf3756
[KYUUBI #1443] MaxPartitionStrategy support file source based table. 
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

Fix [comment](https://github.com/apache/incubator-kyuubi/pull/1086#discussion_r707103617)

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1443 from cfmcgrady/max-partition-strategy.

Closes #1443

e071ebc2 [Fu Chen] typo
f7d4ea16 [Fu Chen] refactor
97a36016 [Fu Chen] truncate RootPaths
c3fe93aa [Fu Chen] remove dpp suite
9075e5ba [Fu Chen] refactor
5a71f459 [Fu Chen] trigger GitHub actions
17df25d2 [Fu Chen] fix style
4dfe6a7e [Fu Chen] fix style
0b2df063 [Fu Chen] update docs
305c6e88 [Fu Chen] fix style
33b9b3d5 [Fu Chen] fix style
885e8ed7 [Fu Chen] data source support

Authored-by: Fu Chen <cfmcgrady@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
 2021-11-25 17:34:17 +08:00
wForget
b7e94ff45c
[KYUUBI #1439] Add KYUUBI_CTL_JAVA_OPTS to configure java opts of kyuubi-ctl. 
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

Add KYUUBI_CTL_JAVA_OPTS to configure java opts of kyuubi-ctl. For details: #1439.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1440 from wForget/KYUUBI-1439.

Closes #1439

bbbf1f23 [wForget] [KYUUBI-1439] Add KYUUBI_CTL_JAVA_OPTS to configure java opts of kyuubi-ctl. update settings.md
da7779a7 [wForget] [KYUUBI-1439] Add KYUUBI_CTL_JAVA_OPTS to configure java opts of kyuubi-ctl.

Authored-by: wForget <643348094@qq.com>
Signed-off-by: Kent Yao <yao@apache.org>
 2021-11-24 15:27:42 +08:00
Cheng Pan
116369ac84
[KYUUBI #1376] Implement MySQL Frontend based on Netty 
### _Why are the changes needed?_

Close #1219.

Implment MySQL text protocol, support basic query use mysql cli to run Spark SQL.

Test:

Add `MySQLSparkQuerySuite`.

Change `OutputSchemaTPCDSSuite` from **Thrift binary protocol** to **MySQL protocol**.

Limitations:

1) only support mysql_native_password authentication mechanism
2) only support MySQL text protocol
3) not support send back result which row larger than 16M
4) not support server-side prepared statement
5) not support overwrite engine_conf/session_conf via JDBC url parameters like Hive JDBC client
6) limited support MySQL metadata queries, thus some MySQL GUI client which send metadata queries implicitly may not work as expected
7) not support SSL

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1376 from pan3793/mysql-fe-impl.

Closes #1376

96d3efb9 [Cheng Pan] Kyuubi MySQL FE

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
 2021-11-22 20:35:18 +08:00
zhenjiaguo
f7e5e6a81b
[KYUUBI #1426] fix maven -pl option usage 
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
Details see issue #1426

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1427 from zhenjiaguo/building_doc_fix.

Closes #1426

2c858fbb [zhenjiaguo] fix maven -pl option usage

Authored-by: zhenjiaguo <zhenjia_guo@163.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
 2021-11-22 09:44:05 +08:00
Jagadesh Adireddi
4a9a853e69
[KYUUBI #1181] [BUG] Format file paths to specifications. 
Issue: https://github.com/apache/incubator-kyuubi/issues/1181
### _Why are the changes needed?_
Format file paths of local path and hdfs path to specifications

Closes #1420 from jadireddi/1181-Format.

Closes #1181

32aacc7f [Jagadesh Adireddi] update doc
679369ea [Jagadesh Adireddi] [KYUUBI #1181] [BUG] Format file paths to specifications.

Authored-by: Jagadesh Adireddi <jadiredd@conviva.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
 2021-11-20 14:26:39 +08:00
fwang12
e5e49b5b1d
[KYUUBI #1414] Add kyuubi-hive-beeline module to support KyuubiConnection and KyuubiStatement 
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
Add kyuubi-hive-beeline module to support KyuubiConnection and KyuubiStatement.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [x] Add screenshots for manual tests if appropriate
<img width="1410" alt="Screen Shot 2021-11-19 at 1 56 21 PM" src="https://user-images.githubusercontent.com/6757692/142572735-409afe25-b96a-49da-8191-e9a8ced3947b.png">

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1414 from turboFei/rebase-kyuubi_beeline.

Closes #1414

4694bbca [fwang12] return engine launch handle anyway
20ca1400 [fwang12] address comments
2999c30a [fwang12] fix ut
4d3abf4f [fwang12] jars.target.dir to target
4659f1f6 [fwang12] add readme
7e74c5c8 [fwang12] integrate beeline and server
969af045 [fwang12] fix beeline shell
90395ade [fwang12] BeeLine version By Apache Kyuubi
4e63695e [fwang12] fix ut
026e41ef [fwang12] refactor properties
a49677af [fwang12] refactor build and beeline script
ca68eba1 [fwang12] use beeline mode property instead of static method
4a1932e7 [fwang12] cli -> beeline
624281e3 [fwang12] nit
d7d27a09 [fwang12] refactor kyuubi cli assembly
9d9b25a9 [fwang12] simplify the waitLaunchEngineToComplete
cc025ecf [fwang12] Rebase PR_1407

Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
 2021-11-19 15:05:10 +08:00
yanyu34946
93e9745a39
[KYUUBI #1293] Fix potential failure of engine discovery when mixed use subdomain 
### _Why are the changes needed?_

Fix #1293

This PR is a rework of #1303, also related to #962

We have two options to fix this issue. This PR implements option 2.

```
[option 1]
kyuubi.engine.share.level.subdomain: String -- default: "default"
kyuubi.engine.pool.size: Int                -- default: -1

val subdomain =
    if (kyuubi.engine.share.level.subdomain == "default" && kyuubi.engine.pool.size > 0) {
        s"engine-pool-[num]"
    } else {
        kyuubi.engine.share.level.subdomain
    }

[option 2]
kyuubi.engine.share.level.subdomain: Option[String] -- default: None
kyuubi.engine.pool.size: Int                        -- default: -1

val subdomain =
    kyuubi.engine.share.level.subdomain match {
        case Some(value) => value
        case None if kyuubi.engine.pool.size > 0 => s"engine-pool-[num]"
        case None => "default"
    }
```
```
[diff]
[[case1]]
kyuubi.engine.share.level.subdomain=default
kyuubi.engine.pool.size=2

option 1 result: subdomain="engine-pool-[num]"
option 2 result: subdomain="default"

[[case2]]
kyuubi.engine.share.level.subdomain=hello
kyuubi.engine.pool.size=2

option 1 result: subdomain="hello"
option 2 result: subdomain="hello"

[[case3]]
kyuubi.engine.share.level.subdomain=
kyuubi.engine.pool.size=2

option 1 result: subdomain="engine-pool-[num]"
option 2 result: subdomain="engine-pool-[num]"

[[case4]]
kyuubi.engine.share.level.subdomain=
kyuubi.engine.pool.size=

option 1 result: subdomain="default"
option 2 result: subdomain="default"

[[case5]]
kyuubi.engine.share.level.subdomain="hello"
kyuubi.engine.pool.size=

option 1 result: subdomain="hello"
option 2 result: subdomain="hello"
```

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1402 from pan3793/1293.

Closes #1293

f6c4a7db [Cheng Pan] nit
fd427bfb [yanyu34946] [KYUUBI #1293] Fix bootstrap potential failure when mixed use subdomain

Lead-authored-by: yanyu34946 <875082356@qq.com>
Co-authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
 2021-11-18 18:00:30 +08:00
Kent Yao
bf9736e31b
[KYUUBI #1399] [DOCS] Add doc for engine share level 
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

doc improvement

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [x] Add screenshots for manual tests if appropriate

![image](https://user-images.githubusercontent.com/8326978/141964500-f0121a54-4ce9-4e9d-84a5-c4a4ab1f0ac9.png)

---

![image](https://user-images.githubusercontent.com/8326978/141965482-f367e486-c68b-44d8-85d9-9c7f0e7e63aa.png)
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1399 from yaooqinn/sldoc.

Closes #1399

d8bcce8a [Kent Yao] [DOCS] Add doc for engine share level
21e2548f [Kent Yao] [DOCS] Add doc for engine share level
bba8dc9d [Kent Yao] [DOCS] Add doc for engine share level

Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
 2021-11-16 19:58:25 +08:00
fwang12
1554032df2 [KYUUBI #1346] Support launch query engine asynchronously during opening session 
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

Support launch query engine asynchronously during opening session, make it more user-friendly.

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1346 from turboFei/kyuubi_hive_jdbc_ext.

Closes #1346

513691d6 [fwang12] address comments
63b8f901 [fwang12] remove hanle to client
4590ad98 [fwang12] remove synchronized
f3b136b4 [fwang12] add ut for open session with failure
2cae9afa [fwang12] address comments
35113d0f [fwang12] add engine session id to session event
34c27f9c [fwang12] refactor log
b5f7e639 [fwang12] block
b8b0cf9d [fwang12] to support aync client
66eb07f3 [fwang12] address comments
623c0511 [fwang12] volatile
282c9f07 [fwang12] refactor conf
5047d91d [fwang12] server session handle is not same with that of engine
dc1cb8bc [fwang12] update docs
b773fd9d [fwang12] refactor
5f657e44 [fwang12] add ut
9e56cbab [fwang12] exclude engine init engine
6abdb52c [fwang12] refactor
d3e37a8c [fwang12] block until engine finished
085b324d [fwang12] save

Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: fwang12 <fwang12@ebay.com>
 2021-11-13 17:25:57 +08:00
zhouyifan279
884de795c2
[KYUUBI #1361] Update available version of backported configurations to 1.3.2 
### _Why are the changes needed?_
Sub task of #1361

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1373 from zhouyifan279/1361-4.

Closes #1361

86b92e86 [zhouyifan279] [KYUUBI #1361] Update available version of backported configurations to 1.3.2

Authored-by: zhouyifan279 <zhouyifan279@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
 2021-11-12 22:40:10 +08:00
Cheng Pan
1061d176c8
[KYUUBI #1358] Add KyuubiMySQLFrontendService stub 
### _Why are the changes needed?_

Add KyuubiMySQLFrontendService stub, without Netty pipeline, part of #1334

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1358 from pan3793/mysql-fe-stub.

Closes #1358

c091d955 [Cheng Pan] nit
61abb0f6 [Cheng Pan] Address comments
9ac70456 [Cheng Pan] Update conf
3d75acdf [Cheng Pan] nit
4a4d8a24 [Cheng Pan] Address comments
0136dd52 [Cheng Pan] nit
477474ff [Cheng Pan] KyuubiMySQLFrontendService Stub

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
 2021-11-11 16:50:52 +08:00
Cheng Pan
32c3f5cb2e
[KYUUBI #1349] Add new conf kyuubi.frontend.protocols 
### _Why are the changes needed?_

Adapt to multiple frontends protocol.

This PR also marks REST API as experimental and exposes it to users.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1349 from pan3793/frontend-conf.

Closes #1349

d113c4a6 [Cheng Pan] Simplify code by SAM
129c2c9a [Cheng Pan] Fix compile
fa8f7f67 [Cheng Pan] Add new conf kyuubi.frontend.protocols

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
 2021-11-09 22:09:50 +08:00
zhenjiaguo
0f1d4ec0e0
[KYUUBI #1347] [DOC] Fix miscellaneous doc typos 
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

Some spelling & grammar format fix.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1347 from zhenjiaguo/typo-fix.

Closes #1347

5c20ae28 [zhenjiaguo] change serial to several
db3fa969 [zhenjiaguo] typo fix

Authored-by: zhenjiaguo <zhenjia_guo@163.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
 2021-11-08 09:27:37 +08:00
AnybodyHome
25becb02b9
[KYUUBI #1342] [DOC] Fix beeline incorrect use of double quotes 
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
fix beeline incorrect use of double quotes. Detials see [discussions 1137](https://github.com/apache/incubator-kyuubi/discussions/1337)
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1342 from zhenjiaguo/architecture-doc-fix.

Closes #1342

03c89edf [AnybodyHome] fix beeline incorrect use of double quotes

Authored-by: AnybodyHome <zhenjia_guo@163.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
 2021-11-05 20:16:34 +08:00
yanghua
aad575ce3e
[KYUUBI #1329] Support download Flink binary package in kyuubi-download module 
…

<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1339 from yanghua/KYUUBI-1329.

Closes #1329

895815fb [yanghua] address review suggestion
eddf0605 [yanghua] [KYUUBI #1329] Support download Flink binary package in kyuubi-download module

Authored-by: yanghua <yanghua1127@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
 2021-11-05 18:03:57 +08:00
AnybodyHome
34c58b9133
[KYUUBI #1335] Spell issue branch 
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
Spell check and punctuation check.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1335 from zhenjiaguo/spell-issue-branch.

Closes #1335

b4d48192 [AnybodyHome] recover beeline change
85603b6f [AnybodyHome] spell check and punctuation check

Authored-by: AnybodyHome <zhenjia_guo@163.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
 2021-11-05 09:33:32 +08:00
Cheng Pan
5421b56440
[KYUUBI #1327] Fix socket timeout error when client sync execute statement cost time longer than engine.login.timeout 
### _Why are the changes needed?_

Fix #1327

#### Analysis

The typical error logs are

```log
2021-11-01 11:27:54.018 INFO client.KyuubiSyncThriftClient: TCloseOperationReq(operationHandle:TOperationHandle(operationId:THandleIdentifier(guid:47 69 37 B3 13 38 48 DA 87 7A 8A B6 BD 22 FA 57, secret:C1 01 AE 0B 6F 5F 48 F1 9A F0 FD 84 E3 0F 2B 1E), operationType:EXECUTE_STATEMENT, hasResultSet:true)) succeed on engine side
2021-11-01 11:27:54.019 INFO operation.ExecuteStatement: Processing sy.yao's query[c581db45-67f2-4f15-ac0d-aaecdb713fe9]: INITIALIZED_STATE -> PENDING_STATE, statement: [...SQL]
2021-11-01 11:27:54.019 INFO operation.ExecuteStatement: Processing sy.yao's query[c581db45-67f2-4f15-ac0d-aaecdb713fe9]: PENDING_STATE -> RUNNING_STATE, statement: [...SQL]
2021-11-01 11:28:09.034 INFO operation.ExecuteStatement: Processing sy.yao's query[c581db45-67f2-4f15-ac0d-aaecdb713fe9]: RUNNING_STATE -> ERROR_STATE, statement: [...SQL], time taken: 15.015 seconds
2021-11-01 11:28:09.035 INFO operation.ExecuteStatement: Processing sy.yao's query[c581db45-67f2-4f15-ac0d-aaecdb713fe9]: ERROR_STATE -> CLOSED_STATE, statement: [...SQL]
2021-11-01 11:28:09.035 WARN server.KyuubiThriftBinaryFrontendService: Error executing statement:
org.apache.kyuubi.KyuubiSQLException: Error operating EXECUTE_STATEMENT: org.apache.thrift.transport.TTransportException: java.net.SocketTimeoutException: Read timed out
[...omit detail stacktrace here]
```
The key points here are:

1. client execute query in sync mode, Hive JDBC client use async mode since 2.1.0 [HIVE-6535](https://issues.apache.org/jira/browse/HIVE-6535)
2. query execute cost time great than `kyuubi.session.engine.login.timeout`, which default is `15s`

Kyuubi server create Thrift clients use `kyuubi.session.engine.login.timeout` as both `socket_timeout` and `connect_timeout`, and there is no heartbeat/keepalive mechanism in Thrift Protocol layer, thus if engine does not send response to Kyuubi server in `socket_timeout`, the Thrift client(Kyuubi server) will fail with `SocketTimeoutException: Read timed out`.

In sync mode, when user send a execute statement request to Kyuubi server, Kyuubi server forword the request to the engine, engine return nothing until the execution finished or any other error happend.

In async mode, the query request passes in same way, but engine will return an `op_handle` immediately instead of waiting for query to finish, the client should use the `op_handle` to check query status periodically, in detail

1. Kyuubi server ask engine for query status, and keep the result in memory
2. User ask Kyuubi server for query status, Kyuubi server get result from memory

#### Solution

Option 1: change `socket_timeout` of Thrift clients created by Kyuubi server to infinite. This approach may cause another issue. if engine crash when executing the query, Kyuubi server will wait until the socket keepalive(controlled by OS) timeout, so the query status will always be `running` in Kyuubi server memory.

Option 2: always run a query in async mode, simulate sync mode in Server side.

This PR implement the option 2, also introduce a new conf `kyuubi.session.engine.request.timeout`

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1328 from pan3793/timeout.

Closes #1327

cc15e6d5 [Cheng Pan] Always execute statement in async mode
dc91d0d5 [Cheng Pan] Add new conf `kyuubi.session.engine.request.timeout`

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
 2021-11-04 18:23:10 +08:00
yanghua
b29b39b116
[KYUUBI #1323] Introduce EngineType to distinguish multiple engines 
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1326 from yanghua/KYUUBI-1323.

Closes #1323

aae6dcdd [yanghua] refactor enum to follow scala style
cb5b67c4 [yanghua] addressed review suggestion and fixed test issue
5d73258a [yanghua] regenerated document
3824e838 [yanghua] [KYUUBI #1323] Introduce EngineType to distinguish multiple engines

Authored-by: yanghua <yanghua1127@gmail.com>
Signed-off-by: Kent Yao <yao@apache.org>
 2021-11-04 09:44:08 +08:00
ulysses-you
1003f5296c
[KYUUBI #1321] Add config to control if zorder using global sort 
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
Add a new config `spark.sql.optimizer.zorderGlobalSort.enabled` to control if we do a global sort using zorder. This is a trade-off with data skew if the zorder columns has low cardinality.

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1321 from ulysses-you/zorder-config.

Closes #1321

422599e4 [ulysses-you] delete table
c11e2f4f [ulysses-you] doc
b984b537 [ulysses-you] Add config to control if zorder using global sort

Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
 2021-11-02 19:17:11 +08:00
Cheng Pan
22ad269dfe
[KYUUBI #1308] [DOC] Update release docs 
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
We need to update docs in release finalize phase

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1308 from pan3793/release-docs.

Closes #1308

a79333df [Cheng Pan] nit
d0e22641 [Cheng Pan] [DOC] Update release docs

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
 2021-11-02 17:24:30 +08:00
Fu Chen
cc22888fa7
[KYUUBI #1320] Fix HA documents 
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

1. the default value of `kyuubi.ha.zookeeper.namespace` is `kyuubi`
2. the beeline's variable `zooKeeperNamespace` is case-sensitive
3. how to configure kyuubi ha

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1320 from cfmcgrady/ha-bug.

Closes #1320

16f56377 [Fu Chen] add ha configurations link
7acee980 [Fu Chen] update high_availability_guide.md
bfcab300 [Fu Chen] update high_availability_guide.md
7d9f88cb [Fu Chen] Revert "add conf kyuubi.ha.enabled"
d6874800 [Fu Chen] Revert "update docs"
b31d9ca8 [Fu Chen] Revert "default false"
343bae44 [Fu Chen] default false
3a4934e8 [Fu Chen] update docs
7acf3d52 [Fu Chen] add conf kyuubi.ha.enabled
936d6e7d [Fu Chen] update high_availability_guide.md

Authored-by: Fu Chen <cfmcgrady@gmail.com>
Signed-off-by: Kent Yao <yao@apache.org>
 2021-11-02 17:22:46 +08:00
Kent Yao
fd17dd0ae4
[KYUUBI #1300] Detecting critical errors 
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

For critical errors at engine side, it is not handled properly. For example,when engine oom
- server may not be able to get operation statuses because of no response of engine side. In this case, client only get a ambiguous `read timeout` as a final cause.
- the oom hook of engine side might directly crash the engine, when the server is still trying to get operation status

In this PR,
- a config for retry to make the operation status updating process more robust
- make the engine oom hook only de register itself to make it able to recover for some transient errors

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1312 from yaooqinn/1300.

Closes #1300

a715ecca [Kent Yao] add comments
a816b0f0 [Kent Yao] refine
3557c927 [Kent Yao] add comments
2ed8bfb4 [Kent Yao] refine
aefd2a7f [Kent Yao] update doc
f2ea7e4c [Kent Yao] restore SparkOperation
386e4eac [Kent Yao] [KYUUBI #1300] Detecting critical errors

Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
 2021-11-02 11:46:51 +08:00
Kent Yao
b97512a805
[KYUUBI #1305] [DOC] Fix document errors in quick start 
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

Doc is a feature but the current quick start is full of errors and become a bug.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1305 from yaooqinn/qs.

Closes #1305

5650015f [Kent Yao] [DOC] Fix document errors in quick start

Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
 2021-10-28 17:41:05 +08:00
Kent Yao
4c707d6ec9
[KYUUBI #1302] [DOC] Monitoring Kyuubi - Logging System 
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

Doc is a feature.

In this PR will explain the Logging System of Kyuubi

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1302 from yaooqinn/doc.

Closes #1302

5f36b00b [Kent Yao] [DOC] Monitoring Kyuubi - Logging System
11342fec [Kent Yao] [DOC] Monitoring Kyuubi - Logging System

Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
 2021-10-27 19:38:13 +08:00
Kent Yao
7b3442a709
[KYUUBI #660] Add UDF session_user 
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

As we now fully support GROUP share level, the session user and the system user can be different, so we now need to add UDF session_user.

Also, we fix the system_user to always return the `sparkUser`

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1274 from yaooqinn/660.

Closes #660

c8ed1f77 [Kent Yao] dead code
20c18640 [Kent Yao] nit
4ab1476c [Kent Yao] nit
081f32f2 [Kent Yao] nit
e332ca78 [Kent Yao] Merge branch 'master' into 660
135b058f [Kent Yao] ci
646ddeec [Kent Yao] [KYUUBI #660] Add UDF session_user

Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
 2021-10-26 17:12:50 +08:00
ulysses-you
275d0bd4eb
[KYUUBI #1292] Refine release.md and announce.tmpl 
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
Make release docs better.

### _How was this patch tested?_
Not need.

Closes #1292 from ulysses-you/refine-release.

Closes #1292

fb467563 [ulysses-you] fork and clone
b2418d1d [ulysses-you] mailing list
aa31567b [ulysses-you] refine

Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
 2021-10-26 16:10:06 +08:00
xiebo03
046317c715
[KYUUBI #1286] Increase kyuubi.session.engine.initialize.timeout default to 3 min 
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
close https://github.com/apache/incubator-kyuubi/issues/1286

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1288 from MyLanPangzi/session.engine.initialize.timeout-3min.

Closes #1286

c8b6a54f [xiebo03] modify doc
40d1fbec [xiebo03]  increase kyuubi.session.engine.initialize.timeout default

Authored-by: xiebo03 <xiebo03@qutoutiao.net>
Signed-off-by: Cheng Pan <chengpan@apache.org>
 2021-10-26 15:44:04 +08:00
fwang12
24cf1bd720
[KYUUBI #1262] Support both KERBEROS and PLAIN authentication at the same time 
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

Support both KERBEROS and PLAIN authentication at the same time.

### _How was this patch tested?_

Added UT & IT.
I make integration testing on our dev cluster with KERBEROS and CUSTOM authentication.
```
kyuubi.authentication	KERBEROS,CUSTOM
kyuubi.authentication.custom.class=org.apache.kyuubi.ZeusCustom
```

For org.apache.kyuubi.ZeusCustom,  it checks whether the user equals password.
```
package org.apache.kyuubi

import javax.security.sasl.AuthenticationException

import org.apache.kyuubi.service.authentication.PasswdAuthenticationProvider

class ZeusCustom
  extends PasswdAuthenticationProvider with Logging {

  override def authenticate(user: String, password: String): Unit = {
    if (user == password) {
      info(s"Success log in of user: $user")
    } else {
      throw new AuthenticationException("Username or password is not valid!")
    }
  }
}
```

1. kerberos testing with user b_zeus
![image](https://user-images.githubusercontent.com/6757692/138547539-23ce62cf-07bc-4027-ba91-d5099126afde.png)

2. CUTOM authentication testing with user b_zeus
![image](https://user-images.githubusercontent.com/6757692/138547423-e68b3d82-64ab-450a-8fe7-990cab292fd7.png)

Note that: they share the same backend spark engine, because they are the same user.

Closes #1266 from turboFei/multiple_auth_KYUUBI-1262.

Closes #1262

71053aef [fwang12] adress nit
850d6b5d [fwang12] fix ut
ea7db79f [fwang12] complete
11f409cb [fwang12] Update docs
b1f83e55 [fwang12] add ut
8d137db9 [fwang12] make ldap password diff with custom
d227aa74 [fwang12] fix ut
d7cfaf4c [fwang12] only the first is valid
2e2283ba [fwang12] after all
ee0e8bc0 [fwang12] make kerberoes enabled
4fc63081 [fwang12] refactor kerbereos helper
6691cc57 [fwang12] save
cd813ecf [fwang12] refactor
dd706740 [fwang12] retest
f4038e93 [fwang12] fix code style
7b590a23 [fwang12] add ut
e39e19e6 [fwang12] add it
7dc7c927 [fwang12] with password
8dadfd32 [fwang12] refactor ldap suite
8545a033 [fwang12] add ut
1aa30a5c [fwang12] refactor
8cc2ea66 [fwang12] fix ut
10f788ae [fwang12] before all
98f93640 [fwang12] revert sth
bb75f8e9 [fwang12] save
314579f1 [fwang12] update default
ac8b195f [fwang12] [KYUUBI #1262] Support multiple kinds of SASL authentication type

Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
 2021-10-25 13:03:55 +08:00
fwang12
fe8933c8ec [KYUUBI #1277] Add KDF engine_id 
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
It is useful for client side.

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1277 from turboFei/engine_id.

Closes #1277

95526f09 [fwang12] remove redundant test
382c96dc [fwang12] test all the jdbc tests with yarn mode
01b2ea13 [fwang12] add udf engine_id

Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: fwang12 <fwang12@ebay.com>
 2021-10-23 21:05:16 +08:00
Brian Yue
7665339244
[KYUUBI #460] Upgrade Hive dependency to 2.3.9 
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
Close #460.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1179 from byyue/feature-460.

Closes #460

92eb71b8 [Brian Yue] Upgrade Hive dependency to 2.3.9

Authored-by: Brian Yue <code.byyue@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
 2021-10-23 13:47:22 +08:00
timothy65535
d6cefb1b9c
[KYUUBI #1188] Add Statement stats on Kyuubi Query Engine Page 
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

Add Statement stats on Kyuubi Query Engine Page

![image](https://user-images.githubusercontent.com/86483005/137263757-398a378f-e0b8-498f-b482-60eef650c0ff.png)

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1234 from timothy65535/ky-1188.

Closes #1188

6e0ac523 [timothy65535] trigger rebuild
2b2b2a03 [timothy65535] update patch
f0431359 [timothy65535] [KYUUBI #1188] Add Statement stats on Kyuubi Query Engine Page

Authored-by: timothy65535 <timothy65535@163.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
 2021-10-22 14:52:35 +08:00
Cheng Pan
589b23f578
[KYUUBI #1257] [BUILD] Reduce unnecessary maven profile 
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
The changes based on https://github.com/apache/incubator-kyuubi/pull/1226#issuecomment-942979767

In this PR, I'm going to remove profiles `kyuubi-extension-spark-3-1` and `kyuubi-extension-spark-3-2`, and keep `spark-3.0`, `spark-3.1`, `spark-3.2`.

After changes, when `spark-3.1` is active, set `spark.version` to 3.1.2 and enable `kyuubi-extension-spark-common` module and `kyuubi-extension-spark-3-1` module, `spark-3.2` does the same thing

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1257 from pan3793/pom.

Closes #1257

1d871b6c [Cheng Pan] nit
874f30c1 [Cheng Pan] Address comments
00a32b25 [Cheng Pan] nit
445666ee [Cheng Pan] typo
bd96097e [Cheng Pan] nit
e85116cd [Cheng Pan] nit
98347c8c [Cheng Pan] Update doc
6615d7cc [Cheng Pan] Exclude HudiTest in cross test
087d3fde [Cheng Pan] Add spark 3.2 binary test
ccff63c1 [Cheng Pan] update GA
23b81dcd [Cheng Pan] Address comments
3055c2a8 [Cheng Pan] nit
49a41e86 [Cheng Pan] nit
8d400798 [Cheng Pan] Reduce unnecessary maven profile
ced8d987 [Cheng Pan] Reduce unnecessary maven profile

Lead-authored-by: Cheng Pan <chengpan@apache.org>
Co-authored-by: Cheng Pan <chengpan@ChengdeMac-mini.local>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
 2021-10-22 14:51:42 +08:00
ulysses-you
936a532d2e
[KYUUBI #1269] Online documentation section in the readme is outdated 
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
Closes https://github.com/apache/incubator-kyuubi/issues/1269

### _How was this patch tested?_
docs only

Closes #1270 from ulysses-you/kyuubi#1269.

Closes #1269

1cb99ef5 [ulysses-you] address comment
7a597e66 [ulysses-you] address comment
4b15c2c9 [ulysses-you] Online documentation section in the readme is outdated

Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
 2021-10-22 14:22:46 +08:00
wForget
0653dada40
[KYUUBI #1204] Enhance zookeeper authentication and acls 
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

Please see #1204 for details.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1213 from wForget/KYUUBI-1204.

Closes #1204

fa88ffbe [wForget] [KYUUBI #1204] Change the default value of kyuubi.ha.zookeeper.auth.type to NONE
13cd7ecd [wForget] Merge branch 'master' of https://github.com/apache/incubator-kyuubi into dev-1.3.0
3a84c0e1 [wForget] [KYUUBI #1204] fix tests and re-generate settings.md
92c331c7 [wForget] [KYUUBI #1204] Adjust the parameters of zookeeper authentication
db016a03 [wForget] [KYUUBI #1204] Adjust the logic of obtaining the keytab file path
8adb812b [wForget] [KYUUBI #1204] Enhance zookeeper authentication and acls

Authored-by: wForget <643348094@qq.com>
Signed-off-by: Kent Yao <yao@apache.org>
 2021-10-22 12:19:04 +08:00
hongdongdong
23d821950e
[KYUUBI #1255] [KYUUBI#1253] Support use helm to deploy kyuubi server on k8s 
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
We usally use helm to manager application yaml on k8s, we can use helm to delopy kyuubi server too.
Usages:
```
helm install kyuubi ${kyuubi_helm_url} -n kyuubi
```
```
helm uninstall kyuubi -n kyuubi
```

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1255 from hddong/add-helm.

Closes #1255

e197de34 [hongdongdong] Add doc
d0f74bbb [hongdongdong] rm hel tgz
e9a0a2e4 [hongdongdong] update and fix
f666d331 [hongdongdong] [KYUUBI#1253] Support use helm to deploy kyuubi server on k8s

Authored-by: hongdongdong <hongdongdong@cmss.chinamobile.com>
Signed-off-by: Kent Yao <yao@apache.org>
 2021-10-20 18:16:28 +08:00
Cheng Pan
8fba917673
[KYUUBI #1248] Change kyuubi.engine.session.initialize.sql default value to Nil 
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
The previous default value of `kyuubi.engine.initialize.sql` and `kyuubi.engine.session.initialize.sql` both are `SHOW DATABASES`.
The intention of `SHOW DATABASES`is to eagerly active HiveClient, it's quite redundant for executing many times.
This PR proposes to change `kyuubi.engine.session.initialize.sql`'s default value to `Nil`

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1248 from pan3793/sess-init.

Closes #1248

ad2bdd7d [Cheng Pan] Update doc
0736857c [Cheng Pan] Change kyuubi.engine.session.initialize.sql default value to Nil

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
 2021-10-18 19:15:07 +08:00
Kent Yao
83197ece6d
[KYUUBI #1246] [DOC] Share kyuubi_ecosystem.drawio with the Kyuubi community 
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

Share kyuubi_ecosystem.drawio with the Kyuubi community for developers and users who want to create presentations on it

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1246 from yaooqinn/drawio.

Closes #1246

655b0eb4 [Kent Yao] [DOC] Share kyuubi_ecosystem.drawio with the Kyuubi community

Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
 2021-10-18 14:31:47 +08:00