fbdbcb0276
1494 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Fei Wang
|
fbdbcb0276 |
[KYUUBI #1715] For kyuubi-beeline, using kyuubi-hive-jdbc instead of kyuubi-hive-jdbc-shaded
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> For kyuubi-beeline module, shaded kyuubi-hive-jdbc is not needed, using kyuubi-hive-jdbc to decrease the binary size. ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1715 from turboFei/lighten_beeline. Closes #1715 ebcde1df [Fei Wang] use kyuubi-hive-jdbc instead of kyuubi-hive-jdbc-shaded Authored-by: Fei Wang <fwang12@ebay.com> Signed-off-by: Fei Wang <fwang12@ebay.com> |
||
|
Kent Yao
|
a9d8fb2e7c
|
[KYUUBI #1712] Bump up maven.version to 3.8.4
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1712 from yaooqinn/mvnv. Closes #1712 93793468 [Kent Yao] Bump up maven.version to 3.8.4 Authored-by: Kent Yao <yao@apache.org> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
Fei Wang
|
e0eeab07eb
|
[KYUUBI #1710] Support to specify OPERATION_LANGUAGE with TExecuteStatementReq confOverlay
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> Now kyuubi supports SQL and SCALA language. Now the way to switch OPERATION_LANAUAGE: ``` // for SCALA spark.sql("set kyuubi.operation.language=SCALA") // for SQL set kyuubi.operation.language=SCALA ``` It is more user friendly that user can specify the OPERATION_LANGUAGE for each ExecuteStatement. ### _How was this patch tested?_ - [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1710 from turboFei/conf_overlay. Closes #1710 385264dc [Fei Wang] fix ut 58fbdbdf [Fei Wang] fix npe 2718fb48 [Fei Wang] Support to get OPERATION_LANGUAGE from execute statement confOverlay Authored-by: Fei Wang <fwang12@ebay.com> Signed-off-by: Cheng Pan <chengpan@apache.org> |
||
|
yanghua
|
58dd4ca71f
|
[KYUUBI #1647] Change GetCatalog returned schema key name to support JDBC specification
… <!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1688 from yanghua/KYUUBI-1687. Closes #1647 d6fda463 [yanghua] [KYUUBI #1647] Change GetCatalog returned schema key name to support JDBC specification Authored-by: yanghua <yanghua1127@gmail.com> Signed-off-by: Cheng Pan <chengpan@apache.org> |
||
|
Fei Wang
|
0dbe06437d
|
[KYUUBI #1690] Checking whether launchEngineOpHandle is null for KyuubiConnection:: getExecLog
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> I am developing internal submit application operation and there is no backend engine needed to launch. But I meet below exception. ``` 00:07:40.744 KyuubiThriftBinaryFrontendServiceHandler-Pool: Thread-37 ERROR server.KyuubiThriftBinaryFrontendService: Error fetching results: java.lang.NullPointerException at org.apache.kyuubi.service.ThriftBinaryFrontendService.FetchResults(ThriftBinaryFrontendService.scala:494) at org.apache.hive.service.rpc.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1837) at org.apache.hive.service.rpc.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1822) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.kyuubi.service.authentication.TSetIpAddressProcessor.process(TSetIpAddressProcessor.scala:36) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834) ``` it was invoked in KyuubiCommands::showRemainingLogsIfAny ``` private void showRemainingLogsIfAny(Object sqlObject) { if (sqlObject instanceof KyuubiLoggable) { KyuubiLoggable kyuubiLoggable = (KyuubiLoggable) sqlObject; List<String> logs = null; do { try { logs = kyuubiLoggable.getExecLog(); ``` And if using the kyuubi beeline to connect old kyuubi server, it will meet the same issue. ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1690 from turboFei/fix_typo. Closes #1690 cb443441 [Fei Wang] fix typo f2e1c812 [Fei Wang] Make the getExecLogs method safe Authored-by: Fei Wang <fwang12@ebay.com> Signed-off-by: Cheng Pan <chengpan@apache.org> |
||
|
zhenjiaguo
|
2a7f366b69
|
[KYUUBI #1697] BackendService method avg call times and rate
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> BackendService method avg call times and rate. Replace Histograms with Timer to record it. Timer include both Meters and Histograms. ### _How was this patch tested?_ - [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1708 from zhenjiaguo/use_timer_for_backend_service. Closes #1697 3a6af4d3 [zhenjiaguo] comment 0914af2f [zhenjiaguo] use timer to replace histograms Authored-by: zhenjiaguo <zhenjiaguo@gmail.com> Signed-off-by: ulysses-you <ulyssesyou@apache.org> |
||
|
Paul Lin
|
e2fb7d6f9b |
[KYUUBI #1681] Extend Flink ExecuteStatement to support more operations
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> ExecuteStatement of Flink engine now supports QueryOperation only. We can extend it to support more operations that Flink Executor already supports. e.g. ShowTableOperation,CreateTableOperation, ExplainOperation. This is a sub-task of KPIP-2 #1322 . ### _How was this patch tested?_ - [X] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1693 from link3280/feature/extend_flink_operation. Closes #1681 520d1f42 [Paul Lin] [KYUUBI #1681] Extend Flink ExecuteStatement to support more operations Authored-by: Paul Lin <paullin3280@gmail.com> Signed-off-by: yanghua <yanghua1127@gmail.com> |
||
|
yanghua
|
6f4427b1ca |
[KYUUBI #1705] Support decimal type for Flink SQL Engine
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1706 from yanghua/KYUUBI-1705. Closes #1705 fad35d78 [yanghua] [KYUUBI #1705] Support decimal type for Flink SQL Engine Authored-by: yanghua <yanghua1127@gmail.com> Signed-off-by: yanghua <yanghua1127@gmail.com> |
||
|
simon
|
c43d3a7d22
|
[KYUUBI #1691] Implement api: /${version}/operations/${operation_identifier}/rowset
### _Why are the changes needed?_ closes #1691 /${version}/operations/${operation_identifier}/rowset?fetchorientation=${fetchorientation}&maxrows=${maxrows} mapping: ICLIService#fetchResults desc: get the row set via the given operation identifier method: GET params: none returns: an instance of RowSet ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1692 from simon824/getNextRowSet. Closes #1691 611aafe0 [simon] desc e6712b1b [simon] getNextRowSet Authored-by: simon <zhangshiming@cvte.com> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
hongdongdong
|
e886524c74
|
[KYUUBI #1641] Add Trino client
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> Add trino client to communicate with trino cluster. ### _How was this patch tested?_ - [X] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1642 from hddong/add-trino-runner. Closes #1641 67c2f1f4 [hongdongdong] fix 3e4f7782 [hongdongdong] use testcontainers 3cd4d944 [hongdongdong] [KYUUBI #1641] Add Trino client Authored-by: hongdongdong <hongdongdong@cmss.chinamobile.com> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
yanghua
|
ce7916c3f4
|
[KYUUBI #1647] Implement GetTableTypes operation
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1686 from yanghua/KYUUBI-1647. Closes #1647 c25f5caa [yanghua] retrigger ci 52ae1a8b [yanghua] [KYUUBI #1647] Implement GetTableTypes operation Authored-by: yanghua <yanghua1127@gmail.com> Signed-off-by: Cheng Pan <chengpan@apache.org> |
||
|
xifeng yang
|
628719fd60
|
[KYUUBI #1477] Use KyuubiHadoopUtils.newHadoopConf instead of new Con…
…figuration() <!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> ### _How was this patch tested?_ - [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [x] Add screenshots for manual tests if appropriate - EventLoggingServiceSuite  - HadoopFsDelegationTokenProviderSuite  - KyuubiHadoopUtilsSuite  - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1661 from xifeng/kyuubi-1477. Closes #1477 ad068c6e [xifeng yang] [KYUUBI #1477] Use KyuubiHadoopUtils.newHadoopConf instead of new Configuration() Authored-by: xifeng yang <xifeng.yang@hotmail.com> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
odone
|
2f1f2589cc
|
[KYUUBI #1678] Fixed: The value of hasResultSet is always set to true in Kyuubi Server
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> Fixes #1678 ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> Updating `hasResultSet` state which is determined by the remote service ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1682 from iodone/kyuubi-1678. Closes #1678 8e553850 [odone] comment Authored-by: odone <odone.zhang@gmail.com> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
Paul Lam
|
e0d9458144 |
[KYUUBI #1679] Remove invalid column and field number check
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> ResultSet checks if the column number of data matches the one in metadata by calling iterator.next() to get a sample row, which would make the first row missing in the final result. The bug didn't appear in the tests, because the condition is wrong. Fixed by c0e9072a4c3383562b0fe1938e3bfa1947376754. ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1680 from link3280/bugfix/flink_invliad_resultset. Closes #1679 238809d0 [Paul Lin] Fix first row missing in ResultSet c0e9072a [Paul Lam] Fix ArrayIndexOutOfBoundsException in ResultSet Lead-authored-by: Paul Lam <link3280@gmail.com> Co-authored-by: Paul Lin <paullin3280@gmail.com> Signed-off-by: yanghua <yanghua1127@gmail.com> |
||
|
Cheng Pan
|
e4f5ba7010
|
[KYUUBI #1673] [BUILD] Improve log4j configurations
### _Why are the changes needed?_ - Flink engine print log to console on UT, and use `$FLINK_CONF_DIR/log4j.properties` in production. - Add missing log4j conf `kyuubi-trino-engine` module. - Fix log4j configuration issue of THRIFT ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1673 from pan3793/log. Closes #1673 8abe9d08 [Cheng Pan] Remove flink-sql-engine conf b1726288 [Cheng Pan] Improve log4j configurations Authored-by: Cheng Pan <chengpan@apache.org> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
Simon
|
dc26f2128c
|
[KYUUBI #1568] [FOLLOWUP] Add getSessionEvent in session
### _Why are the changes needed?_ KyuubiSessionEvent.apply will get incorrect info. The first one is from `Session.getSessionEvent`, the second one is `KyuubiSessionEvent.apply`  ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1644 from simon824/1229. Closes #1568 bc5e4b49 [Simon] Merge branch 'apache:master' into 1229 bb316e6a [simon] Add getSessionEvent in session Lead-authored-by: Simon <3656562@qq.com> Co-authored-by: simon <zhangshiming@cvte.com> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
minyk
|
0f8ebb6006
|
[KYUUBI #1675][FEATURE] Redact secret information from SparkSQLEngine log
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> Secret informations on log is dangerous on the production environment. Secret info should be redacted in some way. ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [x] Add screenshots for manual tests if appropriate capture from Spark Driver's STDERR  - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1677 from minyk/kyuubi-1675. Closes #1675 6a390575 [minyk] redact statement in the log Authored-by: minyk <minykreva@gmail.com> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
Wang Zhen
|
47533100f6
|
[KYUUBI #1674] Uncache cached tables when session closed
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> As mentioned in SPARK-29911 , there may also be memory leaks in kyuubi. #1674 ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [X] Add screenshots for manual tests if appropriate  close session:  - [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1676 from wForget/KYUUBI-1674. Closes #1674 e2bec4ec [Wang Zhen] [KYUUBI-1674] Uncache cached tables when session closed Authored-by: Wang Zhen <wangzhen07@qiyi.com> Signed-off-by: ulysses-you <ulyssesyou@apache.org> |
||
|
simon
|
4655cd8515
|
[KYUUBI #1654][Bug] Flaky OperationsResourceSuite
### _Why are the changes needed?_ closes #1654 ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1655 from simon824/flaky. Closes #1654 92548e46 [simon] style 3a748969 [simon] state 3c507da3 [simon] state 4436bb89 [simon] rm e12d483c [simon] flakyut Authored-by: simon <zhangshiming@cvte.com> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
yanghua
|
4e3c718108
|
[KYUUBI #1665][FOLLOWUP] Rename FlinkEngineProcessBuilder to FlinkProcessBuilder to align with SparkProcessBuilder
… <!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1667 from yanghua/KYUUBI-1665-FOLLOWUP. Closes #1665 aa6f6120 [yanghua] retrigger ci 013005c3 [yanghua] [KYUUBI #1665][FOLLOWUP] Rename FlinkEngineProcessBuilder to FlinkProcessBuilder to align with SparkProcessBuilder Authored-by: yanghua <yanghua1127@gmail.com> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
zwangsheng
|
872551d3aa
|
[KYUUBI #1594] SparkProcessBuild submit add --conf spark.driver.host=
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> detail #1594 When Kyuubi On Kubernetes submit spark engine by client mode, executor pod can not connect to driver by kyuubi pod name, which is default set. ### _How was this patch tested?_ - [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1596 from zwangsheng/kyuubi/1594. Closes #1594 d4b41005 [zwangsheng] add comment f7bcffb6 [zwangsheng] fix test name 581eb508 [zwangsheng] fix test name b19a2c21 [zwangsheng] simple & add testsuit 83979935 [zwangsheng] fix 1a9ae9b1 [zwangsheng] fix doc d857eac7 [zwangsheng] Merge branch 'kyuubi/1594' of https://github.com/zwangsheng/incubator-kyuubi into kyuubi/1594 b723f4ed [zwangsheng] retest 8210de56 [zwangsheng] make import normal 92ec9cf4 [zwangsheng] 1594 Authored-by: zwangsheng <2213335496@qq.com> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
yanghua
|
3eaa1d86c8 |
[KYUUBI #1665] Rename FlinkEngineProcessBuilder to FlinkProcessBuilder to align with SparkProcessBuilder
…r to align with SparkProcessBuilder <!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1666 from yanghua/KYUUBI-1665. Closes #1665 569becd3 [yanghua] [KYUUBI #1665] Rename FlinkEngineProcessBuilder to FlinkProcessBuilder to align with SparkProcessBuilder Authored-by: yanghua <yanghua1127@gmail.com> Signed-off-by: yanghua <yanghua1127@gmail.com> |
||
|
shenbing
|
05b22989ef
|
[KYUUBI #1656] Fix dist failed with --spark-provided
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [x] Add screenshots for manual tests if appropriate   - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1664 from shenbinglife/master. Closes #1656 576f387c [shenbing] [KYUUBI #1656] Add flink profile active to maven opts de9c14d0 [shenbing] [KYUUBI #1656] Fix dist failed with --spark-provided Authored-by: shenbing <shenbinglife@163.com> Signed-off-by: Cheng Pan <chengpan@apache.org> |
||
|
Kent Yao
|
d1de9ca529
|
[KYUUBI #1658] Revamp swagger ui dependencies
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> This PR mainly removes the swagger-scala-module which is currently fine to remove. It also introduces transitive deps with `-jakarta` which could introduce jar conflicts like , ```scala Caused by: java.lang.NoClassDefFoundError: jakarta/xml/bind/annotation/XmlElement at io.swagger.v3.core.jackson.SwaggerAnnotationIntrospector.hasRequiredMarker(SwaggerAnnotationIntrospector.java:29) at com.fasterxml.jackson.databind.introspect.AnnotationIntrospectorPair.hasRequiredMarker(AnnotationIntrospectorPair.java:319) at com.fasterxml.jackson.databind.introspect.AnnotationIntrospectorPair.hasRequiredMarker(AnnotationIntrospectorPair.java:319) at com.fasterxml.jackson.databind.introspect.AnnotationIntrospectorPair.hasRequiredMarker(AnnotationIntrospectorPair.java:319) at com.fasterxml.jackson.databind.introspect.POJOPropertyBuilder.getMetadata(POJOPropertyBuilder.java:229) ``` ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1658 from yaooqinn/dep. Closes #1658 d5ab0765 [Kent Yao] Merge branch 'master' into dep d1ddd5fd [Kent Yao] Revamp swagger ui dependencies ecfcc92a [Kent Yao] Revamp swagger ui dependencies 59196b94 [Kent Yao] Revamp swagger ui dependencies d7ca6a5f [Kent Yao] Revamp swagger ui dependencies 215b1b53 [Kent Yao] Revamp swagger ui dependencies Authored-by: Kent Yao <yao@apache.org> Signed-off-by: ulysses-you <ulyssesyou@apache.org> |
||
|
Kent Yao
|
6a9550938e
|
[KYUUBI #1662] Turn up maven options in mvn script
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1662 from yaooqinn/mvn. Closes #1662 4d310988 [Kent Yao] Turn up maven options in mvn script db153080 [Kent Yao] Turn up maven options in mvn script Authored-by: Kent Yao <yao@apache.org> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
xifeng yang
|
8e9e8c3c7e
|
[KYUUBI #1597] fix row value in Query Execution should be queryExecution
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> row value in Query Execution should be queryExecution ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [x] Add screenshots for manual tests if appropriate  - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1657 from xifeng/kyuubi-1597. Closes #1597 cf9f9aca [xifeng yang] [KYUUBI #1597] fix row value in Query Execution should be queryExecution Authored-by: xifeng yang <xifeng.yang@hotmail.com> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
wForget
|
92163dda42
|
[KYUUBI #1649] Determine the yarn command in stop-application.sh.
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> When HADOOP_HOME is not set, the stop-application.sh script cannot kill the yarn task even if the yarn command exists. #1649 ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [X] Add screenshots for manual tests if appropriate  - [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1650 from wForget/KYUUBI-1649. Closes #1649 aa8c08d8 [wForget] change error msg 4ec523be [wForget] [KYUUBI-1649] Determine the yarn command in stop-application.sh. Authored-by: wForget <643348094@qq.com> Signed-off-by: ulysses-you <ulyssesyou@apache.org> |
||
|
minyk
|
7d68aa07ff
|
[KYUUBI #1634][FEATURE] Redact secret information from SparkSQLEngine UI
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> SparkSQLEngine's Kyuubi UI tab shows SQL statements. In some cases, these informations are secret information like S3 access/secret key. Spark SQL UI tab already redact with `spark.redaction.string.regex` configuration. Kyuubi also should redact secret informations with this property. ### _How was this patch tested?_ - [X] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [X] Add screenshots for manual tests if appropriate with below property and query: spark-defaults.conf: ``` spark.redaction.string.regex (?i)url|access.*|secret.*|password ``` query: ``` SET fs.s3a.access.key=testkey ```  - [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1640 from minyk/redact-ui. Closes #1634 4051522a [minyk] set job description with setJobGroup() 3a8e0e91 [minyk] use redactedStatement c84778e3 [minyk] move to SparkOperation 97a4fb4a [minyk] fix code formatting 79fe52d3 [minyk] redact UI with `spark.redaction.string.regex` Authored-by: minyk <minykreva@gmail.com> Signed-off-by: Cheng Pan <chengpan@apache.org> |
||
|
Kent Yao
|
c4fd287c8b
|
[KYUUBI #1639] Make ApiRequestContext provide KyuubiRestFrontendService directly
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> Make full use of KyuubiRestFrontendService without hacking, e.g. for the swagger UI base. We can do further improvement based on this PR. ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1639 from yaooqinn/ApiRequestContext. Closes #1639 3c0c7061 [Kent Yao] Make ApiRequestContext provides KyuubiRestFrontendService directly 10506c2c [Kent Yao] Make ApiRequestContext provides KyuubiRestFrontendService directly 1482bd78 [Kent Yao] Make ApiRequestContext provides KyuubiRestFrontendService directly 01a0b023 [Kent Yao] Make ApiRequestContext provides KyuubiRestFrontendService directly d3119fa4 [Kent Yao] Make ApiRequestContext provides KyuubiRestFrontendService directly 0f45258c [Kent Yao] Make ApiRequestContext provides KyuubiRestFrontendService directly Authored-by: Kent Yao <yao@apache.org> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
Cheng Pan
|
cb8721fef1
|
[KYUUBI #1629] Flink backend implementation
### _Why are the changes needed?_ This PR covers #1619. Overall, this PR contains the following changs, 1. change `build/dist` script to support flink sql engine 2. enhance `externals/flink-sql-engine/pom.xml` to support create a shaded jar 3. simplify `externals/kyuubi-flink-sql-engine/bin/flink-sql-engine.sh` 4. introduce `FlinkSQLEngine`(flink sql engine entrypoint) and `FlinkProcessBuilder`(kyuubi server launcher) 5. add ut in kyuubi server side After this PR, we can run the basic query e.g. `select now()` from beeline and get result, and Kyuubi Server can auto launch flink engine if there is no proper one. The Flink engine also supports other engine share levels defined in Kyuubi. The implementation based on Flink 1.14 codebase. ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1629 from pan3793/flink-backend. Closes #1629 b7e5f0e3 [Cheng Pan] revert tgz name change a4496c37 [Cheng Pan] Fix reflection 3b4e86a1 [Cheng Pan] deps 68efa42c [Cheng Pan] log 8a9e37f3 [Cheng Pan] nit 10fb2bc3 [Cheng Pan] CI c1560fdb [Cheng Pan] nit 303e2f1e [Cheng Pan] Restore log conf d84720b1 [Cheng Pan] SessionContext b258d81a [Cheng Pan] cleanup 16edd528 [Cheng Pan] Cleanup 9ae54557 [Cheng Pan] Fix CI 25b6b57d [Cheng Pan] hadoop-client-api c12b5ca4 [Cheng Pan] Server UT pass 502d3f08 [Cheng Pan] pass dac4323b [yanghua] Laungh local flink engine container successfully Lead-authored-by: Cheng Pan <chengpan@apache.org> Co-authored-by: yanghua <yanghua1127@gmail.com> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
Kent Yao
|
e1587eeaf4
|
[KYUUBI #1631] Migrating existing rest fe tests to real cases
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> In this PR, we target the existing UTs from a noop server to a real shared Kyuubi server. ### _How was this patch tested?_ - [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1631 from yaooqinn/resttest. Closes #1631 2726ab44 [Kent Yao] address comments c0da8090 [Kent Yao] ci 5d12f70f [Kent Yao] address comments 80e14044 [Kent Yao] Migrating existing rest fe test to real cases a31ab506 [Kent Yao] Migrating existing rest fe test to real cases 0c1feb45 [Kent Yao] Migrating existing rest fe test to real cases fbd9769e [Kent Yao] Migrating existing rest fe test to real cases 75cca5ff [Kent Yao] Migrating existing rest fe test to real cases 119712e8 [Kent Yao] Migrating existing rest fe test to real cases 8196e4df [Kent Yao] Migrating existing rest fe test to real cases Authored-by: Kent Yao <yao@apache.org> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
rayliu
|
2af105a417 |
[KYUUBI #1556] Upgrade swagger-ui version from 4.1.0 to 4.1.3
### _Why are the changes needed?_ Close https://github.com/apache/incubator-kyuubi/issues/1612 ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1630 from ldzhjn/1612. Closes #1556 71fe05bc [rayliu] Upgrade swagger-ui version from 4.1.0 to 4.1.3 Authored-by: rayliu <rayliu@cisco.com> Signed-off-by: yanghua <yanghua1127@gmail.com> |
||
|
Fu Chen
|
21f10d752d
|
[KYUUBI #1614] Fix hardcoded version in OpenAPIDefinition
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> close #1614 curl http://0.0.0.0:10099/openapi.json ```json { "openapi" : "3.0.1", "info" : { "title" : "Apache Kyuubi REST API Documentation", "description" : "Apache Kyuubi REST API Documentation", "contact" : { "name" : "Apache Kyuubi Community", "url" : "https://kyuubi.apache.org/issue_tracking.html", "email" : "devkyuubi.apache.org" }, "license" : { "name" : "Apache 2.0", "url" : "https://www.apache.org/licenses/LICENSE-2.0.html" }, "version" : "1.5.0-SNAPSHOT" }, "tags" : [ { "name" : "Session" }, { "name" : "Operation" } ], "paths" : { "/api/v1/operations/{operationHandle}" : { "put" : { "tags" : [ "Operation" ], "operationId" : "applyOpAction", "parameters" : [ { "name" : "operationHandle", "in" : "path", "required" : true, "schema" : { "type" : "string" } } ], "requestBody" : { "content" : { "*/*" : { "schema" : { "$ref" : "#/components/schemas/OpActionRequest" } } } }, "responses" : { "200" : { "description" : "apply an action for an operation", "content" : { "application/json" : { } } } } } }, "/api/v1/operations/{operationHandle}/event" : { "get" : { "tags" : [ "Operation" ], "operationId" : "getOperationEvent", "parameters" : [ { "name" : "operationHandle", "in" : "path", "required" : true, "schema" : { "type" : "string" } } ], "responses" : { "200" : { "description" : "Get an operation event", "content" : { "application/json" : { } } } } } }, "/api/v1/operations/{operationHandle}/resultsetmetadata" : { "get" : { "tags" : [ "Operation" ], "operationId" : "getResultSetMetadata", "parameters" : [ { "name" : "operationHandle", "in" : "path", "required" : true, "schema" : { "type" : "string" } } ], "responses" : { "200" : { "description" : "get result set metadata", "content" : { "application/json" : { } } } } } }, "/api/v1/ping" : { "get" : { "operationId" : "ping", "responses" : { "default" : { "description" : "default response", "content" : { "text/plain" : { "schema" : { "type" : "string" } } } } } } }, "/api/v1/sessions/{sessionHandle}/operations/{operationHandle}" : { "delete" : { "tags" : [ "Session" ], "operationId" : "closeOperation", "parameters" : [ { "name" : "sessionHandle", "in" : "path", "required" : true, "schema" : { "type" : "string" } }, { "name" : "operationHandle", "in" : "path", "required" : true, "schema" : { "type" : "string" } } ], "responses" : { "200" : { "description" : "Close an operation", "content" : { "application/json" : { } } } } } }, "/api/v1/sessions/{sessionHandle}" : { "get" : { "tags" : [ "Session" ], "operationId" : "sessionInfo", "parameters" : [ { "name" : "sessionHandle", "in" : "path", "required" : true, "schema" : { "type" : "string" } } ], "responses" : { "200" : { "description" : "get a session event via session handle identifier", "content" : { "application/json" : { } } } } }, "delete" : { "tags" : [ "Session" ], "operationId" : "closeSession", "parameters" : [ { "name" : "sessionHandle", "in" : "path", "required" : true, "schema" : { "type" : "string" } } ], "responses" : { "200" : { "description" : "Close a session", "content" : { "application/json" : { } } } } } }, "/api/v1/sessions/execPool/statistic" : { "get" : { "tags" : [ "Session" ], "operationId" : "execPoolStatistic", "responses" : { "200" : { "description" : "Get statistic info of background executors", "content" : { "application/json" : { } } } } } }, "/api/v1/sessions/{sessionHandle}/operations/statement" : { "post" : { "tags" : [ "Session" ], "operationId" : "executeStatement", "parameters" : [ { "name" : "sessionHandle", "in" : "path", "required" : true, "schema" : { "type" : "string" } } ], "requestBody" : { "content" : { "*/*" : { "schema" : { "$ref" : "#/components/schemas/StatementRequest" } } } }, "responses" : { "200" : { "description" : "Create an operation with EXECUTE_STATEMENT type", "content" : { "application/json" : { } } } } } }, "/api/v1/sessions/{sessionHandle}/operations/catalogs" : { "post" : { "tags" : [ "Session" ], "operationId" : "getCatalogs", "parameters" : [ { "name" : "sessionHandle", "in" : "path", "required" : true, "schema" : { "type" : "string" } } ], "responses" : { "200" : { "description" : "Create an operation with GET_CATALOGS type", "content" : { "application/json" : { } } } } } }, "/api/v1/sessions/{sessionHandle}/operations/columns" : { "post" : { "tags" : [ "Session" ], "operationId" : "getColumns", "parameters" : [ { "name" : "sessionHandle", "in" : "path", "required" : true, "schema" : { "type" : "string" } } ], "requestBody" : { "content" : { "*/*" : { "schema" : { "$ref" : "#/components/schemas/GetColumnsRequest" } } } }, "responses" : { "200" : { "description" : "Create an operation with GET_COLUMNS type", "content" : { "application/json" : { } } } } } }, "/api/v1/sessions/{sessionHandle}/operations/functions" : { "post" : { "tags" : [ "Session" ], "operationId" : "getFunctions", "parameters" : [ { "name" : "sessionHandle", "in" : "path", "required" : true, "schema" : { "type" : "string" } } ], "requestBody" : { "content" : { "*/*" : { "schema" : { "$ref" : "#/components/schemas/GetFunctionsRequest" } } } }, "responses" : { "200" : { "description" : "Create an operation with GET_FUNCTIONS type", "content" : { "application/json" : { } } } } } }, "/api/v1/sessions/{sessionHandle}/info/{infoType}" : { "get" : { "tags" : [ "Session" ], "operationId" : "getInfo", "parameters" : [ { "name" : "sessionHandle", "in" : "path", "required" : true, "schema" : { "type" : "string" } }, { "name" : "infoType", "in" : "path", "required" : true, "schema" : { "type" : "integer", "format" : "int32" } } ], "responses" : { "200" : { "description" : "get a information detail via session handle identifier and a specific information type", "content" : { "application/json" : { } } } } } }, "/api/v1/sessions/{sessionHandle}/operations/schemas" : { "post" : { "tags" : [ "Session" ], "operationId" : "getSchemas", "parameters" : [ { "name" : "sessionHandle", "in" : "path", "required" : true, "schema" : { "type" : "string" } } ], "requestBody" : { "content" : { "*/*" : { "schema" : { "$ref" : "#/components/schemas/GetSchemasRequest" } } } }, "responses" : { "200" : { "description" : "Create an operation with GET_SCHEMAS type", "content" : { "application/json" : { } } } } } }, "/api/v1/sessions/{sessionHandle}/operations/tableTypes" : { "post" : { "tags" : [ "Session" ], "operationId" : "getTableTypes", "parameters" : [ { "name" : "sessionHandle", "in" : "path", "required" : true, "schema" : { "type" : "string" } } ], "responses" : { "200" : { "description" : "Create an operation with GET_TABLE_TYPES type", "content" : { "application/json" : { } } } } } }, "/api/v1/sessions/{sessionHandle}/operations/tables" : { "post" : { "tags" : [ "Session" ], "operationId" : "getTables", "parameters" : [ { "name" : "sessionHandle", "in" : "path", "required" : true, "schema" : { "type" : "string" } } ], "requestBody" : { "content" : { "*/*" : { "schema" : { "$ref" : "#/components/schemas/GetTablesRequest" } } } }, "responses" : { "200" : { "description" : "Create an operation with GET_TABLES type", "content" : { "application/json" : { } } } } } }, "/api/v1/sessions/{sessionHandle}/operations/typeInfo" : { "post" : { "tags" : [ "Session" ], "operationId" : "getTypeInfo", "parameters" : [ { "name" : "sessionHandle", "in" : "path", "required" : true, "schema" : { "type" : "string" } } ], "responses" : { "200" : { "description" : "Create an operation with GET_TYPE_INFO type", "content" : { "application/json" : { } } } } } }, "/api/v1/sessions" : { "get" : { "tags" : [ "Session" ], "operationId" : "sessionInfoList", "responses" : { "200" : { "description" : "get all the session list hosted in SessionManager", "content" : { "application/json" : { } } } } }, "post" : { "tags" : [ "Session" ], "operationId" : "openSession", "requestBody" : { "content" : { "application/json" : { "schema" : { "$ref" : "#/components/schemas/SessionOpenRequest" } } } }, "responses" : { "200" : { "description" : "Open(create) a session", "content" : { "application/json" : { } } } } } }, "/api/v1/sessions/count" : { "get" : { "tags" : [ "Session" ], "operationId" : "sessionCount", "responses" : { "200" : { "description" : "Get the current open session count", "content" : { "application/json" : { } } } } } }, "/api/v1/swagger-ui" : { "get" : { "operationId" : "swaggerUi", "responses" : { "default" : { "description" : "default response", "content" : { "text/html" : { } } } } } }, "/api/v1/exception" : { "get" : { "operationId" : "test", "responses" : { "default" : { "description" : "default response", "content" : { "text/plain" : { } } } } } }, "/application.wadl/{path}" : { "get" : { "operationId" : "getExternalGrammar", "parameters" : [ { "name" : "path", "in" : "path", "required" : true, "schema" : { "type" : "string" } } ], "responses" : { "default" : { "description" : "default response", "content" : { "application/xml" : { } } } } } }, "/application.wadl" : { "get" : { "operationId" : "getWadl", "responses" : { "default" : { "description" : "default response", "content" : { "application/vnd.sun.wadl+xml" : { }, "application/xml" : { } } } } } } }, "components" : { "schemas" : { "OpActionRequest" : { "required" : [ "action" ], "type" : "object", "properties" : { "action" : { "type" : "string" } } }, "KyuubiOperationEvent" : { "required" : [ "eventTime", "shouldRunAsync", "statementId" ], "type" : "object", "properties" : { "statementId" : { "type" : "string" }, "remoteId" : { "type" : "string" }, "statement" : { "type" : "string" }, "shouldRunAsync" : { "type" : "boolean" }, "state" : { "type" : "string" }, "eventTime" : { "type" : "integer", "format" : "int64" }, "createTime" : { "type" : "integer", "format" : "int64" }, "startTime" : { "type" : "integer", "format" : "int64" }, "completeTime" : { "type" : "integer", "format" : "int64" }, "exception" : { "type" : "object", "properties" : { "cause" : { "required" : [ "stackTrace", "suppressed" ], "type" : "object", "properties" : { "stackTrace" : { "type" : "array", "items" : { "required" : [ "lineNumber" ], "type" : "object", "properties" : { "methodName" : { "type" : "string" }, "fileName" : { "type" : "string" }, "lineNumber" : { "type" : "integer", "format" : "int32" }, "className" : { "type" : "string" }, "nativeMethod" : { "type" : "boolean" } } } }, "message" : { "type" : "string" }, "localizedMessage" : { "type" : "string" }, "suppressed" : { "type" : "array", "items" : { "type" : "object", "properties" : { "stackTrace" : { "type" : "array", "items" : { "required" : [ "lineNumber" ], "type" : "object", "properties" : { "methodName" : { "type" : "string" }, "fileName" : { "type" : "string" }, "lineNumber" : { "type" : "integer", "format" : "int32" }, "className" : { "type" : "string" }, "nativeMethod" : { "type" : "boolean" } } } }, "message" : { "type" : "string" }, "localizedMessage" : { "type" : "string" } } } } } }, "stackTrace" : { "type" : "array", "items" : { "required" : [ "lineNumber" ], "type" : "object", "properties" : { "methodName" : { "type" : "string" }, "fileName" : { "type" : "string" }, "lineNumber" : { "type" : "integer", "format" : "int32" }, "className" : { "type" : "string" }, "nativeMethod" : { "type" : "boolean" } } } }, "message" : { "type" : "string" }, "localizedMessage" : { "type" : "string" }, "suppressed" : { "type" : "array", "items" : { "type" : "object", "properties" : { "stackTrace" : { "type" : "array", "items" : { "required" : [ "lineNumber" ], "type" : "object", "properties" : { "methodName" : { "type" : "string" }, "fileName" : { "type" : "string" }, "lineNumber" : { "type" : "integer", "format" : "int32" }, "className" : { "type" : "string" }, "nativeMethod" : { "type" : "boolean" } } } }, "message" : { "type" : "string" }, "localizedMessage" : { "type" : "string" } } } } } }, "sessionId" : { "type" : "string" }, "sessionUser" : { "type" : "string" }, "eventType" : { "type" : "string" } } }, "ColumnDesc" : { "required" : [ "columnIndex", "columnName" ], "type" : "object", "properties" : { "columnName" : { "type" : "string" }, "dataType" : { "type" : "string" }, "columnIndex" : { "type" : "integer", "format" : "int32" }, "precision" : { "type" : "integer", "format" : "int32" }, "scale" : { "type" : "integer", "format" : "int32" }, "comment" : { "type" : "string" } } }, "ResultSetMetaData" : { "required" : [ "columns" ], "type" : "object", "properties" : { "columns" : { "type" : "array", "items" : { "$ref" : "#/components/schemas/ColumnDesc" } } } }, "SeqColumnDesc" : { "type" : "array", "items" : { "$ref" : "#/components/schemas/ColumnDesc" } }, "HandleIdentifier" : { "required" : [ "publicId" ], "type" : "object", "properties" : { "publicId" : { "type" : "string", "format" : "uuid" }, "secretId" : { "type" : "string", "format" : "uuid" } } }, "OperationHandle" : { "required" : [ "hasResultSet", "identifier", "protocol", "typ" ], "type" : "object", "properties" : { "identifier" : { "$ref" : "#/components/schemas/HandleIdentifier" }, "typ" : { "$ref" : "#/components/schemas/Value" }, "protocol" : { "type" : "string", "enum" : [ "HIVE_CLI_SERVICE_PROTOCOL_V1", "HIVE_CLI_SERVICE_PROTOCOL_V2", "HIVE_CLI_SERVICE_PROTOCOL_V3", "HIVE_CLI_SERVICE_PROTOCOL_V4", "HIVE_CLI_SERVICE_PROTOCOL_V5", "HIVE_CLI_SERVICE_PROTOCOL_V6", "HIVE_CLI_SERVICE_PROTOCOL_V7", "HIVE_CLI_SERVICE_PROTOCOL_V8", "HIVE_CLI_SERVICE_PROTOCOL_V9", "HIVE_CLI_SERVICE_PROTOCOL_V10" ] }, "hasResultSet" : { "type" : "boolean", "writeOnly" : true } } }, "Value" : { "type" : "object" }, "ExecPoolStatistic" : { "required" : [ "execPoolSize" ], "type" : "object", "properties" : { "execPoolSize" : { "type" : "integer", "format" : "int32" }, "execPoolActiveCount" : { "type" : "integer", "format" : "int32" } } }, "StatementRequest" : { "required" : [ "queryTimeout", "runAsync", "statement" ], "type" : "object", "properties" : { "statement" : { "type" : "string" }, "runAsync" : { "type" : "boolean" }, "queryTimeout" : { "type" : "integer", "format" : "int64" } } }, "GetColumnsRequest" : { "required" : [ "catalogName" ], "type" : "object", "properties" : { "catalogName" : { "type" : "string" }, "schemaName" : { "type" : "string" }, "tableName" : { "type" : "string" }, "columnName" : { "type" : "string" } } }, "GetFunctionsRequest" : { "required" : [ "catalogName" ], "type" : "object", "properties" : { "catalogName" : { "type" : "string" }, "schemaName" : { "type" : "string" }, "functionName" : { "type" : "string" } } }, "InfoDetail" : { "required" : [ "infoType" ], "type" : "object", "properties" : { "infoType" : { "type" : "string" }, "infoValue" : { "type" : "string" } } }, "GetSchemasRequest" : { "required" : [ "catalogName" ], "type" : "object", "properties" : { "catalogName" : { "type" : "string" }, "schemaName" : { "type" : "string" } } }, "GetTablesRequest" : { "required" : [ "catalogName", "tableTypes" ], "type" : "object", "properties" : { "catalogName" : { "type" : "string" }, "schemaName" : { "type" : "string" }, "tableName" : { "type" : "string" }, "tableTypes" : { "type" : "array", "items" : { "type" : "string" } } } }, "SessionHandle" : { "required" : [ "identifier", "protocol" ], "type" : "object", "properties" : { "identifier" : { "$ref" : "#/components/schemas/HandleIdentifier" }, "protocol" : { "type" : "string", "enum" : [ "HIVE_CLI_SERVICE_PROTOCOL_V1", "HIVE_CLI_SERVICE_PROTOCOL_V2", "HIVE_CLI_SERVICE_PROTOCOL_V3", "HIVE_CLI_SERVICE_PROTOCOL_V4", "HIVE_CLI_SERVICE_PROTOCOL_V5", "HIVE_CLI_SERVICE_PROTOCOL_V6", "HIVE_CLI_SERVICE_PROTOCOL_V7", "HIVE_CLI_SERVICE_PROTOCOL_V8", "HIVE_CLI_SERVICE_PROTOCOL_V9", "HIVE_CLI_SERVICE_PROTOCOL_V10" ] } } }, "MapStringString" : { "type" : "object", "additionalProperties" : { "type" : "string" } }, "SessionOpenRequest" : { "required" : [ "configs", "protocolVersion", "user" ], "type" : "object", "properties" : { "protocolVersion" : { "type" : "integer", "format" : "int32" }, "user" : { "type" : "string" }, "password" : { "type" : "string" }, "ipAddr" : { "type" : "string" }, "configs" : { "type" : "object", "additionalProperties" : { "type" : "string" } } } }, "SessionOpenCount" : { "required" : [ "openSessionCount" ], "type" : "object", "properties" : { "openSessionCount" : { "type" : "integer", "format" : "int32" } } }, "KyuubiSessionEvent" : { "required" : [ "clientVersion", "conf", "sessionName", "startTime" ], "type" : "object", "properties" : { "sessionName" : { "type" : "string" }, "user" : { "type" : "string" }, "clientIP" : { "type" : "string" }, "serverIP" : { "type" : "string" }, "conf" : { "type" : "object", "additionalProperties" : { "type" : "string" } }, "startTime" : { "type" : "integer", "format" : "int64" }, "sessionId" : { "type" : "string" }, "remoteSessionId" : { "type" : "string" }, "clientVersion" : { "type" : "integer", "format" : "int32" }, "openedTime" : { "type" : "integer", "format" : "int64" }, "endTime" : { "type" : "integer", "format" : "int64" }, "totalOperations" : { "type" : "integer", "format" : "int32" }, "eventType" : { "type" : "string" } } }, "SeqSessionOverview" : { "type" : "array", "items" : { "$ref" : "#/components/schemas/SessionOverview" } }, "SessionList" : { "required" : [ "sessionList" ], "type" : "object", "properties" : { "sessionList" : { "type" : "array", "items" : { "$ref" : "#/components/schemas/SessionOverview" } } } }, "SessionOverview" : { "required" : [ "createTime", "user" ], "type" : "object", "properties" : { "user" : { "type" : "string" }, "ipAddr" : { "type" : "string" }, "createTime" : { "type" : "integer", "format" : "int64" }, "sessionHandle" : { "$ref" : "#/components/schemas/SessionHandle" } } } } } } ``` curl http://0.0.0.0:10099/openapi.yaml ```yaml openapi: 3.0.1 info: title: Apache Kyuubi REST API Documentation description: Apache Kyuubi REST API Documentation contact: name: Apache Kyuubi Community url: https://kyuubi.apache.org/issue_tracking.html email: devkyuubi.apache.org license: name: Apache 2.0 url: https://www.apache.org/licenses/LICENSE-2.0.html version: 1.5.0-SNAPSHOT tags: - name: Session - name: Operation paths: /api/v1/operations/{operationHandle}: put: tags: - Operation operationId: applyOpAction parameters: - name: operationHandle in: path required: true schema: type: string requestBody: content: '*/*': schema: $ref: '#/components/schemas/OpActionRequest' responses: "200": description: apply an action for an operation content: application/json: {} /api/v1/operations/{operationHandle}/event: get: tags: - Operation operationId: getOperationEvent parameters: - name: operationHandle in: path required: true schema: type: string responses: "200": description: Get an operation event content: application/json: {} /api/v1/operations/{operationHandle}/resultsetmetadata: get: tags: - Operation operationId: getResultSetMetadata parameters: - name: operationHandle in: path required: true schema: type: string responses: "200": description: get result set metadata content: application/json: {} /api/v1/ping: get: operationId: ping responses: default: description: default response content: text/plain: schema: type: string /api/v1/sessions/{sessionHandle}/operations/{operationHandle}: delete: tags: - Session operationId: closeOperation parameters: - name: sessionHandle in: path required: true schema: type: string - name: operationHandle in: path required: true schema: type: string responses: "200": description: Close an operation content: application/json: {} /api/v1/sessions/{sessionHandle}: get: tags: - Session operationId: sessionInfo parameters: - name: sessionHandle in: path required: true schema: type: string responses: "200": description: get a session event via session handle identifier content: application/json: {} delete: tags: - Session operationId: closeSession parameters: - name: sessionHandle in: path required: true schema: type: string responses: "200": description: Close a session content: application/json: {} /api/v1/sessions/execPool/statistic: get: tags: - Session operationId: execPoolStatistic responses: "200": description: Get statistic info of background executors content: application/json: {} /api/v1/sessions/{sessionHandle}/operations/statement: post: tags: - Session operationId: executeStatement parameters: - name: sessionHandle in: path required: true schema: type: string requestBody: content: '*/*': schema: $ref: '#/components/schemas/StatementRequest' responses: "200": description: Create an operation with EXECUTE_STATEMENT type content: application/json: {} /api/v1/sessions/{sessionHandle}/operations/catalogs: post: tags: - Session operationId: getCatalogs parameters: - name: sessionHandle in: path required: true schema: type: string responses: "200": description: Create an operation with GET_CATALOGS type content: application/json: {} /api/v1/sessions/{sessionHandle}/operations/columns: post: tags: - Session operationId: getColumns parameters: - name: sessionHandle in: path required: true schema: type: string requestBody: content: '*/*': schema: $ref: '#/components/schemas/GetColumnsRequest' responses: "200": description: Create an operation with GET_COLUMNS type content: application/json: {} /api/v1/sessions/{sessionHandle}/operations/functions: post: tags: - Session operationId: getFunctions parameters: - name: sessionHandle in: path required: true schema: type: string requestBody: content: '*/*': schema: $ref: '#/components/schemas/GetFunctionsRequest' responses: "200": description: Create an operation with GET_FUNCTIONS type content: application/json: {} /api/v1/sessions/{sessionHandle}/info/{infoType}: get: tags: - Session operationId: getInfo parameters: - name: sessionHandle in: path required: true schema: type: string - name: infoType in: path required: true schema: type: integer format: int32 responses: "200": description: get a information detail via session handle identifier and a specific information type content: application/json: {} /api/v1/sessions/{sessionHandle}/operations/schemas: post: tags: - Session operationId: getSchemas parameters: - name: sessionHandle in: path required: true schema: type: string requestBody: content: '*/*': schema: $ref: '#/components/schemas/GetSchemasRequest' responses: "200": description: Create an operation with GET_SCHEMAS type content: application/json: {} /api/v1/sessions/{sessionHandle}/operations/tableTypes: post: tags: - Session operationId: getTableTypes parameters: - name: sessionHandle in: path required: true schema: type: string responses: "200": description: Create an operation with GET_TABLE_TYPES type content: application/json: {} /api/v1/sessions/{sessionHandle}/operations/tables: post: tags: - Session operationId: getTables parameters: - name: sessionHandle in: path required: true schema: type: string requestBody: content: '*/*': schema: $ref: '#/components/schemas/GetTablesRequest' responses: "200": description: Create an operation with GET_TABLES type content: application/json: {} /api/v1/sessions/{sessionHandle}/operations/typeInfo: post: tags: - Session operationId: getTypeInfo parameters: - name: sessionHandle in: path required: true schema: type: string responses: "200": description: Create an operation with GET_TYPE_INFO type content: application/json: {} /api/v1/sessions: get: tags: - Session operationId: sessionInfoList responses: "200": description: get all the session list hosted in SessionManager content: application/json: {} post: tags: - Session operationId: openSession requestBody: content: application/json: schema: $ref: '#/components/schemas/SessionOpenRequest' responses: "200": description: Open(create) a session content: application/json: {} /api/v1/sessions/count: get: tags: - Session operationId: sessionCount responses: "200": description: Get the current open session count content: application/json: {} /api/v1/swagger-ui: get: operationId: swaggerUi responses: default: description: default response content: text/html: {} /api/v1/exception: get: operationId: test responses: default: description: default response content: text/plain: {} /application.wadl/{path}: get: operationId: getExternalGrammar parameters: - name: path in: path required: true schema: type: string responses: default: description: default response content: application/xml: {} /application.wadl: get: operationId: getWadl responses: default: description: default response content: application/vnd.sun.wadl+xml: {} application/xml: {} components: schemas: OpActionRequest: required: - action type: object properties: action: type: string KyuubiOperationEvent: required: - eventTime - shouldRunAsync - statementId type: object properties: statementId: type: string remoteId: type: string statement: type: string shouldRunAsync: type: boolean state: type: string eventTime: type: integer format: int64 createTime: type: integer format: int64 startTime: type: integer format: int64 completeTime: type: integer format: int64 exception: type: object properties: cause: required: - stackTrace - suppressed type: object properties: stackTrace: type: array items: required: - lineNumber type: object properties: methodName: type: string fileName: type: string lineNumber: type: integer format: int32 className: type: string nativeMethod: type: boolean message: type: string localizedMessage: type: string suppressed: type: array items: type: object properties: stackTrace: type: array items: required: - lineNumber type: object properties: methodName: type: string fileName: type: string lineNumber: type: integer format: int32 className: type: string nativeMethod: type: boolean message: type: string localizedMessage: type: string stackTrace: type: array items: required: - lineNumber type: object properties: methodName: type: string fileName: type: string lineNumber: type: integer format: int32 className: type: string nativeMethod: type: boolean message: type: string localizedMessage: type: string suppressed: type: array items: type: object properties: stackTrace: type: array items: required: - lineNumber type: object properties: methodName: type: string fileName: type: string lineNumber: type: integer format: int32 className: type: string nativeMethod: type: boolean message: type: string localizedMessage: type: string sessionId: type: string sessionUser: type: string eventType: type: string ColumnDesc: required: - columnIndex - columnName type: object properties: columnName: type: string dataType: type: string columnIndex: type: integer format: int32 precision: type: integer format: int32 scale: type: integer format: int32 comment: type: string ResultSetMetaData: required: - columns type: object properties: columns: type: array items: $ref: '#/components/schemas/ColumnDesc' SeqColumnDesc: type: array items: $ref: '#/components/schemas/ColumnDesc' HandleIdentifier: required: - publicId type: object properties: publicId: type: string format: uuid secretId: type: string format: uuid OperationHandle: required: - hasResultSet - identifier - protocol - typ type: object properties: identifier: $ref: '#/components/schemas/HandleIdentifier' typ: $ref: '#/components/schemas/Value' protocol: type: string enum: - HIVE_CLI_SERVICE_PROTOCOL_V1 - HIVE_CLI_SERVICE_PROTOCOL_V2 - HIVE_CLI_SERVICE_PROTOCOL_V3 - HIVE_CLI_SERVICE_PROTOCOL_V4 - HIVE_CLI_SERVICE_PROTOCOL_V5 - HIVE_CLI_SERVICE_PROTOCOL_V6 - HIVE_CLI_SERVICE_PROTOCOL_V7 - HIVE_CLI_SERVICE_PROTOCOL_V8 - HIVE_CLI_SERVICE_PROTOCOL_V9 - HIVE_CLI_SERVICE_PROTOCOL_V10 hasResultSet: type: boolean writeOnly: true Value: type: object ExecPoolStatistic: required: - execPoolSize type: object properties: execPoolSize: type: integer format: int32 execPoolActiveCount: type: integer format: int32 StatementRequest: required: - queryTimeout - runAsync - statement type: object properties: statement: type: string runAsync: type: boolean queryTimeout: type: integer format: int64 GetColumnsRequest: required: - catalogName type: object properties: catalogName: type: string schemaName: type: string tableName: type: string columnName: type: string GetFunctionsRequest: required: - catalogName type: object properties: catalogName: type: string schemaName: type: string functionName: type: string InfoDetail: required: - infoType type: object properties: infoType: type: string infoValue: type: string GetSchemasRequest: required: - catalogName type: object properties: catalogName: type: string schemaName: type: string GetTablesRequest: required: - catalogName - tableTypes type: object properties: catalogName: type: string schemaName: type: string tableName: type: string tableTypes: type: array items: type: string SessionHandle: required: - identifier - protocol type: object properties: identifier: $ref: '#/components/schemas/HandleIdentifier' protocol: type: string enum: - HIVE_CLI_SERVICE_PROTOCOL_V1 - HIVE_CLI_SERVICE_PROTOCOL_V2 - HIVE_CLI_SERVICE_PROTOCOL_V3 - HIVE_CLI_SERVICE_PROTOCOL_V4 - HIVE_CLI_SERVICE_PROTOCOL_V5 - HIVE_CLI_SERVICE_PROTOCOL_V6 - HIVE_CLI_SERVICE_PROTOCOL_V7 - HIVE_CLI_SERVICE_PROTOCOL_V8 - HIVE_CLI_SERVICE_PROTOCOL_V9 - HIVE_CLI_SERVICE_PROTOCOL_V10 MapStringString: type: object additionalProperties: type: string SessionOpenRequest: required: - configs - protocolVersion - user type: object properties: protocolVersion: type: integer format: int32 user: type: string password: type: string ipAddr: type: string configs: type: object additionalProperties: type: string SessionOpenCount: required: - openSessionCount type: object properties: openSessionCount: type: integer format: int32 KyuubiSessionEvent: required: - clientVersion - conf - sessionName - startTime type: object properties: sessionName: type: string user: type: string clientIP: type: string serverIP: type: string conf: type: object additionalProperties: type: string startTime: type: integer format: int64 sessionId: type: string remoteSessionId: type: string clientVersion: type: integer format: int32 openedTime: type: integer format: int64 endTime: type: integer format: int64 totalOperations: type: integer format: int32 eventType: type: string SeqSessionOverview: type: array items: $ref: '#/components/schemas/SessionOverview' SessionList: required: - sessionList type: object properties: sessionList: type: array items: $ref: '#/components/schemas/SessionOverview' SessionOverview: required: - createTime - user type: object properties: user: type: string ipAddr: type: string createTime: type: integer format: int64 sessionHandle: $ref: '#/components/schemas/SessionHandle' ``` ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1628 from cfmcgrady/kyuubi-1614. Closes #1614 9310f5a0 [Fu Chen] address comment ef3cb812 [Fu Chen] fix hardcoded version in OpenAPIDefinition Authored-by: Fu Chen <cfmcgrady@gmail.com> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
Kent Yao
|
99e278ce76
|
[KYUUBI #1623] KyuubiSessionEvent shall always have Id
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> session id is a val ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1623 from yaooqinn/val. Closes #1623 23fc3b2b [Kent Yao] test 6cd86610 [Kent Yao] npe 6fb75d47 [Kent Yao] KyuubiSessionEvent shall always have Id Authored-by: Kent Yao <yao@apache.org> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
ulysses-you
|
69efac59ef
|
[KYUUBI #1618] Correct hadoop version in RELEASE file
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> correct verion of - Spark Hadoop - Kyuubi Hadoop ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1618 from ulysses-you/correct-version. Closes #1618 64c8a5a8 [ulysses-you] grep -v 86d48df9 [ulysses-you] fix eb39f636 [ulysses-you] fix Authored-by: ulysses-you <ulyssesyou18@gmail.com> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
hongdongdong
|
4c6a07f847
|
[KYUUBI #1581] Add SchemaHelper for trino engine
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> Add SchemaHelper for trino engine ### _How was this patch tested?_ - [X] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1617 from hddong/add-schemaHelper. Closes #1581 f6a03fad [hongdongdong] fix 3c7d9079 [hongdongdong] fix a8e9463d [hongdongdong] add throw unrecognized type a03220bb [hongdongdong] [KYUUBI #1581] Add SchemaHelper for trino engine Authored-by: hongdongdong <hongdongdong@cmss.chinamobile.com> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
simon
|
c42f31e211
|
[KYUUBI #1620] Implement api: /${version}/operations/${operation_identifier}/log
### _Why are the changes needed?_ closes #1620 Implement api: /${version}/operations/${operation_identifier}/log ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1621 from simon824/1223. Closes #1620 5f211473 [simon] rm f6d0ce7c [simon] fix 346df798 [simon] operationLog c60d3447 [simon] init Authored-by: simon <zhangshiming@cvte.com> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
Wang Zhen
|
fbccba90f4
|
[KYUUBI #1613] Record the engine's applicationId in KyuubiSessionEvent
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> Record the engine's applicationId in KyuubiSessionEvent. #1613 ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [X] Add screenshots for manual tests if appropriate  - [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1622 from wForget/KYUUBI-1613. Closes #1613 24e001f6 [Wang Zhen] comment 29453fc3 [Wang Zhen] [KYUUBI-1613] Record the engine's applicationId in KyuubiSessionEvent Authored-by: Wang Zhen <wangzhen07@qiyi.com> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
Fu Chen
|
54a1b88245
|
[KYUUBI #1611] Fix swagger-ui redirect wrong to other service address
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> close #1611 ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1615 from cfmcgrady/kyuubi-1611. Closes #1611 f44bf3b1 [Fu Chen] fix swagger-ui Authored-by: Fu Chen <cfmcgrady@gmail.com> Signed-off-by: ulysses-you <ulyssesyou@apache.org> |
||
|
Kent Yao
|
1b48b1874f
|
[KYUUBI #1601] Align SparkStatementEvent to KyuubiOperationEvent
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> Align SparkStatementEvent to KyuubiOperationEvent ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1601 from yaooqinn/soe. Closes #1601 090dee8f [Kent Yao] Merge branch 'master' of github.com:apache/incubator-kyuubi into soe a61ce3b7 [Kent Yao] Align SparkStatementEvent to KyuubiOperationEven efbbdf79 [Kent Yao] Align SparkStatementEvent to KyuubiOperationEven Authored-by: Kent Yao <yao@apache.org> Signed-off-by: ulysses-you <ulyssesyou@apache.org> |
||
|
Cheng Pan
|
152e394016
|
[KYUUBI #1592] [TEST][ICEBERG][DELTA] Introduce row level operation test for data lake format
### _Why are the changes needed?_ Introduce row level operation test for data lake format and remove redundant tests ### _How was this patch tested?_ - [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1592 from pan3793/test. Closes #1592 892feb8a [Cheng Pan] Simplify test 72fd0939 [Cheng Pan] Fix import 9d208392 [Cheng Pan] Add row level operation tests for Iceberg and Delta 7730b4df [Cheng Pan] Also test JDBC connection MetaData in Kyuubi server 7a41dfdf [Cheng Pan] [TEST] Remove redundant type info test in DeltaMetadataTests Authored-by: Cheng Pan <chengpan@apache.org> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
hongdongdong
|
fcc6471fec
|
[KYUUBI #1582] Use ClientTypeSignature to further analysis of trino column type
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> Use ClientTypeSignature to further analysis of trino column type ### _How was this patch tested?_ - [X] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1598 from hddong/use-ClientTypeSignature. Closes #1582 36ef139e [hongdongdong] Add check 22060208 [hongdongdong] remove guard 337f11ee [hongdongdong] [KYUUBI #1582] Use ClientTypeSignature to further analysis of trino column type Authored-by: hongdongdong <hongdongdong@cmss.chinamobile.com> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
zhenjiaguo
|
3ffb12de24
|
[KYUUBI #1607] [DOC] Add operation and backend service metrics in metrics doc
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> Add operation and backend service metrics doc. ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [x] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request  Closes #1607 from zhenjiaguo/add_be_metric_doc. Closes #1607 e10bf146 [zhenjiaguo] Comment 0062f3d8 [zhenjiaguo] Metric format adjustment af84534c [zhenjiaguo] Add metrics version compatibility description c6f9eb26 [zhenjiaguo] Add operation and backend service method metric info doc Authored-by: zhenjiaguo <zhenjia_guo@163.com> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
yanghua
|
3673399329
|
[KYUUBI #1579] Implement basic ability of executing statement in Flink engine
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1603 from yanghua/KYUUBI-1579. Closes #1579 48db76b3 [Cheng Pan] cleanup 36707516 [Cheng Pan] Address comments 25ca5ae2 [yanghua] reduce code 6f18a4a0 [yanghua] [KYUUBI #1579] Implement basic ability of executing statement Lead-authored-by: yanghua <yanghua1127@gmail.com> Co-authored-by: Cheng Pan <chengpan@apache.org> Signed-off-by: Cheng Pan <chengpan@apache.org> |
||
|
simon
|
c972139b51
|
[KYUUBI #1575] Implement api: /${version}/operations/${operation_identifier}/resultsetmetadata
### _Why are the changes needed?_ #1575 Implement api: /${version}/operations/${operation_identifier}/resultsetmetadata - /${version}/operations/${operation_identifier}/resultsetmetadata - mapping: ICLIService#getResultSetMetadata - desc: get the table schema of the result set via the given operation identifier - method: GET - params: none - returns: an instance of TableSchema ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1576 from simon824/api1. Closes #1575 ecc976d1 [simon] reorder 0ef2accf [simon] add ea635da4 [simon] fix 310ad983 [simon] resultsetmetadata 595294fa [simon] init Authored-by: simon <zhangshiming@cvte.com> Signed-off-by: ulysses-you <ulyssesyou@apache.org> |
||
|
zwangsheng
|
d595eb97a1
|
[KYUUBI #1593] use user set host or ip instead of read hostname from user set
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> User can select IP or hostname for binding. Detail #1593 ### _How was this patch tested?_ - [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1602 from zwangsheng/kyuubi/1593-01. Closes #1593 8897ec46 [zwangsheng] fix test 67452451 [zwangsheng] use use set 76af3072 [zwangsheng] fix test f75d2db0 [zwangsheng] 1593 Authored-by: zwangsheng <2213335496@qq.com> Signed-off-by: ulysses-you <ulyssesyou@apache.org> |
||
|
Fu Chen
|
2b8304d154
|
[KYUUBI #1591] Watchdog support for Spark-3.2
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> <!-- 2. move spark-3.1 `MarkAggregateOrderRule` to spark-common and rename to `MarkAggregateOrderBase` --> 1. move spark-3.1 `ForcedMaxOutputRowsRule` to spark-common and rename to `ForcedMaxOutputRowsBase` 2. handle `WithCTE` logical plan in spark-3.2 3. move spark-3.1 `MaxPartitionStrategy` to spark-common 4. add netsted cte unit test for `ForcedMaxOutputRowsRule` ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1591 from cfmcgrady/watchdog-spark32. Closes #1591 5399a3f0 [Fu Chen] fix style 0ce83ba5 [Fu Chen] remove MarkAggregateOrderRule 364fc26e [Fu Chen] add license header 44726dee [Fu Chen] fix style 4847dbf3 [Fu Chen] watchdog support for spark-3.2 Authored-by: Fu Chen <cfmcgrady@gmail.com> Signed-off-by: ulysses-you <ulyssesyou@apache.org> |
||
|
Cheng Pan
|
d529402168
|
[KYUUBI #1587] [SUB-TASK][KPIP-2] Bump Flink 1.14.2
### _Why are the changes needed?_ Bump the latest version of 1.14.x to suppress log4-shell issue ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1587 from pan3793/flink-version. Closes #1587 9d7ca949 [Cheng Pan] Bump Flink 1.14.2 Authored-by: Cheng Pan <chengpan@apache.org> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
Wang Zhen
|
df1d9f3bb2
|
[KYUUBI #1577] Add DropIgnoreNonexistent Rule.
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> Like the `hive.exec.drop.ignorenonexistent` configuration in hive, do not report an error if DROP DATABASE/TABLE/VIEW/Function/Partition specifies a non-existent database/table/view/function/partition. For details: #1577. ### _How was this patch tested?_ - [X] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1583 from wForget/KYUUBI-1577. Closes #1577 63382660 [Wang Zhen] delete useless comment adadf424 [Wang Zhen] add DropIgnoreNonexistent rule using injectPostHocResolutionRule 860b15d3 [Wang Zhen] add DropIgnoreNonexistent rule for spark 3.2 aad43cdb [Wang Zhen] adjust code ca1b9f11 [Wang Zhen] make some adjustments and update rules.md 804bf40e [Wang Zhen] [KYUUBI-1577] Add DropIgnoreNonexistent Rule. Authored-by: Wang Zhen <wangzhen07@qiyi.com> Signed-off-by: ulysses-you <ulyssesyou@apache.org> |
||
|
zhenjiaguo
|
f3dc1fdecd
|
[KYUUBI #1586] Add time metric on each KyuubiBackendService method
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> Add time metric on each` KyuubiBackendService` method can help us inspect Kyuubi Server running status inside. It can Indirect reflecting our RPC call time when using Kyuubi. ### _How was this patch tested?_ - [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [x] Add screenshots for manual tests if appropriate  - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1588 from zhenjiaguo/add_be_method_metric. Closes #1586 a001eb96 [zhenjiaguo] Add time metric on each KyuubiBackendService method Authored-by: zhenjiaguo <zhenjia_guo@163.com> Signed-off-by: Kent Yao <yao@apache.org> |
