e4f5ba7010
255 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Cheng Pan
|
e4f5ba7010
|
[KYUUBI #1673] [BUILD] Improve log4j configurations
### _Why are the changes needed?_ - Flink engine print log to console on UT, and use `$FLINK_CONF_DIR/log4j.properties` in production. - Add missing log4j conf `kyuubi-trino-engine` module. - Fix log4j configuration issue of THRIFT ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1673 from pan3793/log. Closes #1673 8abe9d08 [Cheng Pan] Remove flink-sql-engine conf b1726288 [Cheng Pan] Improve log4j configurations Authored-by: Cheng Pan <chengpan@apache.org> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
minyk
|
0f8ebb6006
|
[KYUUBI #1675][FEATURE] Redact secret information from SparkSQLEngine log
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> Secret informations on log is dangerous on the production environment. Secret info should be redacted in some way. ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [x] Add screenshots for manual tests if appropriate capture from Spark Driver's STDERR  - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1677 from minyk/kyuubi-1675. Closes #1675 6a390575 [minyk] redact statement in the log Authored-by: minyk <minykreva@gmail.com> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
Wang Zhen
|
47533100f6
|
[KYUUBI #1674] Uncache cached tables when session closed
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> As mentioned in SPARK-29911 , there may also be memory leaks in kyuubi. #1674 ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [X] Add screenshots for manual tests if appropriate  close session:  - [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1676 from wForget/KYUUBI-1674. Closes #1674 e2bec4ec [Wang Zhen] [KYUUBI-1674] Uncache cached tables when session closed Authored-by: Wang Zhen <wangzhen07@qiyi.com> Signed-off-by: ulysses-you <ulyssesyou@apache.org> |
||
|
xifeng yang
|
8e9e8c3c7e
|
[KYUUBI #1597] fix row value in Query Execution should be queryExecution
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> row value in Query Execution should be queryExecution ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [x] Add screenshots for manual tests if appropriate  - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1657 from xifeng/kyuubi-1597. Closes #1597 cf9f9aca [xifeng yang] [KYUUBI #1597] fix row value in Query Execution should be queryExecution Authored-by: xifeng yang <xifeng.yang@hotmail.com> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
minyk
|
7d68aa07ff
|
[KYUUBI #1634][FEATURE] Redact secret information from SparkSQLEngine UI
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> SparkSQLEngine's Kyuubi UI tab shows SQL statements. In some cases, these informations are secret information like S3 access/secret key. Spark SQL UI tab already redact with `spark.redaction.string.regex` configuration. Kyuubi also should redact secret informations with this property. ### _How was this patch tested?_ - [X] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [X] Add screenshots for manual tests if appropriate with below property and query: spark-defaults.conf: ``` spark.redaction.string.regex (?i)url|access.*|secret.*|password ``` query: ``` SET fs.s3a.access.key=testkey ```  - [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1640 from minyk/redact-ui. Closes #1634 4051522a [minyk] set job description with setJobGroup() 3a8e0e91 [minyk] use redactedStatement c84778e3 [minyk] move to SparkOperation 97a4fb4a [minyk] fix code formatting 79fe52d3 [minyk] redact UI with `spark.redaction.string.regex` Authored-by: minyk <minykreva@gmail.com> Signed-off-by: Cheng Pan <chengpan@apache.org> |
||
|
Cheng Pan
|
cb8721fef1
|
[KYUUBI #1629] Flink backend implementation
### _Why are the changes needed?_ This PR covers #1619. Overall, this PR contains the following changs, 1. change `build/dist` script to support flink sql engine 2. enhance `externals/flink-sql-engine/pom.xml` to support create a shaded jar 3. simplify `externals/kyuubi-flink-sql-engine/bin/flink-sql-engine.sh` 4. introduce `FlinkSQLEngine`(flink sql engine entrypoint) and `FlinkProcessBuilder`(kyuubi server launcher) 5. add ut in kyuubi server side After this PR, we can run the basic query e.g. `select now()` from beeline and get result, and Kyuubi Server can auto launch flink engine if there is no proper one. The Flink engine also supports other engine share levels defined in Kyuubi. The implementation based on Flink 1.14 codebase. ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1629 from pan3793/flink-backend. Closes #1629 b7e5f0e3 [Cheng Pan] revert tgz name change a4496c37 [Cheng Pan] Fix reflection 3b4e86a1 [Cheng Pan] deps 68efa42c [Cheng Pan] log 8a9e37f3 [Cheng Pan] nit 10fb2bc3 [Cheng Pan] CI c1560fdb [Cheng Pan] nit 303e2f1e [Cheng Pan] Restore log conf d84720b1 [Cheng Pan] SessionContext b258d81a [Cheng Pan] cleanup 16edd528 [Cheng Pan] Cleanup 9ae54557 [Cheng Pan] Fix CI 25b6b57d [Cheng Pan] hadoop-client-api c12b5ca4 [Cheng Pan] Server UT pass 502d3f08 [Cheng Pan] pass dac4323b [yanghua] Laungh local flink engine container successfully Lead-authored-by: Cheng Pan <chengpan@apache.org> Co-authored-by: yanghua <yanghua1127@gmail.com> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
hongdongdong
|
4c6a07f847
|
[KYUUBI #1581] Add SchemaHelper for trino engine
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> Add SchemaHelper for trino engine ### _How was this patch tested?_ - [X] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1617 from hddong/add-schemaHelper. Closes #1581 f6a03fad [hongdongdong] fix 3c7d9079 [hongdongdong] fix a8e9463d [hongdongdong] add throw unrecognized type a03220bb [hongdongdong] [KYUUBI #1581] Add SchemaHelper for trino engine Authored-by: hongdongdong <hongdongdong@cmss.chinamobile.com> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
Wang Zhen
|
fbccba90f4
|
[KYUUBI #1613] Record the engine's applicationId in KyuubiSessionEvent
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> Record the engine's applicationId in KyuubiSessionEvent. #1613 ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [X] Add screenshots for manual tests if appropriate  - [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1622 from wForget/KYUUBI-1613. Closes #1613 24e001f6 [Wang Zhen] comment 29453fc3 [Wang Zhen] [KYUUBI-1613] Record the engine's applicationId in KyuubiSessionEvent Authored-by: Wang Zhen <wangzhen07@qiyi.com> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
Kent Yao
|
1b48b1874f
|
[KYUUBI #1601] Align SparkStatementEvent to KyuubiOperationEvent
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> Align SparkStatementEvent to KyuubiOperationEvent ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1601 from yaooqinn/soe. Closes #1601 090dee8f [Kent Yao] Merge branch 'master' of github.com:apache/incubator-kyuubi into soe a61ce3b7 [Kent Yao] Align SparkStatementEvent to KyuubiOperationEven efbbdf79 [Kent Yao] Align SparkStatementEvent to KyuubiOperationEven Authored-by: Kent Yao <yao@apache.org> Signed-off-by: ulysses-you <ulyssesyou@apache.org> |
||
|
Cheng Pan
|
152e394016
|
[KYUUBI #1592] [TEST][ICEBERG][DELTA] Introduce row level operation test for data lake format
### _Why are the changes needed?_ Introduce row level operation test for data lake format and remove redundant tests ### _How was this patch tested?_ - [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1592 from pan3793/test. Closes #1592 892feb8a [Cheng Pan] Simplify test 72fd0939 [Cheng Pan] Fix import 9d208392 [Cheng Pan] Add row level operation tests for Iceberg and Delta 7730b4df [Cheng Pan] Also test JDBC connection MetaData in Kyuubi server 7a41dfdf [Cheng Pan] [TEST] Remove redundant type info test in DeltaMetadataTests Authored-by: Cheng Pan <chengpan@apache.org> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
hongdongdong
|
fcc6471fec
|
[KYUUBI #1582] Use ClientTypeSignature to further analysis of trino column type
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> Use ClientTypeSignature to further analysis of trino column type ### _How was this patch tested?_ - [X] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1598 from hddong/use-ClientTypeSignature. Closes #1582 36ef139e [hongdongdong] Add check 22060208 [hongdongdong] remove guard 337f11ee [hongdongdong] [KYUUBI #1582] Use ClientTypeSignature to further analysis of trino column type Authored-by: hongdongdong <hongdongdong@cmss.chinamobile.com> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
yanghua
|
3673399329
|
[KYUUBI #1579] Implement basic ability of executing statement in Flink engine
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1603 from yanghua/KYUUBI-1579. Closes #1579 48db76b3 [Cheng Pan] cleanup 36707516 [Cheng Pan] Address comments 25ca5ae2 [yanghua] reduce code 6f18a4a0 [yanghua] [KYUUBI #1579] Implement basic ability of executing statement Lead-authored-by: yanghua <yanghua1127@gmail.com> Co-authored-by: Cheng Pan <chengpan@apache.org> Signed-off-by: Cheng Pan <chengpan@apache.org> |
||
|
yanghua
|
01364e589f
|
[KYUUBI #1578] Introduce flink-sql-client dependency to reduce some duplicated code
…uplicated code <!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1584 from yanghua/KYUUBI-1578. Closes #1578 90c2f89e [yanghua] [KYUUBI #1578] Introduce flink-sql-client dependency to reduce some duplicated code Authored-by: yanghua <yanghua1127@gmail.com> Signed-off-by: Cheng Pan <chengpan@apache.org> |
||
|
Kent Yao
|
b2e679d5c7
|
[KYUUBI #1499] Introduce DataFrameHolder for cli result fetching
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> Replace ArrayList with DataFrameHolder ### _How was this patch tested?_ - [X] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1507 from yaooqinn/1499. Closes #1499 7fd2b0ef [Kent Yao] fix it 6278009b [Kent Yao] root de3c601d [Kent Yao] fi 358d7a68 [Kent Yao] refine 8373b9b3 [Kent Yao] refine ab95f7dc [Kent Yao] refine 86d90b80 [Kent Yao] loader a07117c4 [Kent Yao] nit 90a7dd4f [Kent Yao] [KYUUBI #1499] Introduce DataFrameHolder for cli result fetching 8d97b51d [Kent Yao] [KYUUBI #1499] Introduce DataFrameHolder for cli result fetching Authored-by: Kent Yao <yao@apache.org> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
yanghua
|
4b58b0ab9b
|
[KYUUBI #1555][FLINK] Bump flink version to 1.14.0
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1573 from yanghua/KYUUBI-1555. Closes #1555 4f5ce746 [yanghua] [KYUUBI #1555] [SUB-TASK][KPIP-2] Bump flink version to 1.14.1 Authored-by: yanghua <yanghua1127@gmail.com> Signed-off-by: Cheng Pan <chengpan@apache.org> |
||
|
hongdongdong
|
11e208a688
|
[KYUUBI #1565] Move time functions to RowSetUtils and move schema to spark package
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> 1. move time functions from Rowset to RowSetUtils 2. move schema to spark package ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1566 from hddong/move-time-functions. Closes #1565 75de2e8d [hongdongdong] private 8b16217b [hongdongdong] [KYUUBI #1565] Move time functions to RowSetUtils and move schema to spark package Authored-by: hongdongdong <hongdongdong@cmss.chinamobile.com> Signed-off-by: Cheng Pan <chengpan@apache.org> |
||
|
simon
|
22d967e9c0
|
[KYUUBI #1539] Replace operationDetail with KyuubiEvent
### _Why are the changes needed?_ #1539 Replace operationDetail with KyuubiEvent ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1540 from simon824/event. Closes #1539 9d4e2a04 [Simon] Merge branch 'apache:master' into event 0cd7d15a [simon] replace operationDetail with OperationEvent adf327e8 [simon] replace operationDetail with OperationEvent 06b6ef8c [simon] replace operationDetail with OperationEvent b0973e20 [simon] replace operationDetail with OperationEvent cf83fafb [simon] replace operationDetail with OperationEvent bf955fbf [simon] replace operationDetail with OperationEvent 49c17988 [simon] replace operationDetail with OperationEvent 8990b701 [simon] replace operationDetail with OperationEvent 03937c2c [simon] replace operationDetail with OperationEvent a3981776 [simon] replace operationDetail with KyuubiEvent fc5d747d [simon] replace operationDetail with KyuubiEvent 344eba6a [simon] Revert b4461411 [simon] codestyle 1d758920 [simon] replace operationDetail with KyuubiEvent 258303ae [simon] init Lead-authored-by: simon <zhangshiming@cvte.com> Co-authored-by: Simon <3656562@qq.com> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
hongdongdong
|
c1a0ce8e01
|
[KYUUBI #1515] Add RowSet for trino engine
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> Add RowSet for trino engine ### _How was this patch tested?_ - [X] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1549 from hddong/add-rowset-for-trino. Closes #1515 cc885130 [hongdongdong] remove timeZone 2fb10934 [hongdongdong] fix toHiveString 9066caff [hongdongdong] rm optionalStart 491b7682 [hongdongdong] Reset 27895ebe [hongdongdong] fix RowSetUtils 7ce4edeb [hongdongdong] fix 34dd01b8 [hongdongdong] move functions to Utils 79e0621a [hongdongdong] fix 2dcd91e5 [hongdongdong] fix 71bdcfd2 [hongdongdong] [KYUUBI #1515] Add RowSet for trino engine Authored-by: hongdongdong <hongdongdong@cmss.chinamobile.com> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
yanghua
|
53ff56be29
|
[KYUUBI #1525][FLINK] Implement GetCatalogs operation and do the further initialization
… <!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1529 from yanghua/KYUUBI-1525. Closes #1525 e166965d [yanghua] Address review suggestion 9929aa29 [yanghua] Remove some comment eafb200b [yanghua] refactor code 58df34c7 [yanghua] refactor code 675e14e7 [yanghua] pruning unnecessary code 15e97283 [yanghua] pruning unnecessary code c6d4f549 [yanghua] pruning unnecessary code b099a455 [yanghua] pruning unnecessary code 316086bb [yanghua] pruning unnecessary code fe7af8ef [yanghua] pruning unnecessary code ecf3ccf4 [yanghua] pruning unnecessary code 4c174517 [yanghua] pruning unnecessary code b9226c3e [yanghua] pruning unnecessary code fb2f6a62 [yanghua] pruning unnecessary code 21e21a50 [yanghua] pruning unnecessary code a58076ff [yanghua] pruning unnecessary code acd24fc5 [yanghua] pruning unnecessary code 333d4275 [yanghua] pruning unnecessary code 7416b3de [yanghua] pruning unnecessary code 84d0db91 [yanghua] pruning unnecessary code and refactor code a55db41a [yanghua] pruning unnecessary code b9909308 [yanghua] [KYUUBI #1525] Implement GetCatalogs operation and do the further initialization Authored-by: yanghua <yanghua1127@gmail.com> Signed-off-by: Cheng Pan <chengpan@apache.org> |
||
|
yanghua
|
6045104092
|
[KYUUBI #1542] Move FetchIterator into kyuubi-common for multi-engine support
… <!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1543 from yanghua/KYUUBI-1542. Closes #1542 1ca1c08a [yanghua] Move FetchIteratorSuite into kyuubi-common module 1307d886 [yanghua] [KYUUBI #1542] Move FetchIterator into kyuubi-common for multi-engine support Authored-by: yanghua <yanghua1127@gmail.com> Signed-off-by: Cheng Pan <chengpan@apache.org> |
||
|
Wang Zhen
|
c363311e39
|
[KYUUBI #1528] Record the kyuubi server ip address in event log
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> Record the kyuubi server ip address in event log. For details: #1528 ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [X] Add screenshots for manual tests if appropriate - [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1530 from wForget/KYUUBI-1528. Closes #1528 6bc18571 [Wang Zhen] [KYUUBI-1528] Record the kyuubi server ip address in event log, set serverIpAddress in SessionEvent.apply 5f0307d1 [Wang Zhen] [KYUUBI-1528] Record the kyuubi server ip address in event log Authored-by: Wang Zhen <wangzhen07@qiyi.com> Signed-off-by: ulysses-you <ulyssesyou@apache.org> |
||
|
Wang Zhen
|
16add92299
|
[KYUUBI #1522] Pass the client ip to engine
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> Pass the client ip to engine. For details: #1522 ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1524 from wForget/KYUUBI-1522. Closes #1522 c9d54ed9 [Wang Zhen] fix word error a16d914d [Wang Zhen] [KYUUBI-1522] Pass the client ip to engine Authored-by: Wang Zhen <wangzhen07@qiyi.com> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
Kent Yao
|
fe31c3b62f
|
[KYUUBI #1489] Support operation log for ExecuteScala
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1518 from yaooqinn/1498. Closes #1489 890d1561 [Kent Yao] [KYUUBI #1489] Support operation log for ExecuteScala Authored-by: Kent Yao <yao@apache.org> Signed-off-by: ulysses-you <ulyssesyou@apache.org> |
||
|
Wang Zhen
|
986d9830db
|
[KYUUBI #1509] Make KYUUBI_WORK_DIR_ROOT as the default root path.
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> Make KYUUBI_WORK_DIR_ROOT as the default root path. For details: #1509. ### _How was this patch tested?_ - [X] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [X] Add screenshots for manual tests if appropriate - [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1519 from wForget/KYUUBI-1509. Closes #1509 ff078e5f [Wang Zhen] [KYUUBI-1509] Make KYUUBI_WORK_DIR_ROOT as the default root path. Authored-by: Wang Zhen <wangzhen07@qiyi.com> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
hongdongdong
|
6ea83e040e
|
[KYUUBI #1511] Initialize kyuubi trino engine module
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> 1. Initialize kyuubi trino engine module 2. Add `TrinoThriftBinaryFrontendService` ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1514 from hddong/init-trino-engine. Closes #1511 9dd0e55f [hongdongdong] [KYUUBI #1511] Initialize kyuubi trino engine module Authored-by: hongdongdong <hongdongdong@cmss.chinamobile.com> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
yanghua
|
bd3d4d7421
|
[KYUUBI #1503] Introduce FlinkThriftBinaryFrontendService to do the further initialization
… <!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1512 from yanghua/KYUUBI-1503. Closes #1503 b41fa735 [yanghua] [KYUUBI #1503] Introduce FlinkThriftBinaryFrontendService to do the further initialization Authored-by: yanghua <yanghua1127@gmail.com> Signed-off-by: ulysses-you <ulyssesyou@apache.org> |
||
|
ulysses-you
|
dad48c9d74
|
[KYUUBI #1508] Add Kyuubi version in UI
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> Help to get the Kyuubi version. ### _How was this patch tested?_  Closes #1508 from ulysses-you/version. Closes #1508 d2567664 [ulysses-you] version Authored-by: ulysses-you <ulyssesyou18@gmail.com> Signed-off-by: ulysses-you <ulyssesyou@apache.org> |
||
|
Fei Wang
|
920c8e5ad6 |
[KYUUBI #1504] Support to capture console out/error into interpreter output for ExecuteScala
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> Redirect console output and error when interpreting scala code, see details in #1504 ### _How was this patch tested?_ - [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1500 from turboFei/scala_out. Closes #1504 c3a7df1b [Fei Wang] refactor d85b6791 [Fei Wang] address comments f19be18f [Fei Wang] redirect output error Authored-by: Fei Wang <fwang12@ebay.com> Signed-off-by: Fei Wang <fwang12@ebay.com> |
||
|
yanghua
|
339fa7682a
|
[KYUUBI #1492] Initialize kyuubi flink sql engine module
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1497 from yanghua/KYUUBI-1492. Closes #1492 ba9f221a [yanghua] [KYUUBI #1492] Initialize kyuubi flink sql engine module Authored-by: yanghua <yanghua1127@gmail.com> Signed-off-by: Cheng Pan <chengpan@apache.org> |
||
|
Kent Yao
|
97c3835738
|
[KYUUBI #1490] Introduce the basic framework for running scala
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> Introduce the basic framework for running scala, see #1490 for the detail ### _How was this patch tested?_ - [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [x] Add screenshots for manual tests if appropriate ``` Beeline version 1.5.0-SNAPSHOT by Apache Kyuubi (Incubating) 0: jdbc:hive2://10.242.189.214:10009/> spark.version; 2021-12-03 13:47:07.556 INFO operation.ExecuteStatement: Processing kent's query[08b8b6da-d434-4296-b613-2027e3518441]: INITIALIZED_STATE -> PENDING_STATE, statement: spark.version 2021-12-03 13:47:07.560 INFO operation.ExecuteStatement: Processing kent's query[08b8b6da-d434-4296-b613-2027e3518441]: PENDING_STATE -> RUNNING_STATE, statement: spark.version 2021-12-03 13:47:07.558 INFO operation.ExecuteStatement: Processing kent's query[321dc15d-68d0-4f91-9216-1e08f09842df]: INITIALIZED_STATE -> PENDING_STATE, statement: spark.version 2021-12-03 13:47:07.559 INFO operation.ExecuteStatement: Processing kent's query[321dc15d-68d0-4f91-9216-1e08f09842df]: PENDING_STATE -> RUNNING_STATE, statement: spark.version 2021-12-03 13:47:07.560 INFO operation.ExecuteStatement: Spark application name: kyuubi_USER_SPARK_SQL_kent_default_61cff9fb-7035-4435-b509-80c1730876ed application ID: local-1638510289918 application web UI: http://10.242.189.214:65027 master: local[*] deploy mode: client version: 3.1.2 Start time: 2021-12-03T13:44:49.313 User: kent 2021-12-03 13:47:07.562 INFO scheduler.DAGScheduler: Asked to cancel job group 321dc15d-68d0-4f91-9216-1e08f09842df 2021-12-03 13:47:07.565 INFO operation.ExecuteStatement: Processing kent's query[321dc15d-68d0-4f91-9216-1e08f09842df]: RUNNING_STATE -> ERROR_STATE, statement: spark.version, time taken: 0.006 seconds 2021-12-03 13:47:07.565 ERROR operation.ExecuteStatement: Error operating EXECUTE_STATEMENT: org.apache.spark.sql.catalyst.parser.ParseException: mismatched input 'spark' expecting {'(', 'ADD', 'ALTER', 'ANALYZE', 'CACHE', 'CLEAR', 'COMMENT', 'COMMIT', 'CREATE', 'DELETE', 'DESC', 'DESCRIBE', 'DFS', 'DROP', 'EXPLAIN', 'EXPORT', 'FROM', 'GRANT', 'IMPORT', 'INSERT', 'LIST', 'LOAD', 'LOCK', 'MAP', 'MERGE', 'MSCK', 'REDUCE', 'REFRESH', 'REPLACE', 'RESET', 'REVOKE', 'ROLLBACK', 'SELECT', 'SET', 'SHOW', 'START', 'TABLE', 'TRUNCATE', 'UNCACHE', 'UNLOCK', 'UPDATE', 'USE', 'VALUES', 'WITH'}(line 1, pos 0) == SQL == spark.version ^^^ at org.apache.spark.sql.catalyst.parser.ParseException.withCommand(ParseDriver.scala:255) at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parse(ParseDriver.scala:124) at org.apache.spark.sql.execution.SparkSqlParser.parse(SparkSqlParser.scala:49) at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parsePlan(ParseDriver.scala:75) at org.apache.spark.sql.SparkSession.$anonfun$sql$2(SparkSession.scala:616) at org.apache.spark.sql.catalyst.QueryPlanningTracker.measurePhase(QueryPlanningTracker.scala:111) at org.apache.spark.sql.SparkSession.$anonfun$sql$1(SparkSession.scala:616) at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:775) at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:613) at org.apache.kyuubi.engine.spark.operation.ExecuteStatement.$anonfun$executeStatement$1(ExecuteStatement.scala:98) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23) at org.apache.kyuubi.engine.spark.operation.ExecuteStatement.withLocalProperties(ExecuteStatement.scala:157) at org.apache.kyuubi.engine.spark.operation.ExecuteStatement.org$apache$kyuubi$engine$spark$operation$ExecuteStatement$$executeStatement(ExecuteStatement.scala:92) at org.apache.kyuubi.engine.spark.operation.ExecuteStatement$$anon$1.run(ExecuteStatement.scala:125) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) org.apache.spark.sql.catalyst.parser.ParseException: mismatched input 'spark' expecting {'(', 'ADD', 'ALTER', 'ANALYZE', 'CACHE', 'CLEAR', 'COMMENT', 'COMMIT', 'CREATE', 'DELETE', 'DESC', 'DESCRIBE', 'DFS', 'DROP', 'EXPLAIN', 'EXPORT', 'FROM', 'GRANT', 'IMPORT', 'INSERT', 'LIST', 'LOAD', 'LOCK', 'MAP', 'MERGE', 'MSCK', 'REDUCE', 'REFRESH', 'REPLACE', 'RESET', 'REVOKE', 'ROLLBACK', 'SELECT', 'SET', 'SHOW', 'START', 'TABLE', 'TRUNCATE', 'UNCACHE', 'UNLOCK', 'UPDATE', 'USE', 'VALUES', 'WITH'}(line 1, pos 0) == SQL == spark.version ^^^ at org.apache.spark.sql.catalyst.parser.ParseException.withCommand(ParseDriver.scala:255) at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parse(ParseDriver.scala:124) at org.apache.spark.sql.execution.SparkSqlParser.parse(SparkSqlParser.scala:49) at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parsePlan(ParseDriver.scala:75) at org.apache.spark.sql.SparkSession.$anonfun$sql$2(SparkSession.scala:616) at org.apache.spark.sql.catalyst.QueryPlanningTracker.measurePhase(QueryPlanningTracker.scala:111) at org.apache.spark.sql.SparkSession.$anonfun$sql$1(SparkSession.scala:616) at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:775) at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:613) at org.apache.kyuubi.engine.spark.operation.ExecuteStatement.$anonfun$executeStatement$1(ExecuteStatement.scala:98) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23) at org.apache.kyuubi.engine.spark.operation.ExecuteStatement.withLocalProperties(ExecuteStatement.scala:157) at org.apache.kyuubi.engine.spark.operation.ExecuteStatement.org$apache$kyuubi$engine$spark$operation$ExecuteStatement$$executeStatement(ExecuteStatement.scala:92) at org.apache.kyuubi.engine.spark.operation.ExecuteStatement$$anon$1.run(ExecuteStatement.scala:125) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2021-12-03 13:47:07.569 INFO operation.ExecuteStatement: Query[08b8b6da-d434-4296-b613-2027e3518441] in ERROR_STATE 2021-12-03 13:47:07.569 INFO operation.ExecuteStatement: Processing kent's query[08b8b6da-d434-4296-b613-2027e3518441]: RUNNING_STATE -> ERROR_STATE, statement: spark.version, time taken: 0.009 seconds Error: Error operating EXECUTE_STATEMENT: org.apache.spark.sql.catalyst.parser.ParseException: mismatched input 'spark' expecting {'(', 'ADD', 'ALTER', 'ANALYZE', 'CACHE', 'CLEAR', 'COMMENT', 'COMMIT', 'CREATE', 'DELETE', 'DESC', 'DESCRIBE', 'DFS', 'DROP', 'EXPLAIN', 'EXPORT', 'FROM', 'GRANT', 'IMPORT', 'INSERT', 'LIST', 'LOAD', 'LOCK', 'MAP', 'MERGE', 'MSCK', 'REDUCE', 'REFRESH', 'REPLACE', 'RESET', 'REVOKE', 'ROLLBACK', 'SELECT', 'SET', 'SHOW', 'START', 'TABLE', 'TRUNCATE', 'UNCACHE', 'UNLOCK', 'UPDATE', 'USE', 'VALUES', 'WITH'}(line 1, pos 0) == SQL == spark.version ^^^ at org.apache.spark.sql.catalyst.parser.ParseException.withCommand(ParseDriver.scala:255) at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parse(ParseDriver.scala:124) at org.apache.spark.sql.execution.SparkSqlParser.parse(SparkSqlParser.scala:49) at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parsePlan(ParseDriver.scala:75) at org.apache.spark.sql.SparkSession.$anonfun$sql$2(SparkSession.scala:616) at org.apache.spark.sql.catalyst.QueryPlanningTracker.measurePhase(QueryPlanningTracker.scala:111) at org.apache.spark.sql.SparkSession.$anonfun$sql$1(SparkSession.scala:616) at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:775) at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:613) at org.apache.kyuubi.engine.spark.operation.ExecuteStatement.$anonfun$executeStatement$1(ExecuteStatement.scala:98) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23) at org.apache.kyuubi.engine.spark.operation.ExecuteStatement.withLocalProperties(ExecuteStatement.scala:157) at org.apache.kyuubi.engine.spark.operation.ExecuteStatement.org$apache$kyuubi$engine$spark$operation$ExecuteStatement$$executeStatement(ExecuteStatement.scala:92) at org.apache.kyuubi.engine.spark.operation.ExecuteStatement$$anon$1.run(ExecuteStatement.scala:125) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) (state=,code=0) 0: jdbc:hive2://10.242.189.214:10009/> set kyuubi.operation.language=scala; 2021-12-03 13:47:11.982 INFO operation.ExecuteStatement: Processing kent's query[e4e4dc55-4e05-460e-9d9d-86b2cf5c054b]: INITIALIZED_STATE -> PENDING_STATE, statement: set kyuubi.operation.language=scala 2021-12-03 13:47:11.985 INFO operation.ExecuteStatement: Processing kent's query[e4e4dc55-4e05-460e-9d9d-86b2cf5c054b]: PENDING_STATE -> RUNNING_STATE, statement: set kyuubi.operation.language=scala 2021-12-03 13:47:11.983 INFO operation.ExecuteStatement: Processing kent's query[2b6ec68c-2ae9-4d93-a35b-9e2319df4d4f]: INITIALIZED_STATE -> PENDING_STATE, statement: set kyuubi.operation.language=scala 2021-12-03 13:47:11.984 INFO operation.ExecuteStatement: Processing kent's query[2b6ec68c-2ae9-4d93-a35b-9e2319df4d4f]: PENDING_STATE -> RUNNING_STATE, statement: set kyuubi.operation.language=scala 2021-12-03 13:47:11.985 INFO operation.ExecuteStatement: Spark application name: kyuubi_USER_SPARK_SQL_kent_default_61cff9fb-7035-4435-b509-80c1730876ed application ID: local-1638510289918 application web UI: http://10.242.189.214:65027 master: local[*] deploy mode: client version: 3.1.2 Start time: 2021-12-03T13:44:49.313 User: kent 2021-12-03 13:47:11.995 INFO operation.ExecuteStatement: Processing kent's query[2b6ec68c-2ae9-4d93-a35b-9e2319df4d4f]: RUNNING_STATE -> RUNNING_STATE, statement: set kyuubi.operation.language=scala 2021-12-03 13:47:11.995 INFO operation.ExecuteStatement: Execute in full collect mode 2021-12-03 13:47:12.006 INFO operation.ExecuteStatement: Processing kent's query[2b6ec68c-2ae9-4d93-a35b-9e2319df4d4f]: RUNNING_STATE -> FINISHED_STATE, statement: set kyuubi.operation.language=scala, time taken: 0.022 seconds 2021-12-03 13:47:12.007 INFO operation.ExecuteStatement: Query[e4e4dc55-4e05-460e-9d9d-86b2cf5c054b] in FINISHED_STATE 2021-12-03 13:47:12.007 INFO operation.ExecuteStatement: Processing kent's query[e4e4dc55-4e05-460e-9d9d-86b2cf5c054b]: RUNNING_STATE -> FINISHED_STATE, statement: set kyuubi.operation.language=scala, time taken: 0.022 seconds +----------------------------+--------+ | key | value | +----------------------------+--------+ | kyuubi.operation.language | scala | +----------------------------+--------+ 1 row selected (0.052 seconds) 0: jdbc:hive2://10.242.189.214:10009/> spark.version; 2021-12-03 13:47:13.685 INFO operation.ExecuteStatement: Processing kent's query[178ff72d-b870-44d4-99d8-08a0fa0e8efa]: INITIALIZED_STATE -> PENDING_STATE, statement: spark.version 2021-12-03 13:47:15.541 INFO operation.ExecuteStatement: Processing kent's query[178ff72d-b870-44d4-99d8-08a0fa0e8efa]: PENDING_STATE -> RUNNING_STATE, statement: spark.version 2021-12-03 13:47:15.543 INFO operation.ExecuteStatement: Query[178ff72d-b870-44d4-99d8-08a0fa0e8efa] in FINISHED_STATE 2021-12-03 13:47:15.544 INFO operation.ExecuteStatement: Processing kent's query[178ff72d-b870-44d4-99d8-08a0fa0e8efa]: RUNNING_STATE -> FINISHED_STATE, statement: spark.version, time taken: 0.003 seconds +-----------------------+ | output | +-----------------------+ | res0: String = 3.1.2 | +-----------------------+ 1 row selected (1.871 seconds) 0: jdbc:hive2://10.242.189.214:10009/> spark.sql("select current_date()") . . . . . . . . . . . . . . . . . . .> ; 2021-12-03 13:47:36.512 INFO operation.ExecuteStatement: Processing kent's query[602c2d88-7e8a-4175-9b53-e34adfff007a]: INITIALIZED_STATE -> PENDING_STATE, statement: spark.sql("select current_date()") 2021-12-03 13:47:36.689 INFO operation.ExecuteStatement: Processing kent's query[602c2d88-7e8a-4175-9b53-e34adfff007a]: PENDING_STATE -> RUNNING_STATE, statement: spark.sql("select current_date()") 2021-12-03 13:47:36.692 INFO operation.ExecuteStatement: Query[602c2d88-7e8a-4175-9b53-e34adfff007a] in FINISHED_STATE 2021-12-03 13:47:36.692 INFO operation.ExecuteStatement: Processing kent's query[602c2d88-7e8a-4175-9b53-e34adfff007a]: RUNNING_STATE -> FINISHED_STATE, statement: spark.sql("select current_date()"), time taken: 0.003 seconds +----------------------------------------------------+ | output | +----------------------------------------------------+ | res1: org.apache.spark.sql.DataFrame = [current_date(): date] | +----------------------------------------------------+ 1 row selected (0.187 seconds) 0: jdbc:hive2://10.242.189.214:10009/> results += spark.range(1, 5, 2, 3); Error: Error operating EXECUTE_STATEMENT: org.apache.kyuubi.KyuubiSQLException: Interpret error: results += spark.range(1, 5, 2, 3) <console>:26: error: type mismatch; found : org.apache.spark.sql.Dataset[Long] required: org.apache.spark.sql.Dataset[org.apache.spark.sql.Row] results += spark.range(1, 5, 2, 3) ^ at org.apache.kyuubi.KyuubiSQLException$.apply(KyuubiSQLException.scala:69) at org.apache.kyuubi.engine.spark.operation.ExecuteScala.runInternal(ExecuteScala.scala:70) at org.apache.kyuubi.operation.AbstractOperation.run(AbstractOperation.scala:130) at org.apache.kyuubi.session.AbstractSession.runOperation(AbstractSession.scala:93) at org.apache.kyuubi.engine.spark.session.SparkSessionImpl.runOperation(SparkSessionImpl.scala:62) at org.apache.kyuubi.session.AbstractSession.$anonfun$executeStatement$1(AbstractSession.scala:121) at org.apache.kyuubi.session.AbstractSession.withAcquireRelease(AbstractSession.scala:75) at org.apache.kyuubi.session.AbstractSession.executeStatement(AbstractSession.scala:118) at org.apache.kyuubi.service.AbstractBackendService.executeStatement(AbstractBackendService.scala:61) at org.apache.kyuubi.service.ThriftBinaryFrontendService.ExecuteStatement(ThriftBinaryFrontendService.scala:265) at org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1557) at org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1542) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.kyuubi.service.authentication.TSetIpAddressProcessor.process(TSetIpAddressProcessor.scala:36) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:310) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) (state=,code=0) 0: jdbc:hive2://10.242.189.214:10009/> results += spark.range(1, 5, 2, 3).toDF; 2021-12-03 13:48:30.327 INFO operation.ExecuteStatement: Processing kent's query[daccd163-978a-4d66-af85-af466086d1db]: INITIALIZED_STATE -> PENDING_STATE, statement: results += spark.range(1, 5, 2, 3).toDF 2021-12-03 13:48:31.700 INFO operation.ExecuteStatement: Processing kent's query[daccd163-978a-4d66-af85-af466086d1db]: PENDING_STATE -> RUNNING_STATE, statement: results += spark.range(1, 5, 2, 3).toDF 2021-12-03 13:48:31.702 INFO operation.ExecuteStatement: Query[daccd163-978a-4d66-af85-af466086d1db] in FINISHED_STATE 2021-12-03 13:48:31.702 INFO operation.ExecuteStatement: Processing kent's query[daccd163-978a-4d66-af85-af466086d1db]: RUNNING_STATE -> FINISHED_STATE, statement: results += spark.range(1, 5, 2, 3).toDF, time taken: 0.003 seconds +-----+ | id | +-----+ | 1 | | 3 | +-----+ 2 rows selected (1.387 seconds) ``` ** Session level isolated **  - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1491 from yaooqinn/scala. Closes #1490 af4d0a13 [Kent Yao] Merge branch 'master' into scala 2ebdf6a3 [Kent Yao] provided scala dep d58b2bfe [Kent Yao] [KYUUBI #1490] Introduce the basic framework for running scala d256bfde [Kent Yao] [KYUUBI #1490] Introduce the basic framework for running scala 52090434 [Kent Yao] init Authored-by: Kent Yao <yao@apache.org> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
Brian Yue
|
a876678a91 |
[KYUUBI #1493] Make a reasonable thread pool size for listing file statuses
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> Close #1493. ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1494 from byyue/feature-1493. Closes #1493 9a280926 [Brian Yue] Make a reasonable thread pool size for listing file statuses Authored-by: Brian Yue <code.byyue@gmail.com> Signed-off-by: Fei Wang <fwang12@ebay.com> |
||
|
Muru Muthusamy
|
b87d206ffb
|
[KYUUBI #1458] Delta lake table columns won't show up in DBeaver.
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. Delta lake table columns won't show up in DBeaver client. This fix will address this issue and able to view table columns in the DBeaver now. --> ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1476 from mmuru/delta-table-cols-dbeaver-fix. Closes #1458 7801c701 [Muru Muthusamy] Reformatted to fix style issue. 4126da47 [Muru Muthusamy] Updated based on review feedback and added ignore case sensitive check. 28a4a12d [Muru Muthusamy] [KYUUBI#1458] Delta lake table columns won't show up in DBeaver. Authored-by: Muru Muthusamy <mmuthusamy@womply.com> Signed-off-by: Cheng Pan <chengpan@apache.org> |
||
|
fwang12
|
a6d23238cb |
[KYUUBI #1382] Sync the flink-provided profile in docs and testing suite comments
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> Sync the flink-provided profile in docs and testing suite comments. ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1382 from turboFei/flink_provided. Closes #1382 3ae96963 [Fei Wang] address nit ef5bba7f [fwang12] reorder e9c9c527 [fwang12] save Lead-authored-by: fwang12 <fwang12@ebay.com> Co-authored-by: Fei Wang <fwang12@ebay.com> Signed-off-by: Fei Wang <fwang12@ebay.com> |
||
|
sychen
|
4bf17a25ed
|
[KYUUBI #1473] Exit gracefully when engine idle
### _Why are the changes needed?_ https://github.com/apache/incubator-kyuubi/issues/1473 It is not graceful to exit when engine is idle, which may cause AM to try again. The current workaround is to configure `spark.yarn.maxAppAttempts`. ``` yarn.ApplicationMaster: Final app status: FAILED, exitCode: 16, (reason: Shutdown hook called before final status was reported.) ``` ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [x] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1474 from cxzl25/KYUUBI-1473. Closes #1473 dc07b387 [sychen] assert engine 6df54ebe [sychen] Exit gracefully when engine idle Authored-by: sychen <sychen@trip.com> Signed-off-by: ulysses-you <ulyssesyou@apache.org> |
||
|
fwang12
|
119e7f2c2c |
[KYUUBI #1441] [BUILD] Bump 1.5.0-SNAPSHOT
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1441 from turboFei/bump_1.5.0. Closes #1441 0c03c75d [fwang12] [BUILD] Bump 1.5.0-SNAPSHOT Authored-by: fwang12 <fwang12@ebay.com> Signed-off-by: fwang12 <fwang12@ebay.com> |
||
|
ulysses-you
|
ac24c8fb8c
|
[KYUUBI #1429][TASK-1] Move ui package from org.apache.spark to org.apache.spark.ui
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> In order to compaible with both Spark-2 and Spark-3, we should respect package `ui`, since https://github.com/apache/spark/pull/22645, some class cope changed from package `ui` to `spark`. ### _How was this patch tested?_ Pass CI Closes #1430 from ulysses-you/spark-2.4. Closes #1429 8b40c600 [ulysses-you] nit f875cda3 [ulysses-you] ui Authored-by: ulysses-you <ulyssesyou18@gmail.com> Signed-off-by: ulysses-you <ulyssesyou@apache.org> |
||
|
zhang1002
|
73eeea5f5c
|
[KYUUBI #1436] unified the field name as kyuubiStatementEvent
### _Why are the changes needed?_ unified the field name as kyuubiStatementEvent ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1436 from zhang1002/unified-field-name. Closes #1436 4c83e44a [zhang1002] unified the field name as kyuubiStatementEvent Authored-by: zhang1002 <zhang1002@126.com> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
zhang1002
|
c8a7623578
|
[KYUUBI #1437] Fix error message on Spark UI
### _Why are the changes needed?_ error message modification query execution is just the sql's logical plan not the error message. ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1437 from zhang1002/error-message-modification. Closes #1437 744c218a [zhang1002] error message modification Authored-by: zhang1002 <zhang1002@126.com> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
zhouyifan279
|
ffe41631ce
|
[KYUUBI #1383] Leverage Scalafmt to auto format scala code
### _Why are the changes needed?_ #1383 As IntelliJ IDEA's `Optimize Imports` result does not meet rules in `scalastyle-config.xml`, currently developers have to arrange package imports manually. In this PR, we introduce a `.scalafmt.conf` file to solve the problem. It works as follows: 1. Set scala code style formatter to `scalafmt`. 2. Execute `Reformat Code` 3. IDEA picks up configurations in `.scalafmt.conf` and format both code lines and imports. We also formatted all historical scala codes. ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1391 from zhouyifan279/1383. Closes #1383 e0696a9e [zhouyifan279] [KYUUBI #1383] Leverage Scalafmt to auto format scala code 18b8e229 [zhouyifan279] [KYUUBI #1383] Leverage Scalafmt to auto format scala code Authored-by: zhouyifan279 <zhouyifan279@gmail.com> Signed-off-by: ulysses-you <ulyssesyou@apache.org> |
||
|
Jagadesh Adireddi
|
15844dc373 |
[KYUUBI #1056] [BUG] Log error if no current spark sql engine is created.
Issue: https://github.com/apache/incubator-kyuubi/issues/1056 ### _Why are the changes needed?_ Log an error message if no current spark sql engine is created. ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1423 from jadireddi/1056-no-currEng. Closes #1056 a8d0443d [Jagadesh Adireddi] refactor 565d41a4 [Jagadesh Adireddi] [KYUUBI #1056] [BUG] Log error if no current spark sql engine is created. Authored-by: Jagadesh Adireddi <jadiredd@conviva.com> Signed-off-by: fwang12 <fwang12@ebay.com> |
||
|
ulysses-you
|
fda295865e |
[KYUUBI #1421] Improve beeline print
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> We don't support isolation, so it's unnecessary to print isolation level. before this PR ``` Connected to: Spark SQL (version 1.4.0-SNAPSHOT) Driver: Kyuubi Project Hive JDBC Shaded Client (version 1.4.0-SNAPSHOT) Transaction isolation: TRANSACTION_REPEATABLE_READ Beeline version 1.4.0-SNAPSHOT by Apache Kyuubi ``` After: ``` Connected to: Apache Kyuubi (Incubating) (version 1.4.0-SNAPSHOT) Driver: Kyuubi Project Hive JDBC Shaded Client (version 1.4.0-SNAPSHOT) Beeline version 1.4.0-SNAPSHOT by Apache Kyuubi (Incubating) ``` ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1421 from ulysses-you/improve-log. Closes #1421 2e0e168e [ulysses-you] improve Authored-by: ulysses-you <ulyssesyou18@gmail.com> Signed-off-by: fwang12 <fwang12@ebay.com> |
||
|
yanghua
|
20ee845c0b
|
[KYUUBI #1386] Refactor SparkThriftBinaryFrontendService to extract connectionUrl into super class
… <!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1387 from yanghua/KYUUBI-1386. Closes #1386 29415904 [yanghua] [KYUUBI #1386] Refactor SparkThriftBinaryFrontendService to extract connectionUrl into super class Authored-by: yanghua <yanghua1127@gmail.com> Signed-off-by: ulysses-you <ulyssesyou@apache.org> |
||
|
Cheng Pan
|
cb9e55a9b4
|
[KYUUBI #1380] Fix kyuubi download duplicated plugins
### _Why are the changes needed?_ Fix maven pom.xml issue ``` (apache-kyuubi) ➜ apache-kyuubi git:(mysql-fe-impl) ✗ build/mvn clean install Using `mvn` from path: /Users/chengpan/.sdkman/candidates/maven/current/bin/mvn [INFO] Scanning for projects... [WARNING] [WARNING] Some problems were encountered while building the effective model for org.apache.kyuubi:kyuubi-download:pom:1.4.0-SNAPSHOT [WARNING] 'build.plugins.plugin.(groupId:artifactId)' must be unique but found duplicate declaration of plugin com.googlecode.maven-download-plugin:download-maven-plugin line 62, column 21 [WARNING] [WARNING] It is highly recommended to fix these problems because they threaten the stability of your build. [WARNING] [WARNING] For this reason, future Maven versions might no longer support building such malformed projects. [WARNING] [INFO] ------------------------------------------------------------------------ ``` ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request ``` (apache-kyuubi) ➜ apache-kyuubi git:(download) ✗ build/mvn compile -pl :kyuubi-download -Pspark-provided,flink-provided | grep -A 100 'scala-maven-plugin:4.3.0:compile' Using `mvn` from path: /Users/chengpan/.sdkman/candidates/maven/current/bin/mvn [INFO] --- scala-maven-plugin:4.3.0:compile (scala-compile-first) kyuubi-download --- [INFO] compile in 0.0 s [INFO] No sources to compile [INFO] [INFO] --- download-maven-plugin:1.6.6:wget (download-spark-release) kyuubi-download --- [INFO] maven-download-plugin:wget skipped [INFO] [INFO] --- download-maven-plugin:1.6.6:wget (download-flink-release) kyuubi-download --- [INFO] maven-download-plugin:wget skipped [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 1.932 s [INFO] Finished at: 2021-11-15T10:41:05+08:00 [INFO] ------------------------------------------------------------------------ ``` ``` (apache-kyuubi) ➜ apache-kyuubi git:(download) ✗ build/mvn compile -pl :kyuubi-download -Pflink-provided | grep -A 100 'scala-maven-plugin:4.3.0:compile' Using `mvn` from path: /Users/chengpan/.sdkman/candidates/maven/current/bin/mvn [INFO] --- scala-maven-plugin:4.3.0:compile (scala-compile-first) kyuubi-download --- [INFO] compile in 0.0 s [INFO] No sources to compile [INFO] [INFO] --- download-maven-plugin:1.6.6:wget (download-spark-release) kyuubi-download --- [INFO] Expanding: /Users/chengpan/Projects/apache-kyuubi/externals/kyuubi-download/target/spark-3.1.2-bin-hadoop3.2.tgz into /Users/chengpan/Projects/apache-kyuubi/externals/kyuubi-download/target [INFO] [INFO] --- download-maven-plugin:1.6.6:wget (download-flink-release) kyuubi-download --- [INFO] maven-download-plugin:wget skipped [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 4.486 s [INFO] Finished at: 2021-11-15T10:42:22+08:00 [INFO] ------------------------------------------------------------------------ ``` ``` (apache-kyuubi) ➜ apache-kyuubi git:(download) ✗ build/mvn compile -pl :kyuubi-download -Pspark-provided | grep -A 100 'scala-maven-plugin:4.3.0:compile' Using `mvn` from path: /Users/chengpan/.sdkman/candidates/maven/current/bin/mvn [INFO] --- scala-maven-plugin:4.3.0:compile (scala-compile-first) kyuubi-download --- [INFO] compile in 0.0 s [INFO] No sources to compile [INFO] [INFO] --- download-maven-plugin:1.6.6:wget (download-spark-release) kyuubi-download --- [INFO] maven-download-plugin:wget skipped [INFO] [INFO] --- download-maven-plugin:1.6.6:wget (download-flink-release) kyuubi-download --- [INFO] Expanding: /Users/chengpan/Projects/apache-kyuubi/externals/kyuubi-download/target/flink-1.12.5-bin-scala_2.12.tgz into /Users/chengpan/Projects/apache-kyuubi/externals/kyuubi-download/target [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 5.114 s [INFO] Finished at: 2021-11-15T10:41:53+08:00 [INFO] ------------------------------------------------------------------------ ``` Closes #1380 from pan3793/download. Closes #1380 21c7cc22 [Cheng Pan] Fix kyuubi download duplicated plugins Authored-by: Cheng Pan <chengpan@apache.org> Signed-off-by: Cheng Pan <chengpan@apache.org> |
||
|
Cheng Pan
|
4ac8b09b2f
|
[KYUUBI #1331] [TEST] Refactor test classes to match multi frontend protocol
### _Why are the changes needed?_ As Kyuubi supports REST frontend protocol now, and MySQL frontend protocol is coming soon, Thrift is not the only one, this PR refacotor the test classes to match the changes. ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1331 from pan3793/test. Closes #1331 67c65810 [Cheng Pan] nit f9168358 [Cheng Pan] [TEST] Refactor test classes to match multi frontend protocol Authored-by: Cheng Pan <chengpan@apache.org> Signed-off-by: Cheng Pan <chengpan@apache.org> |
||
|
yanghua
|
aad575ce3e
|
[KYUUBI #1329] Support download Flink binary package in kyuubi-download module
… <!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1339 from yanghua/KYUUBI-1329. Closes #1329 895815fb [yanghua] address review suggestion eddf0605 [yanghua] [KYUUBI #1329] Support download Flink binary package in kyuubi-download module Authored-by: yanghua <yanghua1127@gmail.com> Signed-off-by: ulysses-you <ulyssesyou@apache.org> |
||
|
Kent Yao
|
fd17dd0ae4
|
[KYUUBI #1300] Detecting critical errors
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> For critical errors at engine side, it is not handled properly. For example,when engine oom - server may not be able to get operation statuses because of no response of engine side. In this case, client only get a ambiguous `read timeout` as a final cause. - the oom hook of engine side might directly crash the engine, when the server is still trying to get operation status In this PR, - a config for retry to make the operation status updating process more robust - make the engine oom hook only de register itself to make it able to recover for some transient errors ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1312 from yaooqinn/1300. Closes #1300 a715ecca [Kent Yao] add comments a816b0f0 [Kent Yao] refine 3557c927 [Kent Yao] add comments 2ed8bfb4 [Kent Yao] refine aefd2a7f [Kent Yao] update doc f2ea7e4c [Kent Yao] restore SparkOperation 386e4eac [Kent Yao] [KYUUBI #1300] Detecting critical errors Authored-by: Kent Yao <yao@apache.org> Signed-off-by: Cheng Pan <chengpan@apache.org> |
||
|
timothy65535
|
0724f444c5
|
[KYUUBI #1310] Remove redundant store parameter
### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> The `store` is a member variable of `SparkSQLEngine`, we can get it from `SparkSQLEngine` directly. ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1310 from timothy65535/patch. Closes #1310 1a97a1c7 [root] trigger rebuild eae0fe34 [timothy65535] Remove redundant parameter Lead-authored-by: timothy65535 <timothy65535@163.com> Co-authored-by: root <root@XTZJ-20211013DV.localdomain> Signed-off-by: Cheng Pan <chengpan@apache.org> |
||
|
Cheng Pan
|
6539731417
|
[KYUUBI #1287] [TEST] Refacotor test hierarchical
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> Change the name to reflect functionalities, restore `SparkQueryTests` both on Engine&Server sides Rename `JDBCTestUtils` to `HiveJDBCTestHelper` Rename `BasicJDBCTests` to `SparkMetadataTests` Rename `HiveJDBCTests` to `HiveMetadataTests` Rename `BasicDeltaJDBCTests` to `DeltaMetadataTests` Rename `BasicHudiJDBCTests` to `HudiMetadataTests` Rename `BasicIcebergJDBCTests` to `IcebergMetadataTests` Rename `KyuubiOperationGroupSuite` to `KyuubiOperationPerGroupSuite` Rename sorts of `spark` prefixed fields in `HiveJDBCTestHelper` Use `UserGroupInformation.createUserForTesting` for `KyuubiOperationPerGroupSuite` Hierarchical ``` - HiveJDBCTestHelper - DeltaMetadataTests - [ENGINE] SparkDeltaOperationSuite - [SERVER] DeltaOperationSuite - HudiMetadataTests - [ENGINE] SparkHudiOperationSuite - [SERVER] HudiOperationSuite - IcebergMetadataTests - [ENGINE] SparkIcebergOperationSuite - [SERVER] IcebergOperationSuite - SparkMetadataTests - [SERVER] KyuubiJdbcDriverHASuite - HiveMetadataTests - [ENGINE] SparkOperationSuite mixin SparkQueryTests - [SERVER] KyuubiOperationHiveCatalogSuite - SparkQueryTests - [ENGINE] SparkOperationSuite mixin HiveMetadataTests - [SERVER] KubernetesJDBCTestsSuite - [SERVER] KyuubiOperationPerServerSuite - [SERVER] KyuubiOperationPerGroupSuite - [SERVER] KyuubiOperationPerUserSuite - [SERVER] KyuubiOperationYarnClusterSuite ... other Suites for specific cases ``` ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1287 from pan3793/ut. Closes #1287 d9a410ca [Cheng Pan] Fix K8s tests 478a60dc [Cheng Pan] Fix scalastyle 91f2a27b [Cheng Pan] Rename BasicQueryTests to SparkQueryTests 40154223 [Cheng Pan] Rename JDBCTestHelper to HiveJDBCTestHelper efb99110 [Cheng Pan] Use UGI createUserForTesting d82b0635 [Cheng Pan] Reset UGI 11927101 [Cheng Pan] Refresh group mappings after update Hadoop conf b21f36e7 [Cheng Pan] Restore lazy val user 25008ad3 [Cheng Pan] Rename `KyuubiOperationGroupSuite` to `KyuubiOperationPerGroupSuite` d71c7e1b [Cheng Pan] Refacotor test hierarchical Authored-by: Cheng Pan <chengpan@apache.org> Signed-off-by: Cheng Pan <chengpan@apache.org> |
||
|
timothy65535
|
14dfd14a93
|
[KYUUBI #1284] Add Kyuubi Query Sesstion Page
### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. -->   ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1285 from timothy65535/1284. Closes #1284 13bffaaf [timothy65535] [KYUUBI #1284] Add Kyuubi Query Sesstion Page Authored-by: timothy65535 <timothy65535@163.com> Signed-off-by: Kent Yao <yao@apache.org> |
||
|
Kent Yao
|
7b3442a709
|
[KYUUBI #660] Add UDF session_user
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> As we now fully support GROUP share level, the session user and the system user can be different, so we now need to add UDF session_user. Also, we fix the system_user to always return the `sparkUser` ### _How was this patch tested?_ - [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1274 from yaooqinn/660. Closes #660 c8ed1f77 [Kent Yao] dead code 20c18640 [Kent Yao] nit 4ab1476c [Kent Yao] nit 081f32f2 [Kent Yao] nit e332ca78 [Kent Yao] Merge branch 'master' into 660 135b058f [Kent Yao] ci 646ddeec [Kent Yao] [KYUUBI #660] Add UDF session_user Authored-by: Kent Yao <yao@apache.org> Signed-off-by: Kent Yao <yao@apache.org> |
