### _Why are the changes needed?_
Expose the session name in engine tab to make it more clear.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
<img width="1724" alt="image" src="https://user-images.githubusercontent.com/6757692/176633011-7a3e9af3-7949-4a45-a7bc-8debebc2d376.png">
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2976 from turboFei/session_name.
Closes#2976
05975619 [Fei Wang] save
03b577c3 [Fei Wang] expose session name
Authored-by: Fei Wang <fwang12@ebay.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
### _Why are the changes needed?_
This is a follow up of #2850
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2971 from turboFei/remove_on_match_from_default.
Closes#2850
10e6067e [Fei Wang] refactor
929fde7d [Fei Wang] comments
4ad12f11 [Fei Wang] add log4j1
91168b6c [Fei Wang] fix
e5ac285e [Fei Wang] fix
efe789eb [Fei Wang] [KYUUBI #2850][FOLLOWUP] Use log4j2 xml to take replace of log4j2 properties
Authored-by: Fei Wang <fwang12@ebay.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
### _Why are the changes needed?_
Test against delta 2.0.0rc1
https://github.com/delta-io/delta/releases/tag/v2.0.0rc1
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2963 from cfmcgrady/delta-2-preview.
Closes#2963
db3686e7 [Fu Chen] bump delta 2.0.0rc1
Authored-by: Fu Chen <cfmcgrady@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Fix#2918
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2967 from zhouyifan279/2918.
Closes#2918
79800d5b [zhouyifan279] [KYUUBI #2918][Bug] Kyuubi integrated Ranger failed to query: table stats must be specified
b279d2f5 [zhouyifan279] [KYUUBI #2918][Bug] Kyuubi integrated Ranger failed to query: table stats must be specified
fcb1f8a3 [Min Zhao] [KYUUBI #2918][Bug] Kyuubi integrated Ranger failed to query: table stats must be specified
3cab67b1 [Min Zhao] [KYUUBI #2918][Bug] Kyuubi integrated Ranger failed to query: table stats must be specified
cff04d1c [zhouyifan279] [KYUUBI #2918][Bug] Kyuubi integrated Ranger failed to query: table stats must be specified
24aaf81e [zhouyifan279] [KYUUBI #2918][Bug] Kyuubi integrated Ranger failed to query: table stats must be specified
306508f8 [zhouyifan279] [KYUUBI #2918][Bug] Kyuubi integrated Ranger failed to query: table stats must be specified
Lead-authored-by: zhouyifan279 <zhouyifan279@gmail.com>
Co-authored-by: Min Zhao <zhaomin1423@163.com>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
Remove TProtocolVersion from SessionHandle/OperationHandle/Operation classes, and now it only kept in Session
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2966 from yaooqinn/proto.
Closes#2966
f698d7d3 [Kent Yao] [WIP] Remove TProtocolVersion from SessionHandle/OperationHandle
83f3676a [Kent Yao] [WIP] Remove TProtocolVersion from SessionHandle/OperationHandle
10024e9e [Kent Yao] [WIP] Remove TProtocolVersion from SessionHandle/OperationHandle
Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
A minor improvement by using static.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2973 from zhaomin1423/fix_log.
Closes#2973
285ac0a2 [Min Zhao] Decorate LOG in the RetryableRestClient with static final
Authored-by: Min Zhao <zhaomin1423@163.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Throw exception if the metadata update count is zero, in case that the pre insert metadata operation failed.
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2962 from turboFei/retry_on_created.
Closes#2962
3610be91 [Fei Wang] If update count is zero, throw exception
25d957ca [Fei Wang] Revert "If the metadata does not be created successfully, put into retry queue"
24cdc7e0 [Fei Wang] If the metadata does not be created successfully, put into retry queue
Authored-by: Fei Wang <fwang12@ebay.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
### _Why are the changes needed?_
Close#2956
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2958 from lightning-L/kyuubi-2956.
Closes#2956
a88de508 [Tianlin Liao] allow 0 timeout for rest client; increase default timeout
94a57cce [Tianlin Liao] update
d569abe8 [Tianlin Liao] [KYUUBI #2956] Support to config the connection/socket timeout of rest client for kyuubi-ctl
Authored-by: Tianlin Liao <tiliao@ebay.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
### _Why are the changes needed?_
1. bugfix for issue #2960
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2961 from CavemanIV/kyuubi-2960.
Closes#2960
8aa38eb9 [zhangliang] TFrontendService always use newest hive cli TProtocolVersion
25e5110e [zhangliang] [KYUUBI #2960] TFrontendService.SERVER_VERSION shall be HIVE_CLI_SERVICE_PROTOCOL_V11
Authored-by: zhangliang <zhangliang@trip.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
### _Why are the changes needed?_
We have to use canonical host name as kyuubi instance, otherwise, the ip is always change after pod restart, and the kyuubi instance can not get recovered.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2957 from turboFei/kyuubi_instance_url.
Closes#2957
ac8d9df4 [Fei Wang] respect FRONTEND_CONNECTION_URL_USE_HOSTNAME
c4dcd7d3 [Fei Wang] use getCanonicalHostName for EmbeddedZookeeper
c183b41f [Fei Wang] use getCanonicalHostName for TFrontendService
59f3c0a6 [Fei Wang] Use getCanonicalHostName
Authored-by: Fei Wang <fwang12@ebay.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
### _Why are the changes needed?_
The Operation Type is only used for converting `TOperationType` through specific Thrift Operation APIs. So we can TOperationType in these APIs directly, such as, during calling TFrontendService.ExecuteStatement, we can set TOperationType.EXECUTE_STATEMENT at the end. There is no need to add it to OperationType.
This PR removes `OperationType` completely to simply `OperationHandle` which benefits other `kyuubi.frontend.protocols`.
The thrift protocol shall be able to be removed in following PRs.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2952 from yaooqinn/opType.
Closes#2952
af00c04a [Kent Yao] address comments
4e4f4341 [Kent Yao] address comments
54a2c4ff [Kent Yao] [WIP] Remove Operation Type
ba08ccaf [Kent Yao] [WIP] Remove Operation Type
1737176b [Kent Yao] [WIP] Remove Operation Type
8fe00820 [Kent Yao] [WIP] Remove Operation Type
f969d73d [Kent Yao] [WIP] Remove Operation Type
778f8f4f [Kent Yao] [WIP] Remove Operation Type
54f6573c [Kent Yao] [WIP] Remove Operation Type
Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
Refine the CreateBatchCommand.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2955 from lightning-L/fix.
Closes#2955
61ee0d05 [Tianlin Liao] move BatchType to request
be40748d [Tianlin Liao] BatchRequest args fix: need toString operation for different data types
Authored-by: Tianlin Liao <tiliao@ebay.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
### _Why are the changes needed?_
to close#2949
Unquoted the function name in the error `SECOND_FUNCTION_ARGUMENT_NOT_INTEGER` since Spark-3.4.0, for more details, see https://github.com/apache/spark/pull/36693
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2950 from cfmcgrady/kyuubi-2949.
Closes#2949
4d4da0ba [Fu Chen] fix
Authored-by: Fu Chen <cfmcgrady@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
CompositeService is not needed, use AbstractService.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2951 from lightning-L/fix.
Closes#2951
4d563344 [Tianlin Liao] no need to extend CompositeService for MetadataManager
Authored-by: Tianlin Liao <tiliao@ebay.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
### _Why are the changes needed?_
Remove thrift request timeout for KyuubiSyncThriftClient
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2948 from turboFei/remove_request_timeout.
Closes#2948
a9c5c084 [Fei Wang] remove request timeout
Authored-by: Fei Wang <fwang12@ebay.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
### _Why are the changes needed?_
With #2943close#2943
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2944 from zwangsheng/fix/remove_useless_server.
Closes#2943
6656d0ca [zwangsheng] fix test
faac8ff9 [zwangsheng] fix
57677ea5 [zwangsheng] remove
Authored-by: zwangsheng <2213335496@qq.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
CI has a lot of flaky test, it is because we will mark server is started when we call thread.start, but the server (bianry or http) may not be work at that moment. As a result, the client can not connect a started server.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2925 from ulysses-you/correct-start-state.
Closes#2924
b5a30696 [ulysses-you] test
bb4b4a8a [ulysses-you] sleep
Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
Support to reload hadoop conf for KyuubiTHttpFrontendService.
See details in #2742
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2899 from turboFei/http_followup.
Closes#886
c480aa20 [Fei Wang] [KYUUBI #886][FOLLOWUP] Support to reload hadoop conf for KyuubiTHttpFrontendService
Authored-by: Fei Wang <fwang12@ebay.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
### _Why are the changes needed?_
close https://github.com/apache/incubator-kyuubi/issues/2929#issue-1279486176
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2931 from zhaomin1423/test_ranger.
Closes#2929
36218c46 [Min Zhao] [KYUUBI #2929] Kyuubi integrated Ranger does not support the CTAS syntax
d9cca71f [Min Zhao] [Bug] Kyuubi integrated Ranger does not support the CTAS syntax
Authored-by: Min Zhao <zhaomin1423@163.com>
Signed-off-by: Shaoyun Chen <csy@apache.org>
### _Why are the changes needed?_
Reuse the AuthenticationFilter for KyuubiTHttpFrontendService
Close#2907
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2935 from turboFei/http_servelet.
Closes#2935
26ba8e7f6 [Fei Wang] comments
241c047dd [Fei Wang] refactor
7de5ffb5a [Fei Wang] fix
030d5ce98 [Fei Wang] save
f82fbb0e4 [Fei Wang] save
085075430 [Fei Wang] save
Authored-by: Fei Wang <fwang12@ebay.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
### _Why are the changes needed?_
The cipher is not thread-safe.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2933 from turboFei/cipher_not_thread_safe.
Closes#2894
ea3f79c03 [Fei Wang] not thread safe
Authored-by: Fei Wang <fwang12@ebay.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
fix#2927
### _Why are the changes needed?_
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2928 from iodone/dev-2.
Closes#2927
a5cb1908 [odone] [KYUUBI #2927] fixed
Authored-by: odone <odone.zhang@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
Now `SparkConsoleProgressBar` is initialized after constructing `SQLOperationListener`.
Sometimes the user's SQL syntax may fail.
At this time, because there is no `SparkListenerSQLExecutionEnd` event, `SparkConsoleProgressBar` is not cleaned up, resulting in a memory leak.
In this PR, make the following changes
- Added cleanup method
- Some objects use lazy to avoid constructing when SQL syntax is wrong
close#2922
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2923 from cxzl25/KYUUBI-2922.
Closes#2922
42dbdf2f [sychen] fix leak
Authored-by: sychen <sychen@ctrip.com>
Signed-off-by: Shaoyun Chen <csy@apache.org>
### _Why are the changes needed?_
Fix typo and wording for JDBCMetadataStoreConf.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2919 from lightning-L/typo-1.
Closes#2919
f02ffb8a [Tianlin Liao] fix typo and wording for JDBCMetadataStoreConf
Authored-by: Tianlin Liao <tiliao@ebay.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
### _Why are the changes needed?_
Fix typo for mysql metadata schema.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2920 from lightning-L/typo-2.
Closes#2920
5f54b6f5 [Tianlin Liao] fix minor typos
Authored-by: Tianlin Liao <tiliao@ebay.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
### _Why are the changes needed?_
to close#2890, and this issue is related to #2438
Get the db From Sparksession When TableIdentifier's Database Field Is Empty, but view's TableIdentifier would not.
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2900 from packyan/branch-fix-tableindent-without-db.
Closes#2890
93bf96ad [packyan] reformat code.
b672641c [Deng An] Update PrivilegesBuilder.scala
ac41ed54 [Deng An] Update extensions/spark/kyuubi-spark-authz/src/main/scala/org/apache/kyuubi/plugin/spark/authz/PrivilegesBuilder.scala
1db6ef22 [Deng An] Update PrivilegesBuilder.scala
7e27549c [packyan] format code.
b46f4be6 [packyan] format code style
6dcadcf9 [packyan] format import.
c8992238 [packyan] view TableIdnet will not get db from sparksession.
241eebae [packyan] when TableIdentifier's database field is empty, try get the db from spark session.
Lead-authored-by: packyan <packyande@gmail.com>
Co-authored-by: Deng An <36296995+packyan@users.noreply.github.com>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
https://github.com/apache/incubator-kyuubi/issues/2904
This reverts commit e250c4ae35.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2915 from pan3793/revert.
Closes#2915Closes#2000
9c789eb1 [Cheng Pan] Revert "[KYUUBI #2000][DEPS] Bump Hadoop 3.3.2"
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Now for the batch state, we rely on the metadata store, which is only updated by the local kyuubi instance.
But if the kyuubi instance is down or unreachable sometimes, the metadata state will not change and we can not get the correct state, it might cause the client side stuck if it is waiting the batch completion.
So, if the kyuubi instance is not reachable, we need backfill the batch state according to resource manager batch application state it the batch was in RUNNING state.
If the batch was stuck in PENDING state and the kyuubi instance can not get recovered in short-time, we need support to close the batch session and prevent the batch job re-submitted if it get recovered.
In this pr, I introduce a new field `remoteClosed` to present whether it has been marked as CLOSED state by remote kyuubi instance.
The `remoteClosed` should only be updated by remote kyuubi instance and set its value to true, so there is no race condition.
- When opening the batch session in recovery mode, if `remoteClosed` is true, set the state to CANCELED directly
- check the local batch sessions that is marked as remoteClosed periodically and close them.
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2911 from turboFei/across_kill_one.
Closes#2911
d582459c [Fei Wang] remoteClosed => peerInstanceClosed
cd4207c2 [Fei Wang] unify the state
ede40d0a [Fei Wang] Redirect the get batch request
f4b86868 [Fei Wang] save
5faee761 [Fei Wang] save
Authored-by: Fei Wang <fwang12@ebay.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
### _Why are the changes needed?_
Help release in future.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2912 from ulysses-you/release-md.
Closes#2912
3cf2e272 [ulysses-you] release
Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
update new committers number: 2 -> 6
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2913 from ulysses-you/committer.
Closes#2905
e088a5ee [ulysses-you] 6
Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
close https://github.com/apache/incubator-kyuubi/issues/2745#issue-1248428722
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2776 from zhaomin1423/doris_get_tables.
Closes#2745
d0141dde [Min Zhao] fix
de6ea21b [Min Zhao] fix
0ed24bad [zhaomin3] add get tables
86ac8c61 [Min Zhao] get tables
7e7e5479 [Min Zhao] [KYUUBI #2745] [Subtask] DorisSQLEngine - GetTables Operation
Lead-authored-by: Min Zhao <zhaomin1423@163.com>
Co-authored-by: zhaomin3 <zhaomin3@corp.netease.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
Support waitCompletion for submit batch
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2909 from lightning-L/kyuubi-2628-1.
Closes#2628
23fe761f [Tianlin Liao] minor fix
06cd66ff [Tianlin Liao] support waitUntilComplete for submit batch; use kyuubi instance url for log batch/submit batch
Authored-by: Tianlin Liao <tiliao@ebay.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
### _Why are the changes needed?_
This PR moves engine integration tests except to spark engine from `kyuubi-server` module to dedicated IT modules e.g. `kyuubi-hive-it`, then we can spilt different engine test into jobs to speed up CI tes
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
before
<img width="836" alt="image" src="https://user-images.githubusercontent.com/26535726/173265839-5d0f1ce6-e133-4d60-9a95-c95aa16f817f.png">
after
<img width="825" alt="image" src="https://user-images.githubusercontent.com/26535726/173265903-fb01e910-8f78-4091-91da-dca25be4357d.png">
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2827 from pan3793/hive-it.
Closes#2827
df3798f1 [Cheng Pan] Remove Java 11 test for Hive
e1645338 [Cheng Pan] nit
66524226 [Cheng Pan] nit
3aef2cef [Cheng Pan] review
500c9cff [Cheng Pan] MVN_OPT
3b4ad142 [Cheng Pan] nit
daa6e0ab [Cheng Pan] review
3383d4c1 [Cheng Pan] [BUILD][TEST] Decouple integration tests from kyuubi-server
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
For better interaction with JUnit 5 dependencies
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2898 from pan3793/surefire.
Closes#2898
485e792c [Cheng Pan] Disable failIfNoSpecifiedTests
7b4c3219 [Cheng Pan] Bump maven-surefire-plugin 3.0.0-M7
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
For SubmitBatchCommand, reuse the CreateBatchCommand and LogBatchCommand.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2892 from turboFei/reuse_batch_command.
Closes#2628
67ef50d0 [Fei Wang] comments
cd4a2fec [Fei Wang] refactor
c2a52b15 [Fei Wang] comments for doRun & render
37e0efd3 [Fei Wang] comments
d629c470 [Fei Wang] remove duplicate
cd0d6859 [Fei Wang] only return batch report if delete failed
509c731b [Fei Wang] init
Authored-by: Fei Wang <fwang12@ebay.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
### _Why are the changes needed?_
Remove Hive metastore dependencies from Kyuubi Hive JDBC, and minor improve the UT.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2897 from pan3793/jdbc.
Closes#2897
31cb3068 [Cheng Pan] Remove Hive metastore dependencies from Kyuubi Hive JDBC
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
fix#2782
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2878 from jiaoqingbo/kyuubi-2782.
Closes#2782
23b19897 [jiaoqingbo] code review
6cb40195 [jiaoqingbo] code review
90839742 [jiaoqingbo] [KYUUBI #2782] Decouple Kyuubi Hive JDBC from Hive Serde
Authored-by: jiaoqingbo <1178404354@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
close#2854
### _How was this patch tested?_
- [X] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2860 from wForget/KYUUBI-2854.
Closes#2854
3225be7a [wforget] comment
834aedcf [Wang Zhen] update scaladoc
d71c1f26 [wforget] fix checkstyle
da0e92d2 [wforget] [KYUUBI-2854] Add exception field in KyuubiSessionEvent
Lead-authored-by: wforget <643348094@qq.com>
Co-authored-by: Wang Zhen <wangzhen07@qiyi.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
close https://github.com/apache/incubator-kyuubi/issues/2761#issue-1250326421
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2816 from zhaomin1423/2761.
Closes#2761
02c142c6 [Min Zhao] add comment
c3475dba [Min Zhao] [KYUUBI #2761] Flaky Test: engine.jdbc.doris.StatementSuite - test select
Authored-by: Min Zhao <zhaomin1423@163.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
We need support to retry the jdbc requests.
Because we add new jdbc dependency for batch HA, and the database might be in maintenance window.
And there might be master-slave switch, and the database might be unavailable for few minutes.
We need tolerant this issue and unblock the main thread.
BTW, this pr refactor the metadata store naming.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2834 from turboFei/retrying_jdbc_event.
Closes#2834
109a63e70 [Fei Wang] fix docs
a186484be [Fei Wang] refactor the jdbc pool name
cfac9da53 [Fei Wang] rename metadata table name
bb8710ba3 [Fei Wang] comments
1197a098b [Fei Wang] add private package scope
a95694ba9 [Fei Wang] comments
e2f09e784 [Fei Wang] refactor
057eb73c5 [Fei Wang] refactor
88f3ffb6f [Fei Wang] refactor
e0595bdee [Fei Wang] comments
48270e869 [Fei Wang] refactor conf keys
e4f190c5c [Fei Wang] update docs
ed5e90e37 [Fei Wang] add max queues limitation
c5a03c9e2 [Fei Wang] remove getOrCreate
3e39ca94c [Fei Wang] remove
b79fb0954 [Fei Wang] comments
1d309cee0 [Fei Wang] comments
9108a3f0e [Fei Wang] fix docs
5cdffe710 [Fei Wang] fix rat
2e7228fc7 [Fei Wang] fix ut
520236d7b [Fei Wang] rename more
f66d68bcc [Fei Wang] merge into metadata manager
2f70a2f12 [Fei Wang] rename
80e6100fb [Fei Wang] refactor
09cad092a [Fei Wang] add ut
ea4eea74f [Fei Wang] refactor
6b00eb0b3 [Fei Wang] refactor to MetadataRequest
58f1c0587 [Fei Wang] commements
198feb53f [Fei Wang] add more docs
670a23561 [Fei Wang] refactor
a20d54df2 [Fei Wang] waitStateStoreRetryCompletion after super.close closes operation
97a763aef [Fei Wang] refactor RetryingStateStoreRequest
3e136cf34 [Fei Wang] [SUB-TASK][KPIP-4] Support to retry the session state operations if meet transient jdbc issue
Authored-by: Fei Wang <fwang12@ebay.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
### _Why are the changes needed?_
close#2543
### _How was this patch tested?_
- [X] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2882 from wForget/KYUUBI-2543-2.
Closes#2543
14fa002e [Wang Zhen] fix
386c92d0 [wforget] comments
8c1fcea8 [wforget] [KYUUBI-2543] Add `maxPartitionBytes` configuration
Lead-authored-by: wforget <643348094@qq.com>
Co-authored-by: Wang Zhen <wangzhen07@qiyi.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Refine kyuubi-ctl batch commands
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2889 from turboFei/refine_batch_command.
Closes#2628
56bb7e3d [Fei Wang] print current state
d7be6b72 [Fei Wang] print batch report
3c94fb2d [Fei Wang] set debug for doRequest
7c5fd65a [Fei Wang] fix ut
bcd28a74 [Fei Wang] exit if batch job fail
5d732267 [Fei Wang] refactor
4a2bad8e [Fei Wang] refactor the state
9c982190 [Fei Wang] revert --conf
c2b7008d [Fei Wang] fix ut
10778df5 [Fei Wang] print close resp first
a453084e [Fei Wang] support --conf
Authored-by: Fei Wang <fwang12@ebay.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
To close#2859
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2875 from turboFei/kyuubi_2859_retry_delete.
Closes#2859
7908d827 [Fei Wang] revert commands related
23d357dc [Fei Wang] fix ut
76730014 [Fei Wang] print close resp first
76449dad [Fei Wang] support --conf
Authored-by: Fei Wang <fwang12@ebay.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
Bump Spark-3.3.0
### _How was this patch tested?_
Pass CI.
Closes#2888 from cfmcgrady/spark33-release.
Closes#2888
88216059 [Fu Chen] bump spark-3.3.0
Authored-by: Fu Chen <cfmcgrady@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
To close#2708
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2856 from turboFei/open_in_one.
Closes#2708
2fd77951 [Fei Wang] comments
35ae00d4 [Fei Wang] comment
e02c13e3 [Fei Wang] use kyuubiconf
70d96775 [Fei Wang] refactor
cfb1a36b [Fei Wang] refresh token for startup
f55912f5 [Fei Wang] comments
130a3bb3 [Fei Wang] remove credentials to prevent escalate to session conf
d14ebf78 [Fei Wang] fix ut
71ec3c9e [Fei Wang] renew token when opening session
Authored-by: Fei Wang <fwang12@ebay.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
### _Why are the changes needed?_
Help debug rest client CI.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2884 from ulysses-you/rest-client-log4j.
Closes#2668
fc2e39af [ulysses-you] comment
240c90da [ulysses-you] nit
6228cd8d [ulysses-you] Add log4j for test
Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
