### _Why are the changes needed?_
Add metrics for user opened connections with session type
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4364 from turboFei/user_session_type.
Closes#4364
d9505f256 [fwang12] get
Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: fwang12 <fwang12@ebay.com>
### _Why are the changes needed?_
Support to refresh the unlimited users for session limiter, so that we can unblock some customers without restart the kyuubi server.
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4360 from turboFei/limier_whitelist.
Closes#4360
c846148bd [fwang12] typo
839e71365 [fwang12] nit
904bc9fbc [fwang12] save
Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: fwang12 <fwang12@ebay.com>
### _Why are the changes needed?_
Fix broken link in `kerberos.rst`
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4361 from Yikf/kerberos-doc.
Closes#4361
26145902 [Yikf] fix break link
Authored-by: Yikf <yikaifei@apache.org>
Signed-off-by: Yikf <yikaifei@apache.org>
### _Why are the changes needed?_
- add config type `userDefaultsConf` (which is introduced in #3982 ) for config refreshing of admin tool
<img width="784" alt="image" src="https://user-images.githubusercontent.com/1935105/219381858-76b5d1dd-2a49-4fd7-bd3a-06178b6871e9.png">
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4346 from bowenliang123/doc-refresh-userconfigs.
Closes#4346
3ada9db4 [liangbowen] add config type `userDefaultsConf` for `kyuubi-admin refresh config`
cc3df715 [liangbowen] add config type `userDefaultsConf` for `kyuubi-admin refresh config`
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
It can help to know the backend pressure if the exec pool is full.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4344 from turboFei/wait_queue.
Closes#4344
161e3808a [fwang12] nit
6d122e238 [fwang12] save
55a4b499d [fwang12] version
668ff8bfe [fwang12] save
9f56b98a8 [fwang12] save
a401771ec [fwang12] wait
Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: fwang12 <fwang12@ebay.com>
### _Why are the changes needed?_
Bump up markdown version from 3.3.7 to 3.4.1, the sphinx-markdown-tables has been upgraded to 0.0.17 in which the regression is resolved. It now conflicts with markdown 3.3.7
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4340 from yaooqinn/markdown.
Closes#4340
da950a74 [Kent Yao] Bump up markdown version form 3.3.7 to 3.4.1
Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Supply a FTS connector doc in Hive
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4337 from a49a/hive-fts-doc.
Closes#3085
7ec5b73f [Luning Wang] [KYUUBI #3085][DOCS] Add Hive FTS Connector
Authored-by: Luning Wang <wang4luning@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Add Publish docker image step. The step details is maintained by docker repo.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4324 from ulysses-you/release.
Closes#4324
0f6a6646 [ulysses-you] Add Publish docker image step
Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: ulyssesyou <ulyssesyou@apache.org>
### _Why are the changes needed?_
The changes are needed to make doc page clearer and a bit nicer, and reflect latest changes in the chart.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
This patch had conflicts when merged, resolved by
Committer: Cheng Pan <chengpan@apache.org>
Closes#4314 from dnskr/update_helm_chart_doc_page.
Closes#4314
d7c18fd53 [dnskr] [DOCS][HELM] Refine helm chart docs
Authored-by: dnskr <dnskrv88@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Add a GA Job For Documentation Verification
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4269 from yaooqinn/docga.
Closes#4269
c9b39de6 [Cheng Pan] Update .github/workflows/docs.yml
db2d37f8 [Kent Yao] Update .github/workflows/docs.yml
c5bc8518 [Kent Yao] trigger ga
6340129b [Kent Yao] trigger ga
fafbf847 [Kent Yao] trigger ga
52f23240 [Kent Yao] trigger ga
1e0000a7 [Kent Yao] Add a GA Job For Documentaion Verification
Lead-authored-by: Kent Yao <yao@apache.org>
Co-authored-by: Cheng Pan <pan3793@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- as pyspark is supported in #3780, exposing PYTHON option for `kyuubi.operation.language` as an experimental feature
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4256 from bowenliang123/doc-language-python.
Closes#4256
d10c0281 [liangbowen] update description
d05cedda [liangbowen] add PYTHON option in kyuubi.operation.language
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
- Prefer `https://` URLs in docs, and all changed URLs are validated.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4235 from bowenliang123/https-link.
Closes#4235
f114dde2 [liangbowen] update AllKyuubiConfiguration
ad8aaedf [liangbowen] style
e973be5a [liangbowen] update
2370f4bf [liangbowen] prefer https URLs in docs
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
This PR proposes to enhance the LDAP support, which mainly referring the code introduced in HIVE-14713.
Currently, Kyuubi has very limited LDAP support, and the implementation is from the early Hive codebase. Hive enhanced the LDAP support in later versions, considering the Hive ecosystem is quite mature, I think we'd better to porting this functionality and keep the same behavior w/ Hive first, and we can improve it if meet certain requirements/issues in the future.
Basically, this PR introduces the following configurations
```
kyuubi.authentication.ldap.url (since 1.0.0)
kyuubi.authentication.ldap.domain (since 1.0.0)
kyuubi.authentication.ldap.guidKey (since 1.2.0)
kyuubi.authentication.ldap.base.dn (since 1.0.0 deprecated)
kyuubi.authentication.ldap.baseDN
kyuubi.authentication.ldap.groupMembershipKey
kyuubi.authentication.ldap.userMembershipKey
kyuubi.authentication.ldap.groupClassKey
kyuubi.authentication.ldap.groupDNPattern
kyuubi.authentication.ldap.userDNPattern
kyuubi.authentication.ldap.groupFilter
kyuubi.authentication.ldap.userFilter
kyuubi.authentication.ldap.customLDAPQuery
kyuubi.authentication.ldap.binddn
kyuubi.authentication.ldap.bindpw
```
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
This PR ports all LDAP-related UT&IT from Hive codebase
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4152 from pan3793/ldap.
Closes#4152
d251c959 [Cheng Pan] nit
6d14f44b [Cheng Pan] nit
6b3d116c [Cheng Pan] nit
ab47d822 [Cheng Pan] nit
a56e8702 [Cheng Pan] nit
4624619a [Cheng Pan] nit
b82c0c05 [Cheng Pan] LDAP test password uses alphanumeric
86a01cca [Cheng Pan] Enhance LDAP authentication
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- fix word spelling typos in docs
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4226 from bowenliang123/doc-word-typo.
Closes#4226
393de90d [liangbowen] update
365cdc4b [liangbowen] fix word typos in docs
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
- Add docs for the `fast` maven profile
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4225 from bowenliang123/doc-fast-profile.
Closes#4225
29aff164 [liangbowen] update
22ebef61 [liangbowen] typo
24a65920 [liangbowen] nit
803b86ad [liangbowen] doc for the `fast` maven profile
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
Trace statement exec time histogram to provide more insights.
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4214 from turboFei/sql_exec.
Closes#4214
12f080597 [fwang12] remove min > 0, incase it is 0
1edd11c8c [fwang12] scale out
68ebe52d8 [fwang12] reforamt
73eb652ca [fwang12] operation.exec_time
41e7da736 [fwang12] reformat
3d071c6c8 [fwang12] docs
6c3e4abac [fwang12] ut
0b0fbd465 [fwang12] save
11bca028a [fwang12] histo
Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: fwang12 <fwang12@ebay.com>
### _Why are the changes needed?_
- Show line number and assertion when verifying generated markdown files, e.g. `settings.md` from `AllKyuubiConfiguration`, for quicker locating the place and differences in files.
- updated hint message to `mvn clean test` instead of `mvn clean install` for faster verification or regeneration
Hints with line num as below if assertion fails in line comparison.
<img width="1121" alt="image" src="https://user-images.githubusercontent.com/1935105/215451115-813b90f0-9d9d-4ebd-974e-8a071424aa42.png">
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4212 from bowenliang123/markdown-verify-linenum.
Closes#4212
82791a88 [liangbowen] style
609c3b35 [liangbowen] update hints
ed889915 [liangbowen] nit
c676653b [liangbowen] updated hints of markdown generation for AllKyuubiConfiguration and KyuubiDefinedFunctionSuite
646832a0 [liangbowen] show line number and content of expected and got when verifying markdown files
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
- Reformat markdown file `rest-api.md` with changes from (https://github.com/apache/kyuubi/pull/4199) in with ( https://github.com/apache/kyuubi/pull/4200)
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4210 from bowenliang123/4199-followup.
Closes#4199
f66cf8ee [liangbowen] reformat changes in markdown files
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
- to consolidate styles in markdown files from manual written or auto-generated
- apply markdown formatting rules with flexmark from [spotless-maven-plugin](https://github.com/diffplug/spotless/tree/main/plugin-maven#markdown) to *.md files in `/docs`
- use `flexmark` to format markdown generation in `TestUtils` of common module used by `AllKyuubiConfiguration` and `KyuubiDefinedFunctionSuite`, as the same way in `FlexmarkFormatterFunc ` of `spotless-maven-plugin` using with `COMMONMARK` as `FORMATTER_EMULATION_PROFILE` (https://github.com/diffplug/spotless/blob/maven/2.30.0/lib/src/flexmark/java/com/diffplug/spotless/glue/markdown/FlexmarkFormatterFunc.java)
- using `flexmark` of` 0.62.2`, as the last version requiring Java 8+ (checked from pom file and bytecode version)
```
<markdown>
<includes>
<include>docs/**/*.md</include>
</includes>
<flexmark></flexmark>
</markdown>
```
- Changes applied to markdown doc files,
- no style change or breakings in built docs by `make html`
- removal all the first blank in licences and comments to conform markdown style rules
- tables regenerated by flexmark following as in [GitHub Flavored Markdown](https://help.github.com/articles/organizing-information-with-tables/) (https://github.com/vsch/flexmark-java/wiki/Extensions#tables)
### _How was this patch tested?_
- [x] regenerate docs using `make html` successfully and check all the markdown pages available
- [x] regenerate `settings.md` and `functions.md` by `AllKyuubiConfiguration` and `KyuubiDefinedFunctionSuite`, and pass the checks by both themselves and spotless check via `dev/reformat`
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4200 from bowenliang123/markdown-formatting.
Closes#4200
1eeafce4 [liangbowen] revert minor changes in AllKyuubiConfiguration
4f892857 [liangbowen] use flexmark in markdown doc generation
8c978abd [liangbowen] changes on markdown files
a9190556 [liangbowen] apply markdown formatting rules with `spotless-maven-plugin` to markdown files with in `/docs`
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
- add docs in REST API for POST /batches with uploading resource, which was introduced in #4106 .
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4199 from bowenliang123/rest-batchupload-doc.
Closes#4199
957249d4 [liangbowen] add doc for POST /batches (with uploading resource)
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
Trace connection metrics with session type.
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4204 from turboFei/conn_type_open.
Closes#4204
67f45fd5d [fwang12] add ut
ef0bed665 [fwang12] save
ed4be0233 [fwang12] align
b29396306 [fwang12] order
94cdc5bfc [fwang12] order
91bee9082 [fwang12] docs
37128668a [fwang12] save
6cef34f10 [fwang12] session type
6a73760a8 [fwang12] reuse
Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: fwang12 <fwang12@ebay.com>
### _Why are the changes needed?_
Add catalog tree for `kyuubi.authentication` options
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
<img width="746" alt="image" src="https://user-images.githubusercontent.com/26535726/212236908-294b7fa9-e3db-4730-8c16-41c8fd409ae8.png">
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4159 from pan3793/auth-doc.
Closes#4159
8d65768d [Cheng Pan] [DOCS] Refine doc for kyuubi.authentication
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4198 from a49a/fts-typo.
Closes#4197
306b16b2 [Luning Wang] [KYUUBI #4197][DOCS] Fix typo in a title of the Flink FTS Connector
Authored-by: Luning Wang <wang4luning@gmail.com>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
As Kyuubi graduated as top level project, the setting page will be more often requested and should be increasingly reliable and readable with less grammar and spelling mistakes.
This PR is to
- correct mistakes in grammar, spelling, abbreviation and terminology
- with no config name or essential meanings changed
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4161 from bowenliang123/conf-grammar.
Closes#4161
038edfbea [liangbowen] nit
1ec073a4b [liangbowen] to JSON
4f5259a32 [liangbowen] to Prometheus
523855008 [liangbowen] to K8s
fc7a3a81e [liangbowen] to AUTO-GENERATED
da64f54fa [liangbowen] update
d54f9a528 [liangbowen] fix `comma separated` to `comma-separated`
f1d7cc1f1 [liangbowen] update
d84208844 [liangbowen] update
1b75f011c [liangbowen] correction of grammar and spelling mistakes
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
Some links of the website are not available.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4165 from yuruguo/update_doc-url.
Closes#4165
f7a847c3 [Ruguo Yu] [DOCS] Update the docs link
Authored-by: Ruguo Yu <jiang7chengzitc@163.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
We should put the authorization info in request header, user,password,ip in SessionOpenRequest are unnecessary.
### _How was this patch tested?_
- [X] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3739 from hddong/rm-unused-params.
Closes#3739
7fac61853 [hongdongdong] update
8f1f4ae01 [hongdongdong] added on migration guide
95962b472 [hongdongdong] fix
63606350b [hongdongdong] [REST] Remove unused parameters in SessionOpenRequest
Authored-by: hongdongdong <hongdd@apache.org>
Signed-off-by: fwang12 <fwang12@ebay.com>
### _Why are the changes needed?_
Remove improper docs saving dataframe for pyspark, as hive-like JDBC driver not supporting `addBatch` method which is required by Spark JDBC datasource in `JDBCUtils`.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4133 from bowenliang123/pyspark-remove-doc.
Closes#4133
45c9aa7f [liangbowen] remove improper docs saving dataframe for pyspark,as hive like JDBC driver not supporting `addBatch` which is reuqired by Spark JDBC datasource in `JDBCUtils`
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
The changelogs are exported from the git command, which is quite duplicated and does not help much, after removing, users still can get them from GitHub or git history easily.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4094 from pan3793/changelog.
Closes#4094
2d0bfe07 [Cheng Pan] [INFRA] Remove changelogs and generate_changelog.sh
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
This is part of https://github.com/apache/kyuubi/issues/4020, and should finalize the update for repo https://github.com/apache/kyuubi
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4089 from pan3793/graduate.
Closes#4089
60ece0c1 [Cheng Pan] [INFRA] Update release scripts and templates because of graduation
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- DISCLAIMER file on top-level folder of project was already removed after graduation in (https://github.com/apache/kyuubi/issues/4020)
- fix build/dist for `cp` non-existed DISCLAIMER file
- remove DISCLAIMER file reference in dev/dist script, doc
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4086 from bowenliang123/desclaimer-ref.
Closes#4086
18be7489 [liangbowen] remove DISCLAIMER file reference in dev/dist script, doc, docker , labeler.yml
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
After some offline discussions, I propose to change the configuration key from`kyuubi.operation.result.codec` to `kyuubi.operation.result.format`.
### _How was this patch tested?_
Pass CI.
Closes#4075 from cfmcgrady/arrow-conf-rename.
Closes#4075
5ad45507 [Fu Chen] fix
214b43fd [Fu Chen] rename
Authored-by: Fu Chen <cfmcgrady@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
fix#4070 ，all commands in alphabetical order
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4072 from jiaoqingbo/kyuubi4070.
Closes#4070
abb62aeb [jiaoqingbo] [KYUUBI #4070] Add missing spark commands to lineage.md
Authored-by: jiaoqingbo <1178404354@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- add doc for`POST /admin/refresh/user_defaults_conf` REST Api , which introduced in #3982
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4077 from bowenliang123/3982-followup.
Closes#3982
a6acb389 [liangbowen] add REST api doc of `POST /admin/refresh/user_defaults_conf`
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4069 from lightning-L/kyuubi-4020.
Closes#4020
97406ca0 [Tianlin Liao] [KYUUBI #4020] remove incubating from kyuubi source code
Authored-by: Tianlin Liao <tiliao@ebay.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
Detect and inject a tag if plan is for writing, then skip doing final stage isolation at query preparation phase.
To make final stage config more flexible with complex Spark application.
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3988 from ulysses-you/final-stage.
Closes#3988
d0f2b622 [ulysses-you] fix
e5351fd5 [ulysses-you] nit
39082b20 [ulysses-you] Final stage config isolation support write only
Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
The changes are needed to be compliant with Helm recommendations:
1. https://helm.sh/docs/topics/charts/#the-chart-file-structure
```
The directory name is the name of the chart (without versioning information).
Thus, a chart describing WordPress would be stored in a wordpress/ directory.
```
2. https://helm.sh/docs/howto/chart_releaser_action/#repository-changes
```
The charts should be placed under /charts directory at the top-level of the directory tree.
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4034 from dnskr/move_helm_chart_to_dedicated_directory.
Closes#4034
b6f4b661 [dnskr] [K8S][HELM] Move helm chart to dedicated directory
Authored-by: dnskr <dnskrv88@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
Update outdated docs.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4024 from QianyongY/features/update-outdated-docs.
Closes#4024
0ad7173f [yongqian] [DOCS] Update the rules documentation for Kyuubi Spark SQL extension
Authored-by: yongqian <yongqian@trip.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
Fix spelling(steaming -> streaming) in kyuubi_ecosystem figure
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3992 from yaooqinn/eco.
Closes#3992
d2d95147a [Kent Yao] Fix spelling in kyuubi_ecosystem pic
c1db38a9e [Kent Yao] Fix spelling in kyuubi_ecosystem pic
390d175fe [Kent Yao] Fix spelling in kyuubi_ecosystem pic
d9538ba38 [Kent Yao] Fix spelling in kyuubi_ecosystem pic
77cac723e [Kent Yao] Fix spelling in kyuubi_ecosystem pic
Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: fwang12 <fwang12@ebay.com>
### _Why are the changes needed?_
Now, we support SQL, Scala and Python.
In this pr, refactor the session type: `SQL` => `INTERACTIVE`.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4011 from turboFei/interactive_session_type.
Closes#4011
e4a4cae88 [fwang12] doc
106019d65 [fwang12] fix ut
7e8fa36c2 [fwang12] please
da4876449 [fwang12] retest
8cfe30b18 [fwang12] SQL => interactive
Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: fwang12 <fwang12@ebay.com>
### _Why are the changes needed?_
Unlike `kyuubi.frontend.bind.port`, which is deprecated and replaced by `kyuubi.frontend.thrift.binary.bind.port`, `kyuubi.frontend.bind.host` is still encouraged to use, and it can be overwritten by `kyuubi.frontend.<protocol>.bind.host`, e.g. `kyuubi.frontend.thrift.binary.bind.host`, `kyuubi.frontend.thrift.http.bind.host`, `kyuubi.frontend.rest.bind.host`, `kyuubi.frontend.mysql.bind.host`
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4005 from pan3793/doc.
Closes#4005
9c3b1277 [Cheng Pan] comments
0ef140e4 [Cheng Pan] Remove deprecated from description of kyuubi.frontend.bind.host
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
close#3905
### _Why are the changes needed?_
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3977 from iodone/kyuubi-3905.
Closes#3977Closes#3905
4e4a22b1 [odone] trino server init and run
Authored-by: odone <odone.zhang@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
After some thought and some offline discussions, I propose to change the configuration key from `kyuubi.engine.pool.balance.policy` to `kyuubi.engine.pool.selectPolicy`.
And in the future if we introduce properties for specific policy, e.g. supposing we have a engine selection policy named `LOAD`, the properties would be
```
kyuubi.engine.pool.selectPolicy.load.abc=xxxx
kyuubi.engine.pool.selectPolicy.load.xyz=xxxx
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3966 from pan3793/rename.
Closes#2887
7c1eda70 [Cheng Pan] address comments
bf3a3863 [Cheng Pan] [KYUUBI #2887][FOLLOWUP] Rename configuration to kyuubi.engine.pool.selectPolicy
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
add two conditions to decide if we should add shuffle.
1. make sure AQE is enabled, otherwise it is no meaning to add a shuffle
2. try to reduce the performance regression if add a shuffle
for condition 2: we do not add shuffle if the original plan does not have shuffle
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3962 from ulysses-you/no-shuffle.
Closes#3962
a084cccc [ulysses-you] address comment
9d0aab1b [ulysses-you] address comment
09fc9b21 [ulysses-you] fix ut
06f249a2 [ulysses-you] Reduce the performance regression
Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
Add `Kyuubi Deploy On Kubernetes ServiceAccount-related` documentation, and explicitly assert serviceAccountName: default as the default configuration in the template file `${KYUUBI_HOME}/docker/kyuubi-pod(deployment).yaml` to make it easier for users to set.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3906 from zwangsheng/doc/kyuubi_on_k8s_service_account.
Closes#3902
e0ed06d6 [Cheng Pan] Update docs/deployment/kyuubi_on_kubernetes.md
6cd29dd0 [zwangsheng] build doc for serviceAccount
Lead-authored-by: zwangsheng <2213335496@qq.com>
Co-authored-by: Cheng Pan <pan3793@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Followup for #3957 comments
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3963 from turboFei/unlimited_user.
Closes#3957
987b71f1e [fwang12] refactor
e0c6ca515 [fwang12] [KYUUBI #3957][FOLLOWUP] Refactor the session connection unlimited user list config
Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: fwang12 <fwang12@ebay.com>
### _Why are the changes needed?_
Close#3887
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3933 from lightning-L/kyuubi-3887.
Closes#3887
65566232 [Tianlin Liao] [KYUUBI #3887] Provide kyuubiServerPrincipal as alias for principal in JDBC parameter
Authored-by: Tianlin Liao <tiliao@ebay.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Prevent batch session & operation closed by kyuubi because of timeout
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3923 from turboFei/batch_idle.
Closes#3923
76b0fe16 [fwang12] use val
310119b0 [fwang12] refactor
61ce2036 [fwang12] use session timeout
52fb0cc4 [fwang12] 1.6.2
7819f2091 [fwang12] save
69ffa316 [fwang12] check batch session timeout
dd210329 [fwang12] add batch op timeout
e304ddb4 [fwang12] never idle batch session and never timeout batch op
Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: fwang12 <fwang12@ebay.com>
### _Why are the changes needed?_
Support dedicated limiter for batch session.
Support user white list.
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3957 from turboFei/batch_session_limiter.
Closes#3957
55db79e4 [fwang12] support white list
940b76bd [fwang12] add batch session limiter
Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: fwang12 <fwang12@ebay.com>
