### _Why are the changes needed?_
close#5162
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No
Closes#5163 from lsm1/features/kyuubi-5162.
Closes#5162
c5bba8c20 [senmiaoliu] fix
af5e816fb [Kent Yao] Update docs/contributing/code/developer.md
ceb44358d [senmiaoliu] Add statements about contributions co-authored with AIGC in developer guide page
Lead-authored-by: senmiaoliu <senmiaoliu@trip.com>
Co-authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
It was planned but actually delayed, remove this dummy module to save CI and avoid confusing users and release managers.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
Closes#5113 from pan3793/remove-kudu.
Closes#5113
ff8fd2e6a [Cheng Pan] Remove Spark Kudu connector
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
The main change is copy code from `kyuubi-extension-spark-common_2.12` and `kyuubi-extension-spark-3-3`. The reason copy `kyuubi-extension-spark-common_2.12` is that, Spark3.4 v1 writes does not have ctas...
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
Closes#5018 from ulysses-you/extension.
Closes#5018
1ca4951e1 [ulysses-you] fix
fb20bb1f0 [ulysses-you] fix
338a3120a [ulysses-you] rebase
241a436fd [ulysses-you] fix
70618f368 [ulysses-you] fix
5f5e2b7c8 [ulysses-you] fix
301ae5c04 [ulysses-you] fix
bb9416811 [ulysses-you] fix
a38595d0a [ulysses-you] copy
9dec27d6c [ulysses-you] copy
Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: ulyssesyou <ulyssesyou@apache.org>
### _Why are the changes needed?_
The run test link in the template is broken.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4976 from mans2singh/ISSUE-4975.
Closes#4975
593172786 [Cheng Pan] Update .github/PULL_REQUEST_TEMPLATE
83103e26f [Cheng Pan] Update .github/PULL_REQUEST_TEMPLATE
ddec082d4 [mans2singh] ISSUE-4975 - Reverted unrelated change
37342792d [mans2singh] ISSUE-4975 - Fixed run tests link
Lead-authored-by: mans2singh <mans2singh@yahoo.com>
Co-authored-by: Cheng Pan <pan3793@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- fetch versions for NodeJS and Pnpm from Maven properties
- enabled npm dependency caching for global packages and web-ui module, as the step above takes extra time in builds.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4948 from bowenliang123/pnpm-version.
Closes#4948
80b0f6934 [liangbowen] Reuse versions of NodeJS and Pnpm from maven properties
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
Authz plugin's tests now pass with Spark3.4.
` mvn clean install -pl :kyuubi-spark-authz_2.12 -Pspark-3.4 -am -DskipTests
&& mvn test -pl :kyuubi-spark-authz_2.12 -Pspark-3.4`
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4909 from bowenliang123/authz-enabletest-3.4.
Closes#4909
b0bdc106e [liangbowen] enable authz plugin tests with Spark 3.4
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
After previous efforts, we are close to the goal. This is the latest step to to allow Kyuubi run on JDK 17 by adding `--add-open` in bootstrap scripts and test the plugin's parameter.
This PR also enables JDK 17 test in CI, but only for Kyuubi server and Spark(3.3+) engine moudles, because
- Spark supports JDK 17 since 3.3, see SPARK-33772
- Flink does not support JDK 17 until the latest 1.17
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
- Run Kyuubi server on JDK 17
- Run simple Spark (3.3.2) query on JDK 17
- Run `kyuubi-zk-cli` on JDK 17
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4849 from pan3793/jdk-17.
Closes#4849
165e3558c [Cheng Pan] Enable JDK 17 in CI
f5778e7bd [Cheng Pan] Open modules in bootstrap scripts
76b507ffd [Cheng Pan] Open modules in UT
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
This PR aims to migrate the vanilla Zookeeper and Curator to the Kyuubi Shaded Zookeeper. It's the first step to adapting JDK 17.
There is a known issue [ZOOKEEPER-3779](https://issues.apache.org/jira/browse/ZOOKEEPER-3779) that Zookeeper 3.4 client can not run on JDK 14 and above, in https://github.com/apache/kyuubi-shaded/pull/5, we fixed this issue by a surgical.
With the above fixing, zk-3.4 and zk-3.6 clients both work well on JDK 17, we just randomly pick some cases to make sure zk-3.6 is tested
zk-3.4 client supports zk-3.4+ server, but zk-3.6 client only supports zk-3.5+ server; in the meanwhile, zk-3.4 is adopted widely, (CDH 5/6, HDP, EMR created before 2023).
We are sticky to zk-3.4 to ensure that Kyuubi can be out-of-box in the most existing Hadoop cluster but also provide zk-3.6 as an alternative(simply replace the kyuubi-shaded-zk-3.4 jar w/ kyuubi-shaded-zk-3.6, or build w/ -Pzookeeper-3.6) for users who concerns that zk-3.4 is EOL.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4852 from pan3793/shaded-zk.
Closes#4852
d960cc945 [Cheng Pan] remove staging repo
1b3622080 [Cheng Pan] Switch to Kyuubi Shaded Zookeeper
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
add `restore-keys` to restore the most recently maven binary
[github docs](https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows#matching-a-cache-key)
> restore-keys allows you to specify a list of alternate restore keys to use when there is a cache miss on key. You can create multiple restore keys ordered from the most specific to least specific. The cache action searches the restore-keys in sequential order. When a key doesn't match directly, the action searches for keys prefixed with the restore key. If there are multiple partial matches for a restore key, the action returns the most recently created cache.
logs from [github action](https://github.com/apache/kyuubi/actions/runs/5020359871/jobs/9001759790?pr=4856) (after we update the file `pom.xml` hash)
```
Prepare all required actions
Getting action download info
Run ./.github/actions/setup-maven
Run actions/cachev3
Received 0 of 8173297 (0.0%), 0.0 MBs/sec
Received 8173297 of 8173297 (100.0%), 6.5 MBs/sec
Cache Size: ~8 MB (8173297 B)
/usr/bin/tar -xf /home/runner/work/_temp/0e0ca716-97a4-4d83-af3b-9703003cdd33/cache.tzst -P -C /home/runner/work/kyuubi/kyuubi --use-compress-program unzstd
Cache restored successfully
Cache restored from key: setup-maven-e425be8ee60f158fc4f40b3c6b1a51cb2896e5c97dbbe7c61a96d91380342b89
Run build/mvn -v
Using `mvn` from path: /usr/bin/mvn
Apache Maven 3.8.8 (4c87b05d9aedce574290d1acc98575ed5eb6cd39)
Maven home: /usr/share/apache-maven-3.8.8
Java version: 1.8.0_362, vendor: Temurin, runtime: /usr/lib/jvm/temurin-8-jdk-amd64/jre
Default locale: en, platform encoding: UTF-8
OS name: "linux", version: "5.15.0-1037-azure", arch: "amd64", family: "unix"
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4856 from cfmcgrady/ga-cache.
Closes#4856
779e9030e [Fu Chen] Update .github/actions/setup-maven/action.yaml
d663b1a42 [Fu Chen] Revert "test cache restore"
3d3f16013 [Fu Chen] test cache restore
d2d18b613 [Fu Chen] add restore-keys
Authored-by: Fu Chen <cfmcgrady@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- nodejs v18 is the current active LTS version of nodejs since October 2022, while v16 is in maintenance mode and no longer in the active status
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
This patch had conflicts when merged, resolved by
Committer: Cheng Pan <chengpan@apache.org>
Closes#4784 from bowenliang123/nodejs18.
Closes#4784
483c3ac56 [liangbowen] use nodejs v18 in ci builds and maven builds
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- v8 is now the stable version of pnpm
- lockfile v6 is used by default since pnpm v8.0, which improves the readability of the lockfile by removing hashes from package IDs and more resistant to git merge conflicts (https://github.com/pnpm/pnpm/releases/tag/v8.0.0)
- pnpm 8 supports nodejs 16 and 18 (https://pnpm.io/installation#compatibility)
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4783 from bowenliang123/pnpm8.
Closes#4783
8b3db3820 [liangbowen] use pnpm v8 and update lock file to v6
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Guidelines for Document and Code Contributions.
- code style
- documentation style
- quick starts
- sphinx-copybutton for copy button in code blocks
- add kind:minor,help wanted,good first issue to doc issue template
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
<img width="1325" alt="image" src="https://user-images.githubusercontent.com/8326978/234193368-473e2a1d-2deb-4d91-901a-19cea58a489b.png">
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4768 from yaooqinn/guide.
Closes#4768
3a0bd395b [Kent Yao] Update docs/contributing/doc/build.rst
8fe4f6d14 [Kent Yao] Update docs/contributing/doc/build.rst
a753bdde5 [Kent Yao] Guidelines for Document and Code contributions
f8393fe8d [Kent Yao] Guidelines for Document and Code contributions
8f2d3ce82 [Kent Yao] Guidelines for Document and Code contributions
4f3a2128b [Kent Yao] Guidelines for Document and Code contributions
Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- Add CI for Spark 3.4
- Kyuubi Spark TPC-DS/H connectors support Spark 3.4
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4741 from pan3793/spark-3.4.
Closes#4741
84a2d6ad7 [Cheng Pan] log
b9b2ec1fb [Cheng Pan] Add spark-3.4 profile
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- introduce a custom action for installing and caching Maven, to prevent repeatedly downloading Maven binary from remote.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4747 from bowenliang123/cache-maven.
Closes#4747
bbd9e1f4c [liangbowen] fix cache path
dc342180e [liangbowen] add action for caching maven in ci builds
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
This PR aims to remove Hudi integration tests from the Kyuubi project.
Actually, there is no obvious benefit to running Hudi tests w/ Kyuubi, since the real work happens on the compute engine and Hudi integration. Besides, Hudi's horrible dependency management brings significant maintenance efforts to the Kyuubi community.
This change only affects tests, does not affect any functionality.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4744 from pan3793/remove-hudi.
Closes#4744
ea99f747e [Cheng Pan] Remove Hudi integration tests
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Fixes#4712
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4718 from anurag-rajawat/upgrade-spark.
Closes#4712
79dcf1b79 [Anurag Rajawat] Bump Spark from 3.2.3 to 3.2.4
Authored-by: Anurag Rajawat <anuragsinghrajawat22@gmail.com>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
- super-liner action v5.0.0 release notes: https://github.com/github/super-linter/releases/tag/v5.0.0
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4695 from bowenliang123/superlinter-v5.
Closes#4695
7cd3095a9 [liangbowen] update
7826c8043 [liangbowen] update
89e7b3d7d [liangbowen] bump superlinter to v5
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
Fixes#4712
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4718 from anurag-rajawat/upgrade-spark.
Closes#4712
79dcf1b79 [Anurag Rajawat] Bump Spark from 3.2.3 to 3.2.4
Authored-by: Anurag Rajawat <anuragsinghrajawat22@gmail.com>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
- add ut to check generated Ranger policy file in #4585
- manually activated `genpolicy` profile in CI builds, as the property based activation not auto-triggered as expectedly with property `ranger.version=2.4.0` set in project parent pom
- Support regenerated policy file within the same test suite, by running
`KYUUBI_UPDATE=1 build/mvn clean test -pl :kyuubi-spark-authz_2.12 -Dtest=none -DwildcardSuites=org.apache.kyuubi.plugin.spark.authz.gen.PolicyJsonFileGenerator -Pgenpolicy`
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4677 from bowenliang123/authz-check-policy-gen.
Closes#4677
a372bdfd4 [liangbowen] remove unnecessary profile used in style workflow
7562c88f2 [liangbowen] include in spotless
37b674223 [liangbowen] update policy id
724ec5e28 [liangbowen] replace counter by using zipWithIndex
d322980e7 [liangbowen] extract KRangerPolicyResource object to simplify resource assembly
42c37606a [liangbowen] nit
18a8f4c51 [liangbowen] add usage comments
4ee254d6d [liangbowen] fix issue name in docs
d3cb08d21 [liangbowen] improve file reading
37e4c9c9f [Bowen Liang] Merge branch 'master' into authz-check-policy-gen
6366c50e4 [liangbowen] rename profile to `gen-policy` and remove activation rule by property setting
892faf5ef [liangbowen] update clue
266baa71a [liangbowen] update
cb94e8014 [liangbowen] update
de1f36531 [liangbowen] cleanup
e88c75d46 [liangbowen] check policy file gen
Lead-authored-by: liangbowen <liangbowen@gf.com.cn>
Co-authored-by: Bowen Liang <bowenliang@apache.org>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
Current Labeler mis-tags `module:server` to changes about web-ui.
We should give `module:ui` tag to ui-related PR.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4632 from zwangsheng/ci/labeler_tag_ui.
Closes#4632
33e379d5e [zwangsheng] to low case
801fdc95a [zwangsheng] Add rule fot labeler tag UI
Authored-by: zwangsheng <2213335496@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
bugfix for issue template
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4638 from yaooqinn/yaooqinn-patch-1.
Closes#4638
a4f5f569b [Kent Yao] Update dependency.yml as label in not permitted in markdown block
Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
After pnpm released 8.X version
We should specify pnpm version when npm install.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
- [x] CI with Web UI Build Check & Web UI Style Check
Closes#4626 from zwangsheng/fix/web_ui_ci.
Closes#4626
f7fdc0676 [Cheng Pan] Update .github/workflows/web-ui.yml
61bc10124 [Cheng Pan] Update .github/workflows/style.yml
7d9353218 [zwangsheng] Specify pnpm version
d678c6cb6 [zwangsheng] for feature version
6d83c2b01 [zwangsheng] revert
bbc4b4c2c [zwangsheng] Specify pnpm version
Lead-authored-by: zwangsheng <2213335496@qq.com>
Co-authored-by: Cheng Pan <pan3793@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- Flink 1.15.4 release note: https://flink.apache.org/2023/03/15/apache-flink-1.15.4-release-announcement/
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4598 from bowenliang123/flink-1.15.4.
Closes#4598
40ea2396f [liangbowen] bump flink-1.15 from 1.15.3 to 1.15.4
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
As discussed before, Kyuubi is going to support the latest 3 Flink versions, and to reduce the complexity of supporting Flink 1.17 https://github.com/apache/kyuubi/pull/4368, we are going to remove support Flink 1.14 first.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4588 from pan3793/rm-flink-1.14.
Closes#4387
97d263324 [Cheng Pan] Remove support for Flink 1.14
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- introduce `action/super-linter` with multiple linters, (https://github.com/marketplace/actions/super-linter)
- using the smaller size version of image `github/super-linter/slim` for faster image pulling
- enable linting for `bash_exec` for checking executable of bash scripts, with fixed `tools/spark-block-cleaner/kubernetes/docker/entrypoint.sh`
- enable integrity checks for JSON files in JSONC style
- enable integrity checks for XML files
- enable linting for ENV files
### _How was this patch tested?_
- [x] Pass CI jobs
Closes#4295 from bowenliang123/superlinter-action.
Closes#4295
321d24912 [liangbowen] update
497bf801c [liangbowen] use JSONC instead of JSON to support comments
43787e233 [liangbowen] use superlinter
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
- Setup and use cached `build/mvn` in CI jobs, to prevent repeatedly downloading mvn from remote
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4481 from bowenliang123/setup-maven.
Closes#4481
3906430da [liangbowen] ignore occasional mvnd failure
366806256 [liangbowen] setup cached build/maven
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
Add `Kyuubi Kubernetes IT` Module to CI style check
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
- [X] Run CI
Closes#4461 from zwangsheng/ci/add_kubernetes.
Closes#4461
7ebac75b0 [zwangsheng] fix style for kubernetes-it
c1754d452 [zwangsheng] Add Kyuubi Kubernetes IT Module to CI style check
Authored-by: zwangsheng <2213335496@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- concurrent execution and smart builder is shipped with `mvnd`, but the CI log shows they are not activated
- increase maximum degree of concurrency and utilize `SmartBuilder` feature in `mvnd`. `-Dmvnd.minThreads` is set as fallback and it will be ignored if `--threads` or `-Dmvnd.threads` is used.
#### Before:
(https://github.com/apache/kyuubi/actions/runs/4276652363/jobs/7444896450#step:6:64)
```
[INFO] Build maximum degree of concurrency is 1
```
#### After:
(https://github.com/apache/kyuubi/actions/runs/4279322563/jobs/7449912132#step:8:64)
```
[INFO] Using the SmartBuilder implementation with a thread count of 4
[INFO] Build maximum degree of concurrency is 4
```
(https://github.com/apache/kyuubi/actions/runs/4279322563/jobs/7449912132#step:8:553)
```
[INFO] Segment walltime 13 s, segment projects service time 36 s, effective/maximum degree of concurrency 2.63/4
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4420 from bowenliang123/mvnd-smart.
Closes#4274
64f0cc53e [liangbowen] update
391da26d3 [liangbowen] quite in dep and style jobs
c776356a5 [liangbowen] use -Dmvnd.minThreads
d10bfb70c [liangbowen] use -Dmvnd.minThreads
f793d2b6b [liangbowen] warn
bbb929fe3 [liangbowen] quiet
a5e6d4914 [liangbowen] logging warn level
3cfbaad45 [liangbowen] enable quite option and cancel redirect stderr
09442ff69 [liangbowen] update
ca5f855fa [liangbowen] try to redirect stderr
0ebdd18d4 [liangbowen] set completion and print MAVEN_CLI_OPTS
d3b2c9656 [liangbowen] revert explicitly setting `mvnd.noBuffering`
e2eca0140 [liangbowen] revert explicitly setting `mvnd.noBuffering`
dba25e327 [liangbowen] explicitly set `mvnd.noBuffering` to false prevent displaying events continuously
b7583f936 [liangbowen] shell
d60ebd047 [liangbowen] change mvnd cache key
2cc576ef8 [liangbowen] dep
bbd6414c9 [liangbowen] move multithread setting to mvnd's MAVEN_CLI_OPTS
347b58c8c [liangbowen] increase concurrency for spotless check
15c5519e4 [liangbowen] increase concurrency
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
Correct the Spark image tag to recover "Kyuubi Server On Kubernetes Integration Test"
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4369 from pan3793/spark-3.3.2.
Closes#4338
3232bf9f [Cheng Pan] [KYUUBI #4338][FOLLOWUP] Fix K8s integration tests
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- to avoid violation in Apache archives website's daily total download size quote, prevent repeatedly downloading engine archives by manual cache in actions
### _How was this patch tested?_
- [ ] Pass CI jobs and check cached engine archives
Closes#4348 from bowenliang123/cache-downloaded-archives.
Closes#4348
a9deeea4 [liangbowen] apply engine caching to all jobs in master
d253a497 [liangbowen] Merge commit '38eb74c26ebbdbb57aba51ad18c7e4a6bb9e1144' into cache-downloaded-archives
0cf2a759 [liangbowen] remove conditions for action
38eb74c2 [Bowen Liang] Merge branch 'master' into cache-downloaded-archives
105d507e [liangbowen] remove
c6542797 [liangbowen] extract `cache-engine-archives` action and cache engine archives
Lead-authored-by: liangbowen <liangbowen@gf.com.cn>
Co-authored-by: Bowen Liang <bowenliang@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
according to `io.trino.jdbc.ColumnInfo`, there are some type requring signature parameter.
- varchar(n)
- char(n)
- decimal(precision, scale)
It failed with trino jdbc now
<img width="613" alt="image" src="https://user-images.githubusercontent.com/12025282/218707052-a2e9dc91-0333-483c-bc0a-96baec213578.png">
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4328 from ulysses-you/fix-signature.
Closes#4328
aede5cec [ulysses-you] nit
ae1a7968 [ulysses-you] fix test
8ecdb346 [ulysses-you] Make Trino jdbc driver work
Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: ulyssesyou <ulyssesyou@apache.org>
### _Why are the changes needed?_
- Check whether modules are available by running `mvnd dependency:resolve`. if true, skip the `build` step of `mvnd install`.
- Reducing the time cost of dependency workflow from ~ 10 min to ~4 min ([see log](https://github.com/apache/kyuubi/actions/runs/4162071562/jobs/7200786535)).
### _How was this patch tested?_
- [x] Pass CI workflows
Closes#4315 from bowenliang123/dep-check-first.
Closes#4315
7c20ea24 [liangbowen] nit
65397868 [liangbowen] add validate to bring back enforcer check
1c035136 [liangbowen] add validate to bring back enforcer check
d4e703de [liangbowen] update dep.yml
8f8d0457 [liangbowen] update dep.yml
e7adfacf [liangbowen] update
c88eff98 [liangbowen] check modules available before build
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
```
...
07:43:26.520 KyuubiSessionManager-exec-pool: Thread-779 INFO ProcBuilder: Logging to /home/runner/work/kyuubi/kyuubi/kyuubi-server/target/work/runner/kyuubi-spark-batch-submit.log.0
...
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4317 from pan3793/ci-log.
Closes#4317
dbde5306 [Cheng Pan] Upload kyuubi-spark-batch-submit.log on CI failure
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: ulyssesyou <ulyssesyou@apache.org>
### _Why are the changes needed?_
This PR proposes to change the publish `master-snapshot` image from every commit of the master branch to the daily scheduled, it saves GA resources.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4297 from pan3793/docker-nightly.
Closes#4297
1424287cb [Cheng Pan] nit
32cffe8ab [Cheng Pan] Daily publish master snapshot docker image
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
This PR updates `pom.xml` and CI workflow to change the default Flink version to 1.16, close#4253
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4289 from pan3793/flink-1.16.
Closes#4253
dc3885a38 [Cheng Pan] [KYUUBI #4253] Change default Flink version to 1.16
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Add a GA Job For Documentation Verification
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4269 from yaooqinn/docga.
Closes#4269
c9b39de6 [Cheng Pan] Update .github/workflows/docs.yml
db2d37f8 [Kent Yao] Update .github/workflows/docs.yml
c5bc8518 [Kent Yao] trigger ga
6340129b [Kent Yao] trigger ga
fafbf847 [Kent Yao] trigger ga
52f23240 [Kent Yao] trigger ga
1e0000a7 [Kent Yao] Add a GA Job For Documentaion Verification
Lead-authored-by: Kent Yao <yao@apache.org>
Co-authored-by: Cheng Pan <pan3793@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
bump the minikube version to 1.29.0 and the kubernetes version to 1.26.1.
After ran some tests for the used action manusa/actions-setup-minikubev2.7.2, it turns out that it would have some problems when I bump the minikube version to 1.29.0, please refer to [this link](https://github.com/runzhliu/yum-with-browser/actions/runs/4113082204/jobs/7098775474) for details.
In addition, I have tried another minikube action medyagh/setup-minikubemaster, it's good to bump the version for both minikube and Kubernetes, please refer to [this link](https://github.com/runzhliu/yum-with-browser/actions/runs/4112397435/jobs/7097254604#step:5:8).
However, I would prefer to [this way](https://github.com/apache/spark/blob/master/.github/workflows/build_and_test.yml#L945-L966) to setup a minikube cluster for the rest Kubernetes related tests, it's also good to set up the cluster as well, please refer to [this link](https://github.com/runzhliu/yum-with-browser/actions/runs/4113018527/jobs/7098637193#step:4:2).
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4248 from runzhliu/patch-1.
Closes#4248
7d12e0e8 [runzhliu] Update master.yml
52448ae0 [runzhliu] Update master.yml
5fb0b9ec [runzhliu] Update master.yml
3491b33a [runzhliu] Merge branch 'apache:master' into patch-1
64101d95 [runzhliu] bump the minikube version to 1.29.0 and the kubernetes version to 1.26.1
4b278697 [runzhliu] Merge branch 'apache:master' into patch-1
b19e0983 [runzhliu] Update master.yml
f6bc18ee [runzhliu] Update master.yml
bb133312 [runzhliu] Update master.yml
b44ad030 [runzhliu] Update master.yml
7f22810f [runzhliu] Update master.yml
09a6dadf [runzhliu] Update master.yml
60623715 [runzhliu] Update master.yml
0d2c6568 [runzhliu] Update master.yml
5c8b0cc6 [runzhliu] bump the minikube version to 1.29.0 and the kubernetes version to 1.26.1
Authored-by: runzhliu <runzhliu@163.com>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
close#4250
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4275 from df-Liu/fix_4250.
Closes#4250
723c4b745 [df_liu] [KYUUBI #4250] fix 1.16.1
0e151a408 [df_liu] [KYUUBI #4250] Bump Flink 1.16.1
Authored-by: df_liu <df_liu@trip.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
updated `.github/workflows/master.yml#L161`
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4284 from df-Liu/follow_4252.
Closes#4252
3b0069418 [df_liu] [KYUUBI #4252][FOLLOWUP] Bump Flink 1.14.6
Authored-by: df_liu <df_liu@trip.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
As mentioned in https://github.com/apache/kyuubi/issues/4139, we are not able to use Travis for AMR testing in the future.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4287 from pan3793/rm-travis.
Closes#4287
efe2a4854 [Cheng Pan] nit
c72a8b00c [Cheng Pan] Remove travis
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Since https://github.com/apache/kyuubi/pull/2175, the previous running CI jobs will be canceled on the same branch or PR to save CI resources, this PR keeps the behavior for PR but doesn't cancel the running CI jobs when commit to the stable branch.
Ref: https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#example-using-a-fallback-value
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
I pushed an empty commit to verify the behavior does not change in PR, and I need to merge this to master first, then merge another PR immediately to verify the rest behaviors.
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4285 from pan3793/ci-con.
Closes#4285
3aa7f427e [Cheng Pan] empty
4156eb196 [Cheng Pan] Do not cancel CI jobs on stable branch
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- Add actions/dependency-review-action (https://github.com/actions/dependency-review-action) as new step in the dependency workflow, which fails when future PR is bringing vulnerable dependencies.
- configuring `fail-on-severity` to `moderate` level
<img width="760" alt="image" src="https://user-images.githubusercontent.com/1935105/217551476-e2858c04-ebcb-4028-a209-e547690b5d79.png">
### _How was this patch tested?_
- [x] Pass the Dependency Check CI job
Closes#4266 from bowenliang123/dependency-review.
Closes#4266
63a84b22 [liangbowen] add actions/dependency-review-action to dependency workflow
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
Keep the same version for Integration Tests on Kubernetes and Minikube.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4234 from runzhliu/master.
Closes#4234
94345659d [runzhliu] Keep the same version for Integration test on Kubernetes
Authored-by: runzhliu <runzhliu@163.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- Bump actions/stale from v6 to v7 (release note:https://github.com/actions/stale/releases/tag/v7.0.0)
- Breaking change in v7 which we didn't use. `no longer managing the stale label on items included in exempt-issue-labels and exempt-pr-labels.` as shown in the link above.
### _How was this patch tested?_
CI action pass.
Closes#4193 from bowenliang123/actions-stale-v7.
Closes#4193
7cf9da7f [liangbowen] bump actions/stale from v6 to v7
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
In 21 CI workflow jobs, 18 run on `ubuntu-22.04` and 3 run on `ubuntu-latest` (the same as `ubuntu-22.04` currently),
Making all CI jobs run on `ubuntu-22.04` to ensure alignment in the same operation system version, and it benefits from future version updates.
### _How was this patch tested?_
CI actions pass.
Closes#4192 from bowenliang123/ci-ubuntu-latest.
Closes#4192
8276c3a3 [liangbowen] run ci jobs on ubuntu-22.04
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
Some links of the website are not available.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4165 from yuruguo/update_doc-url.
Closes#4165
f7a847c3 [Ruguo Yu] [DOCS] Update the docs link
Authored-by: Ruguo Yu <jiang7chengzitc@163.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
To recover the failed style check CI job, "ludeeus/action-shellcheckmaster is not allowed to be used in apache/kyuubi."
The ASF only allows ludeeus/action-shellcheck1.1.0, see https://issues.apache.org/jira/browse/INFRA-24032
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4136 from pan3793/shell-check.
Closes#4136
5604d7118 [Cheng Pan] scandir
2d1b0a1de [Cheng Pan] [INFRA] Pin ludeeus/action-shellcheck@1.1.0
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Fu Chen <cfmcgrady@gmail.com>
### _Why are the changes needed?_
- remove blank element line in labeler.yml added in #4112
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4117 from bowenliang123/4112-followup.
Closes#4112
38d8eb6a [liangbowen] remove blank element line
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- label changes in Authz plugin with `module:authz` label in Github labeler workflow
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4112 from bowenliang123/labler-authz.
Closes#4112
3a3e39aa [liangbowen] add `module:authz` label
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Fix#2062
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4078 from xiaoyuandajian/fix-#2062.
Closes#4078
a5a01945 [Cheng Pan] Update .github/workflows/style.yml
3c08e692 [袁福元] fix the shellcheck found error file
a33cbb22 [袁福元] fix the shellcheck found error file
da195782 [袁福元] fix the shellcheck found error file
b24e6adf [袁福元] fix the shellcheck found error file
90ef0712 [袁福元] fix the shellcheck found error file
42acb31f [袁福元] fixup! test1
31c1783d [袁福元] fixup! test
c40b5f38 [袁福元] Revert "test"
d97801fa [袁福元] Merge remote-tracking branch 'origin/fix-#2062' into fix-#2062
39a9ff31 [袁福元] test
fc7198da [xiaoyuandajian] Update style.yml
62e80726 [袁福元] test
0abdef36 [xiaoyuandajian] Delete main.yml
1e252e2a [xiaoyuandajian] Create main.yml
c19728ec [xiaoyuandajian] Delete main.yml
5db61e5a [xiaoyuandajian] Delete main1.yml
9e27a363 [xiaoyuandajian] Create main1.yml
69540652 [xiaoyuandajian] Create main.yml
Lead-authored-by: 袁福元 <yuanfuyuan@mafengwo.com>
Co-authored-by: xiaoyuandajian <51512358+xiaoyuandajian@users.noreply.github.com>
Co-authored-by: Cheng Pan <pan3793@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
This is part of https://github.com/apache/kyuubi/issues/4020, and should finalize the update for repo https://github.com/apache/kyuubi
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4089 from pan3793/graduate.
Closes#4089
60ece0c1 [Cheng Pan] [INFRA] Update release scripts and templates because of graduation
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- DISCLAIMER file on top-level folder of project was already removed after graduation in (https://github.com/apache/kyuubi/issues/4020)
- fix build/dist for `cp` non-existed DISCLAIMER file
- remove DISCLAIMER file reference in dev/dist script, doc
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4086 from bowenliang123/desclaimer-ref.
Closes#4086
18be7489 [liangbowen] remove DISCLAIMER file reference in dev/dist script, doc, docker , labeler.yml
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4069 from lightning-L/kyuubi-4020.
Closes#4020
97406ca0 [Tianlin Liao] [KYUUBI #4020] remove incubating from kyuubi source code
Authored-by: Tianlin Liao <tiliao@ebay.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
This pr aims to tidy up the `Are you willing to submit PR` option in the issue template.
Currently, The `Are you willing to submit PR` option of some Apache kyuubi issue templates has three options as follow:
- [ ] Yes. I can submit a PR independently to improve.
- [ ] Yes. I would be willing to submit a PR with guidance from the Kyuubi community to improve.
- [ ] No. I cannot submit a PR at this time.
The intent of `Are you willing to submit PR` option is to confirm whether the questioner can submit PR at now, So two options are enough.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4025 from Yikf/issue-template.
Closes#4025
11b08ee8 [Yikf] Tidy up the Are you willing to submit PR option in the issue template
Authored-by: Yikf <yikaifei1@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
CI using `build/dockerfile` to build kyuubi docker image, which will build distribution frist.
We should pass servals maven args to help build speed up.
We only test spark engine for kyuubi on kubernetes it test, so we use hive and flink provided.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
- [x] Wait CI
Closes#3907 from zwangsheng/IT/improve_build_docker_image.
Closes#3907
b05e190b [zwangsheng] fix
22f96805 [zwangsheng] respect
e9bc3047 [zwangsheng] download spark
65686b32 [zwangsheng] pass maven to dockerfile
Authored-by: zwangsheng <2213335496@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Fix the issue that connect conf is not used in the Kyuubi On Kubernetes IT's jdbc connection string.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
- [x] Wait for IT ci
Closes#3869 from zwangsheng/bugfix/kyuubi_on_k8s_it_connect_conf.
Closes#3869
3cecd5f4 [zwangsheng] fix
63025a28 [zwangsheng] fix
924949f4 [zwangsheng] fix
21e93298 [zwangsheng] fix
fc5794ef [zwangsheng] fix
6dca96cd [zwangsheng] test
c2c81bb4 [zwangsheng] test
b8bb820b [zwangsheng] add unit test
dad4c739 [zwangsheng] fix bind
406f1de5 [zwangsheng] proxy for 185
c7d6ee6d [zwangsheng] merge
2290a24f [Binjie Yang] Update master.yml
9ffcb498 [Binjie Yang] Update KyuubiOnKubernetesTestsSuite.scala
714b340d [Binjie Yang] Update KyuubiOnKubernetesTestsSuite.scala
fa7fc542 [Binjie Yang] Update KyuubiOnKubernetesTestsSuite.scala
af4b9881 [zwangsheng] set 777 for /
49f705eb [zwangsheng] set 777 for test
fc66843d [zwangsheng] stop ci
e2ba0bcf [zwangsheng] add test name
7db4eab1 [zwangsheng] fast test cluster
33d490d1 [zwangsheng] add unit test
e2e12f4e [zwangsheng] fast test cluster
e8251011 [zwangsheng] test
b66468f5 [zwangsheng] test
633d99e4 [zwangsheng] change host
40ba5740 [zwangsheng] test
e393f9a5 [zwangsheng] test
532cd7df [zwangsheng] merge
4597572e [zwangsheng] test
b8fc86a1 [Binjie Yang] Update KyuubiOnKubernetesTestsSuite.scala
34be2761 [zwangsheng] TEST
a3c60e45 [zwangsheng] Changes
19e3bc22 [zwangsheng] for fast test
3ad2337f [zwangsheng] try cluster
39df2c40 [zwangsheng] try cluster
ed8f8baa [zwangsheng] fix client
7f711acb [zwangsheng] fix
b034731e [zwangsheng] fix
d646f4ac [zwangsheng] fix
2b9591c4 [zwangsheng] debug
a977d907 [zwangsheng] fix
0c3486fa [zwangsheng] debug
f0a0304b [zwangsheng] Add serviceAccount
eb3424ab [zwangsheng] fix user
cac7e69d [zwangsheng] proxy user
2886520f [zwangsheng] debug
25a677c6 [zwangsheng] debug
9f201d89 [zwangsheng] debug
e533664d [zwangsheng] fix it test
d9bf9173 [zwangsheng] fix it test
Lead-authored-by: zwangsheng <2213335496@qq.com>
Co-authored-by: Binjie Yang <52876270+zwangsheng@users.noreply.github.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
to close#3862 .
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3868 from bowenliang123/3862-codecov-v3.
Closes#3862
b5733979 [liangbowen] bump codecov-action from v2 to v3
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
1. `mvn antrun:runbuild-info -pl kyuubi-common`
2. debug engine suite
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3847 from cxzl25/profile_jdbc_debug.
Closes#3847
e1c32067 [Cheng Pan] Merge branch 'master' into profile_jdbc_debug
f41d3cd9 [sychen] use jdbc-shaded
25b54b18 [sychen] use jdbc-shaded
dd23f864 [sychen] use kyuubi-hive-jdbc as default
79b0ffcb [sychen] add jdbc profile
Lead-authored-by: sychen <sychen@ctrip.com>
Co-authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
to close#3861
I test the `manusa/actions-setup-minikubev2` and failed, so just set `manusa/actions-setup-minikubev2.7.1`
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3878 from xuefeimiaoao/feature/bump_actions_setup_minikube.
Closes#3861
8452fcf4 [xuefeimiaoao] [KYUUBI #3861] Bump manusa/actions-setup-minikube from 2.7.0 to 2.7.1
Authored-by: xuefeimiaoao <1255072085@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
test against Spark-3.2.3
https://www.mail-archive.com/devspark.apache.org/msg29609.html
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3825 from cfmcgrady/spark323.
Closes#3825
a2deabcdf [Fu Chen] release
2fea74cf0 [Fu Chen] spark.archive.name
8245c6490 [Fu Chen] bump spark-3.2.3
Authored-by: Fu Chen <cfmcgrady@gmail.com>
Signed-off-by: Fu Chen <cfmcgrady@gmail.com>
### _Why are the changes needed?_
to close#3828.
Python code style checking support.
1. reuse Spotless maven plugin for Python style check
2. add style check to CI style workflow
3. add python linting support to `dev/reformat`. checks whether `black` installed in PATH.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3823 from bowenliang123/spotless-python.
Closes#3828
4a4de885 [liangbowen] simplify empty tags
0bb9ec7c [liangbowen] simplify empty tag in pom
9dd39531 [liangbowen] lint python code with black via spotless
f85020fa [liangbowen] typo
4c93bce0 [liangbowen] install python 3.9 first
23fc4b96 [liangbowen] ci install black version from added `spotless.python.black.version` property
73f746b0 [Bowen Liang] Update dev/reformat
46667a00 [liangbowen] update style.yml
9c20b434 [liangbowen] update style.yml
21017e5e [liangbowen] update style.yml
8272c0bc [liangbowen] add python style to style checking for CI
e102726c [liangbowen] add profile spotless in dev/reformat if black found in path
062e9bf2 [liangbowen] add python scan for spotless. add new profile `spotless-python` for python file path.
Lead-authored-by: liangbowen <liangbowen@gf.com.cn>
Co-authored-by: Bowen Liang <bowen.liang.123@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Currently, we only store unit-tests.log and kyuubi-spark-sql-engine.log for spark-on-k8s-it ci test.
As there are some batch test in spark-on-k8s-it ci , we should also store kyuubi-spark-batch-submit.log for debug.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
- [x] Run CI Test
Closes#3728 from zwangsheng/ci/storage_batch_engine_log.
Closes#3728
502639f3 [zwangsheng] fixx
Authored-by: zwangsheng <2213335496@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Close https://github.com/apache/incubator-kyuubi/issues/3692
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3694 from Yikf/upgrade-GA.
Closes#3692
3fd2a326 [Yikf] Upgrade actions/setup-java to v3
Authored-by: Yikf <yikaifei1@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Fix https://github.com/apache/incubator-kyuubi/issues/3441;
This pr aims to two points as follow:
- Bump Spark 3.3.0 to Spark 3.3.1 in spark-3.3 profile
- Change default Spark version to 3.3.1
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3507 from Yikf/pr/3471.
Closes#3441
7449280b [yikf] Change spark 3.3.0 as default version
Authored-by: yikf <yikaifei1@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Current kyuubi build kubernetes client for `KyuubiApplicationOperator` to find or kill spark engine on kubernetes.
This pr aims to extract the code for building kubernetes client into `KubernetesUtils` and to do auto build following the logic in Spark.
We prefer to building with user-configured parameters; When missing, the kubernetes client logic will build one from environment variables and kubeconfig.
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3663 from zwangsheng/feature/perfer_build_from_pod_env.
Closes#3663
0a4c8c79 [zwangsheng] clear properties
ce408db0 [zwangsheng] fix test
fb28ea7a [zwangsheng] add ci test
943c68dd [zwangsheng] add unit test
ddc9a2b2 [zwangsheng] fix scala
75380d13 [zwangsheng] comment
e6826609 [zwangsheng] master
683cc260 [zwangsheng] context provided master
7b42133c [zwangsheng] add doc
b56829c5 [zwangsheng] fix helm
5884475d [zwangsheng] rebuild
a7f0b551 [zwangsheng] ad
e7f9ec05 [zwangsheng] add setting
b77bbd9b [zwangsheng] add doc
3f3253d3 [zwangsheng] init
Authored-by: zwangsheng <2213335496@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
This pr aims to improve issue template related to test, It
- remove the flaky test template
- merge flaky test template into the current one as a sub-category of the test issue template
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3533 from Yikf/issue-test-template.
Closes#3533
9fcc6d09 [yikf] Improve test issue template
Authored-by: yikf <yikaifei1@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Fix https://github.com/apache/incubator-kyuubi/issues/3497
This PR is intended to optimize improve GitHub issue template, aiming to add `Are you willing to submit PR?` 's options
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3501 from Yikf/master.
Closes#3497
ea6cf4b5 [yikf] improve issue template
Authored-by: yikf <yikaifei1@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Fix https://github.com/apache/incubator-kyuubi/issues/2088
This pr aims to add a template of improve tests only.
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3496 from Yikf/test-improve-issue.
Closes#2088
7203f632 [yikf] add improve tests tmeplate
Authored-by: yikf <yikaifei1@gmail.com>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
Fix https://github.com/apache/incubator-kyuubi/issues/3402
This pr aims to use [stale](https://github.com/actions/stale) to add a GitHub action workflow to automatically close stale PRs, because stable PRs may no longer get forward or get enough attention
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3479 from Yikf/stale-workflow.
Closes#3402
2127b2ff [yikf] Closing Stale Pull Requests
Authored-by: yikf <yikaifei1@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
This PR proposes to add Testcontainers based Zookeeper 3.4/3.5/3.6/3.7 integration test, we need it to verify the compatibility if we upgrade Zookeeper client to the new version. See details in #1941
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2040 from pan3793/zk-3-4-it.
Closes#2040
929540f6 [Cheng Pan] [TEST] Add Zookeeper 3.4/3.5/3.6 integration test
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Check Kyuubi modules available only when `**/pom.xml` is changed
https://github.com/marketplace/actions/paths-changes-filter#conditional-execution
### _How was this patch tested?_
Pass CI.
Closes#3454 from cfmcgrady/style-ci-followup.
Closes#3454
f849fbd8 [Fu Chen] check on pom change
Authored-by: Fu Chen <cfmcgrady@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
I saw the labeler action does not work with some prs which related to Spark extensions. This pr updates the labeler rules.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3433 from ulysses-you/update-labeler.
Closes#3433
daf78f99 [ulysses-you] Update the labeler rules
Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
For code review and check pr, we need Github CI help check web ui.
Run `npm run coverage && npm run build-check` to find syntax issues and build errors.
Actions used node from https://github.com/marketplace/actions/setup-node-js-environmentClose#3335
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3340 from zwangsheng/feature/web_ci.
Closes#3340
037953b7 [zwangsheng] fix symble |
07f6dc09 [zwangsheng] use npm install pnpm
c643333e [zwangsheng] use pnpm instead
a57bc311 [zwangsheng] install
f2a94ca2 [zwangsheng] fix runner
1d85336d [zwangsheng] fix runner
7c5c58c2 [zwangsheng] fix runner
adb94e46 [zwangsheng] fix
33359d99 [zwangsheng] fix
95a34cbb [zwangsheng] fix
7d3207a9 [zwangsheng] add ci
Authored-by: zwangsheng <2213335496@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
fix: #3327
### _Why are the changes needed?_
The contributing link in the PULL_REQUEST_TEMPLATE is out-of-date, so this patch replaces it with the new contributing link.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3329 from YesOrNo828/Kyuubi-3327.
Closes#3327
badf37b0 [yexianxun] [KYUUBI #3327] Replace the contributing link in the PR template
Authored-by: yexianxun <yexianxun@corp.netease.com>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
Fix https://github.com/apache/incubator-kyuubi/issues/3306
Currently, Kyuubi-AuthZ support Apache Spark 3.0 and above, and there are no CI workflow to check compatibility with Apache Spark 3.0, this pr aim to add a CI workflow for AuthZ module
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3307 from Yikf/GA.
Closes#3306
15ab4b76 [yikf] Add Authz module GA for mutiple spark versions
Authored-by: yikf <yikaifei1@gmail.com>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
We should enable spark-3.3 profile as well to check spark-3.3 specific modules, like kyuubi-spark-hive-connector and kyuubi-spark-kudu-connector.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3316 from pan3793/build.
Closes#3316
e8e1036f [Cheng Pan] nit
bbb30fc7 [Cheng Pan] Build: enable spark-3.3 profile on GA
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
Publish daily snapshot on branch-1.6 and stop to publish on branch-1.4
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3317 from pan3793/daily.
Closes#3317
6d54fe8e [Cheng Pan] [BUILD] Publish daily snapshot on branch-1.6
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
close https://github.com/apache/incubator-kyuubi/issues/3239#issue-1338893480
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3240 from zhaomin1423/jdbc_it.
Closes#3239
8da64f29 [Min] fix it
24cceba2 [Min] add dependency
b477326c [Min] add server to dependency
e74aa93f [Min] fix getJdbcUrl conflict
01e6a302 [Min] [KYUUBI #3239] [Subtask] DorisSQLEngine - Add integration tests
Authored-by: Min <zhaomin1423@163.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
To make sure flink sql engine build against flink-1.15 can run on flink-1.14.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3230 from pan3793/flink.
Closes#3230
5c3f859c [Cheng Pan] updated
e776ee63 [Cheng Pan] review
a5cfb0c8 [Cheng Pan] review
004c05d9 [Cheng Pan] nit
9c019554 [Cheng Pan] nit
7c3f553c [Cheng Pan] nit
8e1fb04b [Cheng Pan] nit
6adc7dbe [Cheng Pan] nit
bc10739c [Cheng Pan] fix
cf98111a [Cheng Pan] workflow
a8895e68 [Cheng Pan] nit
8cab5188 [Cheng Pan] Flink SQL engine supports run across versions
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3207 from deadwind4/flink-default-version.
Closes#3206
4c9b5d6c [Luning Wang] Fix test case to fit flink 1.14 and 1.15
09c854e2 [Luning Wang] change sql-client.jar name to fit flink 1.15
eef06d1e [Luning Wang] change the suffix to fit 1.15
93fb44f4 [Luning Wang] Comment out cross validation
8ea10218 [Luning Wang] Fix cross validation
a5e5cdb9 [Luning Wang] Fix github CI Flink version
1ee8a5d9 [Luning Wang] [KYUUBI #3206] Change Flink default version to 1.15
Authored-by: Luning Wang <wang4luning@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Close#2584
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2585 from deadwind4/bump-ubuntu.
Closes#2584
050f1b9a [Luning Wang] Bump actions-setup-minikub to 2.7.0
51a77ddb [Ada Wang] [KYUUBI #2584][INFRA] Migrate CI to Ubuntu 22.04
Lead-authored-by: Ada Wang <wang4luning@gmail.com>
Co-authored-by: Luning Wang <wang4luning@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Fix failed test cases
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3173 from deadwind4/flink115-test.
Closes#3172
c9428fae [Luning Wang] Fix github ci conf
966a1bfb [Luning Wang] Remove assume and fix select row
8d77fbf3 [Luning Wang] Add flink version check
8c584ef9 [Luning Wang] delete unuseful import
ee4dc0fa [Luning Wang] Fix github ci for flink 1.15
1717dc6d [Luning Wang] Fix github ci for flink 1.15
026f4073 [Luning Wang] [KYUUBI #3172][FLINK] Fix failed test cases in Flink 1.15
Authored-by: Luning Wang <wang4luning@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
This PR moves engine integration tests except to spark engine from `kyuubi-server` module to dedicated IT modules e.g. `kyuubi-hive-it`, then we can spilt different engine test into jobs to speed up CI tes
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
before
<img width="836" alt="image" src="https://user-images.githubusercontent.com/26535726/173265839-5d0f1ce6-e133-4d60-9a95-c95aa16f817f.png">
after
<img width="825" alt="image" src="https://user-images.githubusercontent.com/26535726/173265903-fb01e910-8f78-4091-91da-dca25be4357d.png">
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2827 from pan3793/hive-it.
Closes#2827
df3798f1 [Cheng Pan] Remove Java 11 test for Hive
e1645338 [Cheng Pan] nit
66524226 [Cheng Pan] nit
3aef2cef [Cheng Pan] review
500c9cff [Cheng Pan] MVN_OPT
3b4ad142 [Cheng Pan] nit
daa6e0ab [Cheng Pan] review
3383d4c1 [Cheng Pan] [BUILD][TEST] Decouple integration tests from kyuubi-server
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Bump Spark-3.3.0
### _How was this patch tested?_
Pass CI.
Closes#2888 from cfmcgrady/spark33-release.
Closes#2888
88216059 [Fu Chen] bump spark-3.3.0
Authored-by: Fu Chen <cfmcgrady@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
https://www.mail-archive.com/devspark.apache.org/msg29103.html
### _How was this patch tested?_
Pass CI.
Closes#2865 from cfmcgrady/spark33-rc6.
Closes#2865
5dc9579d [Fu Chen] Bump Spark 3.3.0-rc6
Authored-by: Fu Chen <cfmcgrady@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Otherwise `kyuubi-extension-spark-3-3_2.12` does not available in maven repo
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2861 from pan3793/snapshot.
Closes#2861
8f6590b6 [Cheng Pan] [GA] Daily publish snapshot with profile spark-3.3
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
There have been no commits on branch-1.3 for a while, stop daily publish to save resources.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2845 from pan3793/publish.
Closes#2845
681af45b [Cheng Pan] [GA] Stop daily publish on branch-1.3
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Currently, we define a `ExtendedSQLTest` in `kyuubi-common` test sources, and use it to mark some slow test. This PR propose to replace `ExtendedSQLTest` by `Slow` which is provided by scalatest, then we can use the same tag to mark slow tests in some modules which does not depends on `kyuubi-common`, and then seperate them into independent CI job to reduce the CI time
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2824 from pan3793/scalatest-tag.
Closes#2824
2597863c [Cheng Pan] [TEST] Replace test tag ExtendedSQLTest by Slow
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
https://www.mail-archive.com/devspark.apache.org/msg29070.html
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2817 from pan3793/spark-3.3.0-rc5.
Closes#2817
7286ade8 [Cheng Pan] fix
a8079694 [Cheng Pan] update
aba2f427 [Cheng Pan] Build: Bump Spark 3.3.0-rc5
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Provide report flaky test report template to better track problems.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2804 from cxzl25/add_flaky_test_report_template.
Closes#2804
58327662 [sychen] add test label
68d46372 [sychen] add flaky test report template
Authored-by: sychen <sychen@ctrip.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
1. reformat code should include all of the modules in the project.
2. stylecheck should include all of the modules in the project.
3. there is a bug in the stage `Upload style check report`.
Since the `grep` returns code 1 when the pattern is not found, the upload report does not work before this pr.
```
> echo "a" > /tmp/a; grep abc /tmp/a; echo $?
1
```
Before this PR
```
Run for log in `find * -name "scalastyle-output.xml"`; do echo "=========$log========="; grep "error" $log; done
=========dev/kyuubi-tpcds/target/scalastyle-output.xml=========
Error: Process completed with exit code 1.
```
After this PR
```
Run for log in `find * -name "scalastyle-output.xml"`;
<?xml version="1.0" encoding="UTF-8"?>
<checkstyle version="5.0">
<file name="/home/runner/work/incubator-kyuubi/incubator-kyuubi/extensions/spark/kyuubi-extension-spark-3-3/src/main/scala/org/apache/kyuubi/sql/KyuubiSparkSQLExtension.scala">
<error column="0" line="21" source="org.scalastyle.scalariform.ImportOrderChecker" severity="error" message="There should at least one a single empty line separating groups 3rdParty and kyuubi."/>
</file>
</checkstyle>
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2780 from cfmcgrady/refine-stylecheck.
Closes#2780
a9c4a71e [Fu Chen] update reformat
f5d4d4e4 [Fu Chen] address comment
0665e7e5 [Fu Chen] address
91d123ee [Fu Chen] fix
f5d6414f [Fu Chen] style.yml
c56fa081 [Fu Chen] dev/reformat
Authored-by: Fu Chen <cfmcgrady@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
1. Add a static method `builder` in `KyuubiRestClient` to make it easy to use
2. make the default constructor private with `KyuubiRestClient` and `BatchRestApi`
3. enable surefire plugin test for java ut
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2779 from ulysses-you/rest-sdk.
Closes#2643
2b0de7d12 [ulysses-you] label
1d45699b1 [ulysses-you] style
c46c429e0 [ulysses-you] follow up
Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
### _Why are the changes needed?_
Currently the GA jobs share the name log archive file name, it may overwrite each other if multi jobs fail.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2754 from pan3793/ga-log.
Closes#2754
d3b74695 [Cheng Pan] [GA] Seperate log archive name
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
https://www.mail-archive.com/devspark.apache.org/msg29048.html
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2735 from pan3793/spark-33-rc3.
Closes#2735
99ea2b56 [Cheng Pan] simplify
66d0847b [Cheng Pan] Fix ut
b1bad7e6 [Cheng Pan] Test Spark 3.3.0-rc3
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Cross version verification for spark-3.3
### _How was this patch tested?_
Pass CI.
Closes#2733 from cfmcgrady/spark33-ci.
Closes#2733
2968f80e [Fu Chen] cross version verification for spark-3.3
Authored-by: Fu Chen <cfmcgrady@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
Separate events to a submodule - kyuubi-event
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2392 from zhaomin1423/event-module.
Closes#2323
7efa923f [Min Zhao] fix unit test
07126f43 [Min Zhao] [KYUUBI #2323] Separate events to a submodule - kyuubi-event
cccbdf11 [Min Zhao] [KYUUBI #2323] Separate events to a submodule - kyuubi-event
194aca82 [Min Zhao] add shade and log4j to test/resources
f75891a1 [Min Zhao] [KYUUBI #2323] Separate events to a submodule - kyuubi-event
357395fa [Min Zhao] [KYUUBI #2323] Separate events to a submodule - kyuubi-event
b6dfd186 [Min Zhao] [KYUUBI #2323] Separate events to a submodule - kyuubi-event
fc876688 [Min Zhao] [KYUUBI #2323] Separate events to a submodule - kyuubi-event
1b86d41b [Min Zhao] seperate event
Authored-by: Min Zhao <zhaomin1423@163.com>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
Change `kubernetes-deployment-it` to `kubernetes-it`
### _How was this patch tested?_
Pass CI
Closes#2329 from ulysses-you/KYUUBI-2214-FOLLOWUP.
Closes#2329
eae09ddc [ulysses-you] cleanup
Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
Close#2216
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2258 from zwangsheng/feature/add_cluster_k8s_test.
Closes#2216
7eea16b0 [zwangsheng] add linces
8effcf07 [Binjie Yang] Spark On K8s With Cluster
9cd03ad9 [ulysses-you] config
ce7143ee [zwangsheng] fix unused import
0a06521f [zwangsheng] fix
f9293925 [zwangsheng] test
88ec89f9 [zwangsheng] test
0f780f37 [zwangsheng] test
9b27bfa6 [zwangsheng] Test
df43cd76 [zwangsheng] Using Kerberos Submit Spark
99893a64 [zwangsheng] call beforall help set kerbores
0a732d94 [zwangsheng] Spark On K
Lead-authored-by: zwangsheng <2213335496@qq.com>
Co-authored-by: ulysses-you <ulyssesyou18@gmail.com>
Co-authored-by: Binjie Yang <52876270+zwangsheng@users.noreply.github.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
resolved https://github.com/apache/incubator-kyuubi/pull/2293#discussion_r846934352
Move `kyuubi-server-plugin` to `extension/server/`
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2319 from cfmcgrady/kyuubi-2293-followup.
Closes#2292
949fc3d6 [Fu Chen] kyuubi-server -> server
Authored-by: Fu Chen <cfmcgrady@gmail.com>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
Partially fix#2315 problem 1
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2316 from yangrong688/patch-1.
Closes#2316
e76fe65d [yangrong688] [DOCS] Change "Imporving the documentation" to "Improving the documentation"
Authored-by: yangrong688 <70557521+yangrong688@users.noreply.github.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Make the mapping correct
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2311 from ulysses-you/sync-label.
Closes#2311
8c7176e0 [ulysses-you] ordering
3a0556a4 [ulysses-you] refine github label
Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
Move `kyuubi-server-plugin` to `extension/kyuubi-server/`
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2293 from ulysses-you/server-plugin.
Closes#2292
54b1c1e1 [ulysses-you] ordering
55cd92fe [ulysses-you] fix
9e7d4e8c [ulysses-you] unify
Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: Fu Chen <cfmcgrady@gmail.com>
### _Why are the changes needed?_
Move `dev/kyuubi-spark-extension*` to `extension/spark/`
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2291 from ulysses-you/unify-spark-extension.
Closes#2292
4e6f4ca6 [ulysses-you] fix
c2cc018d [ulysses-you] unify
2375af0d [ulysses-you] unify
Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
close https://github.com/apache/incubator-kyuubi/issues/2087
Add an issue template for documentation improvement.
I have self-tested this doc issue template and shown as below.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2235 from yangrong688/KYUUBI-2087.
Closes#2087
176fab4e [yangrong688] Merge branch 'apache:master' into KYUUBI-2087
8c5e9acc [yangrong@asinking.com] [fix] add affected version
e6c79989 [yangrong@asinking.com] [fix] refined descriptions and placeholders based on kent's suggestion
4d546228 [yangrong@asinking.com] [fix] refactor issue template description and remove label kind:bug
f181ba8a [yangrong688] [DOCS] add doc-improvement-report.yml
Lead-authored-by: yangrong@asinking.com <Yr1995!@#>
Co-authored-by: yangrong688 <yangrong.jxufe@gmail.com>
Co-authored-by: yangrong688 <70557521+yangrong688@users.noreply.github.com>
Signed-off-by: KenjiFujima <thanosxnicholas@gmail.com>
### _Why are the changes needed?_
much code about maven options only has `-Pflink-provided,spark-provided` no `hive-provided`.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2219 from deadwind4/KYUUBI-2218.
Closes#2218
5e369978 [Ada Wang] fix build dist help
a5de3ba8 [Ada Wang] [KYUUBI #2218] Fix maven options about hive-provided
Authored-by: Ada Wang <wang4luning@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
Change `kyuubi-common` to `kyuubi-ctl` in ctl label.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2217 from ulysses-you/label.
Closes#2085
69b36156 [ulysses-you] fix
Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
Rename `kyuubi-deployment-kubernetes-it` to `kyuubi-kubernetes-it` which include:
- Kyuubi Server on Kubernetes it
- Kyuubi Spark Engine on Kubernetes it
Now, we use the package name to distingush the scope of it:
- deployment package is for Kyuubi Server
- spark package is for Spark Engine
So it's easy to add Flink Engine on kubernetes it in future.
Note that, this PR only supports Spark client mode on Kubernetes it.
Since Spark only have two tags of docker image https://hub.docker.com/r/apache/spark/tags, this PR only test 3.2.1 version
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2214 from ulysses-you/spark-on-k8s-client.
Closes#2214
f6dda432 [ulysses-you] Add Spark Engine on Kubernetes integration test
Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2183 from Nick-0723/fix_duplicate_labels.
Closes#2086
fee74142 [Nick Song] remove duplicate labels
Authored-by: Nick Song <chun2184@163.com>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
Add issue template for subtasks
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2182 from Nick-0723/subtask.
Closes#2086
545c0c7d [Nick Song] add description textarea
7aec23d9 [Nick Song] subtask issue template
Authored-by: Nick Song <chun2184@163.com>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
Better ASF GHA resource usage, see more https://cwiki.apache.org/confluence/display/BUILDS/GitHub+Actions+status
- cancel-in-progress enabled. w/ this, a previous build can be canceled if new commit(s) have arrived
- style check w/o install. currently, the style check takes 8~10min to finish because an install step before it. now, it shall return within a minute. note that, this might 'fail' itself when we introduce a new module and others depend on it, but it's ok to verify by the log whether a PR ok to be merged.
- dependency check runs only when pom.xml updates. this flow now runs only pom changes as it is the only way that may cause dependency changes.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
**Canceled**: https://github.com/apache/incubator-kyuubi/runs/5604809742?check_suite_focus=true
<img width="548" alt="image" src="https://user-images.githubusercontent.com/8326978/159059722-480f4fd6-d87e-4b07-984e-eb5b08a84beb.png">
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2175 from yaooqinn/gha.
Closes#2175
33777bc5 [Kent Yao] Update .github/workflows/style.yml
ec41ddd0 [Kent Yao] Update .github/workflows/style.yml
79db981f [Kent Yao] fix
4e1fb387 [Kent Yao] fix
733df9cb [Kent Yao] Improve CI with cancel & concurrency
494c3a84 [Kent Yao] Revert "Improve CI with cancel & concurrency"
57641ac4 [Kent Yao] Improve CI with cancel & concurrency
6dafe970 [Kent Yao] Improve CI with cancel & concurrency
40160ab9 [Kent Yao] Improve CI with cancel & concurrency
377285c2 [Kent Yao] Improve CI with cancel & concurrency
dbe8999e [Kent Yao] Improve CI with cancel & concurrency
78a97260 [Kent Yao] Improve CI with cancel & concurrency
7e984ebd [Kent Yao] Improve CI with cancel & concurrency
d2538210 [Kent Yao] Improve CI with cancel & concurrency
51a7c48c [Kent Yao] Improve CI with cancel & concurrency
c5e3177a [Kent Yao] Improve CI with cancel & concurrency
Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
Fix the wrong path of `module:hive`
### _How was this patch tested?_
- https://github.com/cfmcgrady/incubator-kyuubi/pull/12Closes#2139 from cfmcgrady/kyuubi-2085-followup.
Closes#2085
ca93467b [Fu Chen] addressed comment
2dc6253a [Fu Chen] fix the wrong path of module:hive
Authored-by: Fu Chen <cfmcgrady@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
Upload Test Log for CI failure shall contain trino engine log.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2097 from zhaomin1423/update_ci.
Closes#2097
5796174a [Min Zhao] [CI] Upload Test Log for CI failure shall contain trino engine log #2094
Authored-by: Min Zhao <zhaomin1423@163.com>
Signed-off-by: Kent Yao <yao@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
branch-1.5 & 1.4 snapshots are missing
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2051 from yaooqinn/1955.
Closes#1955
693c79b1 [Kent Yao] Add CI for branch-1.5 & 1.4 SNAPSHOTS
Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
Use the default Spark version to run TPC-DS test.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1975 from pan3793/tpcds.
Closes#1975
ecdda1d9 [Cheng Pan] style
1e8468f4 [Cheng Pan] nit
9666cd8b [Cheng Pan] Update golden file
10b38939 [Cheng Pan] default spark version
c1c9b037 [Cheng Pan] [BUILD] Run TPC-DS test using Spark 3.2
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
close#1944
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1954 from cfmcgrady/kyuubi-1944.
Closes#1944
2ce2f069 [Fu Chen] Upgrade Spark 3.1.3
Authored-by: Fu Chen <cfmcgrady@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
1. Add missing asf header
2. Disable fail-fast to let jobs run individually
3. Group defaults maven builds with java & spark version matrix
- so this PR brings fully tests with JDK 11 back
4. Group others(now cross-version check only) in `include`
5. replace `actions/cachev2` with `actions/setup-javav2` - `cache: 'maven'`
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Need to pass all CI jobs
Closes#1947 from yaooqinn/ga.
Closes#1947
7d5d2472 [Kent Yao] address comment
d8d347b0 [Kent Yao] address comment
8c3cba99 [Kent Yao] address comment
7400cdc6 [Kent Yao] checkout
20eeb7bb [Kent Yao] test
b1c70e42 [Kent Yao] test
4ca4f233 [Kent Yao] test
b588cd32 [Kent Yao] test
da4f6aea [Kent Yao] test
057790d5 [Kent Yao] test
01b07b25 [Kent Yao] test
2906ec66 [Kent Yao] test
4d97bf19 [Kent Yao] test
cf16f169 [Kent Yao] test
8c12b8ef [Kent Yao] test
3b84e450 [Kent Yao] test
6e21d97a [Kent Yao] test
dc2d9004 [Kent Yao] test
1cff392d [Kent Yao] test
a4b47476 [Kent Yao] test
c4718022 [Kent Yao] test
d264d33d [Kent Yao] test
ee293c13 [Kent Yao] test
903cb0bb [Kent Yao] test
f74fa1a5 [Kent Yao] test
Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
Remove `spark-hadoop` from matrix to simplify workflow
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1945 from pan3793/ga.
Closes#1945
3bd511eb [Cheng Pan] nit
442f61b9 [Cheng Pan] fix
af26d6f6 [Cheng Pan] list
b1e2fc78 [Cheng Pan] fix
9f372d13 [Cheng Pan] GA: Remove spark-hadoop from matrix to simplify workflow
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
Follow up #1876
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1937 from pan3793/ga.
Closes#1937
e6794cbf [Cheng Pan] Build: Use Spark 3.2 for cross version verfication
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
closes https://github.com/apache/incubator-kyuubi/issues/1536
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1892 from ulysses-you/KYUUBI-1536.
Closes#1536
f0c450a9 [ulysses-you] codecov
35fac4f6 [ulysses-you] fix
f34cfbf4 [ulysses-you] comment
78a54bc1 [ulysses-you] address comment
5cb574ba [ulysses-you] fix test
1033909b [ulysses-you] fix
3c925e56 [ulysses-you] address comment
a74988cd [ulysses-you] style
649f02d0 [ulysses-you] Add session conf advisor
debd3adb [ulysses-you] Add session conf advisor
Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
Fix the version of `actions/checkout` in file .github/workflows/publish-snapshot.yml.
```
Error: Unable to resolve action `actions/checkout2.3.5`, unable to find version `2.3.5`
```
see https://github.com/apache/incubator-kyuubi/runs/5177066290?check_suite_focus=true for details.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1901 from cfmcgrady/ga.
Closes#1901
276380cf [Fu Chen] fix checkout version
Authored-by: Fu Chen <cfmcgrady@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
support test k8s 1.23.
more details see [change list](https://github.com/manusa/actions-setup-minikube/compare/v2.4.2...v2.4.3).
note that, we should upgrade k8s-client first, since k8s-client 5.5.0 does not support k8s 1.22.1 and later. see [Compatibility Matrix](https://github.com/fabric8io/kubernetes-client#kubernetes-compatibility-matrix)
### _How was this patch tested?_
Pass CI
Closes#1871 from ulysses-you/minikube.
Closes#1871
772e358c [ulysses-you] upgrade minikube action from 2.4.2 to 2.4.3
Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: Kent Yao <yao@apache.org>
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
```
Run docker/login-actionv[1](https://github.com/cfmcgrady/incubator-kyuubi/runs/5088959102?check_suite_focus=true#step:3:1)
with:
ecr: auto
logout: true
Error: Username and password required
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1863 from cfmcgrady/avoid-publish-image.
Closes#1804
5d7711bc [Fu Chen] [GA] avoid push docker image for forked repos
Authored-by: Fu Chen <cfmcgrady@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
Close#1848
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1862 from pan3793/spark-3.2.1.
Closes#1848
306da100 [Cheng Pan] [KYUUBI #1848] Bump Spark 3.2.1
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
The previous PR did not handle log4j 1.2 deps of `spark-hive`, and the current detect deps change workflow has the dirty cache issue, this PR also fix it.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#1856 from pan3793/log4j2.
Closes#1769
2785acf4 [Cheng Pan] nit
58affcb5 [Cheng Pan] update workflow
301acb6f [Cheng Pan] update check deps script
7611c7e0 [Cheng Pan] revert exclusion
239e7cc4 [Cheng Pan] Remove duplicated scalatest
e143fecc [Cheng Pan] dependencyList
a7254771 [Cheng Pan] [KYUUBI #1769][FOLLOWUP] Exclude log4j 12 deps from test deps
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Fei Wang <fwang12@ebay.com>
