### _Why are the changes needed?_
Add args `--no-mac-metadata --no-xattrs --no-fflags` to `tar` on macOS in `build/dist` to exclude macOS-specific extended metadata.
The binary tarball created on macOS includes extended macOS-specific metadata and xattrs, which causes warnings when unarchiving it on Linux.
Step to reproduce
1. create tarball on macOS (13.3.1)
```
➜ apache-kyuubi git:(master) tar --version
bsdtar 3.5.3 - libarchive 3.5.3 zlib/1.2.11 liblzma/5.0.5 bz2lib/1.0.8
```
```
➜ apache-kyuubi git:(master) build/dist --tgz
```
2. unarchive the binary tarball on Linux (CentOS-7)
```
➜ ~ tar --version
tar (GNU tar) 1.26
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Written by John Gilmore and Jay Fenlason.
```
```
➜ ~ tar -xzf apache-kyuubi-1.8.0-SNAPSHOT-bin.tgz
tar: Ignoring unknown extended header keyword `SCHILY.fflags'
tar: Ignoring unknown extended header keyword `LIBARCHIVE.xattr.com.apple.FinderInfo'
```
### _How was this patch tested?_
- [x] Manual tests
Create binary tarball on macOS then unarchive on Linux, warnings disappear after this change.
Closes#4828 from pan3793/dist.
Closes#4828
7bc49d847 [Cheng Pan] [BUILD] Exclude macOS tar extended metadata in build/dist
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
- bump Maven from 3.8.7 to 3.9.1, 3.9.1 fixed the performance issue [MNG-7677](https://issues.apache.org/jira/browse/MNG-7677) in 3.9.0, release notes: https://maven.apache.org/docs/3.9.1/release-notes.html
- Mvnd from 0.9.0 to 1.0-m6 (with embedded maven 3.9.1), release notes: https://github.com/apache/maven-mvnd/releases/tag/1.0-m6
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4647 from bowenliang123/maven-3.9.1.
Closes#4647
f803394df [liangbowen] remove property
efd199f7a [liangbowen] fix
87e18d70a [Bowen Liang] Update build/mvnd
10f4a25ff [liangbowen] bump Maven from 3.8.7 to 3.9.1, and Mvnd from 0.9.0 to 1.0-m6 (with embedded maven 3.9.1)
Lead-authored-by: liangbowen <liangbowen@gf.com.cn>
Co-authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
Introduce a brand new CHAT engine, it's supposed to support different backends, e.g. ChatGPT, 文心一言, etc.
This PR implements the following providers:
- ECHO, simply replies a welcome message.
- GPT: a.k.a ChatGPT, powered by OpenAI, which requires a API key for authentication. https://platform.openai.com/account/api-keys
Add the following configurations in `kyuubi-defaults.conf`
```
kyuubi.engine.chat.provider=[ECHO|GPT]
kyuubi.engine.chat.gpt.apiKey=<chat-gpt-api-key>
```
Open an ECHO beeline chat engine.
```
beeline -u 'jdbc:hive2://localhost:10009/?kyuubi.engine.type=CHAT;kyuubi.engine.chat.provider=ECHO'
```
```
Connecting to jdbc:hive2://localhost:10009/
Connected to: Kyuubi Chat Engine (version 1.8.0-SNAPSHOT)
Driver: Kyuubi Project Hive JDBC Client (version 1.7.0)
Beeline version 1.7.0 by Apache Kyuubi
0: jdbc:hive2://localhost:10009/> Hello, Kyuubi!;
+----------------------------------------+
| reply |
+----------------------------------------+
| This is ChatKyuubi, nice to meet you! |
+----------------------------------------+
1 row selected (0.397 seconds)
```
Open a ChatGPT beeline chat engine. (make sure your network can connect the open API and configure the API key)
```
beeline -u 'jdbc:hive2://localhost:10009/?kyuubi.engine.type=CHAT;kyuubi.engine.chat.provider=GPT'
```
<img width="1109" alt="image" src="https://user-images.githubusercontent.com/26535726/225813625-a002e6e2-3b0d-4194-b061-2e215d58ba94.png">
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4544 from pan3793/chatgpt.
Closes#4544
87bdebb6d [Cheng Pan] nit
f7dee18f3 [Cheng Pan] Update docs
9beb55162 [cxzl25] chat api (#1)
af38bdc7c [Cheng Pan] update docs
9aa6d83a6 [Cheng Pan] Initial implement Kyuubi Chat Engine
Lead-authored-by: Cheng Pan <chengpan@apache.org>
Co-authored-by: cxzl25 <cxzl25@users.noreply.github.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- to fix mvnd failure in `Install` step of `Sytle Check` job in (https://github.com/apache/kyuubi/actions/runs/4403008223/jobs/7711886893#step:7:1012)
- by lowering `mvnd.minThreads` to reduce memory pressure in concurrent builds
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4502 from bowenliang123/sytle-mvn.
Closes#4502
a3ddb6270 [liangbowen] lower `mvnd.minThreads`
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
- concurrent execution and smart builder is shipped with `mvnd`, but the CI log shows they are not activated
- increase maximum degree of concurrency and utilize `SmartBuilder` feature in `mvnd`. `-Dmvnd.minThreads` is set as fallback and it will be ignored if `--threads` or `-Dmvnd.threads` is used.
#### Before:
(https://github.com/apache/kyuubi/actions/runs/4276652363/jobs/7444896450#step:6:64)
```
[INFO] Build maximum degree of concurrency is 1
```
#### After:
(https://github.com/apache/kyuubi/actions/runs/4279322563/jobs/7449912132#step:8:64)
```
[INFO] Using the SmartBuilder implementation with a thread count of 4
[INFO] Build maximum degree of concurrency is 4
```
(https://github.com/apache/kyuubi/actions/runs/4279322563/jobs/7449912132#step:8:553)
```
[INFO] Segment walltime 13 s, segment projects service time 36 s, effective/maximum degree of concurrency 2.63/4
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4420 from bowenliang123/mvnd-smart.
Closes#4274
64f0cc53e [liangbowen] update
391da26d3 [liangbowen] quite in dep and style jobs
c776356a5 [liangbowen] use -Dmvnd.minThreads
d10bfb70c [liangbowen] use -Dmvnd.minThreads
f793d2b6b [liangbowen] warn
bbb929fe3 [liangbowen] quiet
a5e6d4914 [liangbowen] logging warn level
3cfbaad45 [liangbowen] enable quite option and cancel redirect stderr
09442ff69 [liangbowen] update
ca5f855fa [liangbowen] try to redirect stderr
0ebdd18d4 [liangbowen] set completion and print MAVEN_CLI_OPTS
d3b2c9656 [liangbowen] revert explicitly setting `mvnd.noBuffering`
e2eca0140 [liangbowen] revert explicitly setting `mvnd.noBuffering`
dba25e327 [liangbowen] explicitly set `mvnd.noBuffering` to false prevent displaying events continuously
b7583f936 [liangbowen] shell
d60ebd047 [liangbowen] change mvnd cache key
2cc576ef8 [liangbowen] dep
bbd6414c9 [liangbowen] move multithread setting to mvnd's MAVEN_CLI_OPTS
347b58c8c [liangbowen] increase concurrency for spotless check
15c5519e4 [liangbowen] increase concurrency
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
`bin/docker-image-tool.sh` should include `web-ui` when build image
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
```
build/dist --web-ui
cd dist
bin/docker-image-tool.sh -t test build
cat > conf/kyuubi-defaults.conf << EOF
kyuubi.frontend.protocols REST
EOF
docker run --rm -v $PWD/conf:/opt/kyuubi/conf -p10099:10099 kyuubi:test
```
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4413 from pan3793/web-ui-package.
Closes#4413
787f89515 [Cheng Pan] [BUILD] Build docker image should include web UI
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
```
Usage:
+----------------------------------------------------------------------------------------------+
| ./build/dist [--name <custom_name>] [--tgz] [--web-ui] [--flink-provided] [--hive-provided] |
| [--spark-provided] [--mvn <maven_executable>] <maven build options> |
+----------------------------------------------------------------------------------------------+
name: - custom binary name, using project version if undefined
tgz: - whether to make a whole bundled package
web-ui: - whether to include web ui
flink-provided: - whether to make a package without Flink binary
hive-provided: - whether to make a package without Hive binary
spark-provided: - whether to make a package without Spark binary
mvn: - external maven executable location
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
Create binary artifacts using `build/dist --tgz --web-ui` and run, then open `http://0.0.0.0:10099/ui`
<img width="1401" alt="image" src="https://user-images.githubusercontent.com/26535726/220753103-ce801f12-f394-4ece-92a4-1902e93c62c7.png">
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4397 from pan3793/webui-build.
Closes#4397
97901d63e [Cheng Pan] doc
37d5e2ad3 [Cheng Pan] mirror-cdn
c5751dd5b [Cheng Pan] remove unused dep
d308defb7 [Cheng Pan] nit
9abca4705 [Cheng Pan] nit
c1d184afd [Cheng Pan] nit
7091d5bf5 [Cheng Pan] regex
f0ac16b3c [Cheng Pan] [BUILD] `build/dist` support --web-ui
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
The release-related shell scripts should have execution permission.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4377 from pan3793/x.
Closes#4377
fb727adc [Cheng Pan] Grant execute permission to release scripts
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
The module is missed because it's only included when `spark-3.3` is activated.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4374 from pan3793/release.
Closes#4374
ffc0dc22 [Cheng Pan] fix
d21738f1 [Cheng Pan] Release uploading should include kyuubi-spark-connector-hive
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
The uploading destination is the svn repo location, not the local temp folder.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4373 from pan3793/minor.
Closes#4373
96aa1d4c9 [Cheng Pan] Fix echo messeage in release script
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
eclipse-temurin is the successor for openjdk, see https://github.com/apache/spark/pull/37705
"The core change is: the OS of base image changes debian-bullseye to ubuntu-focal (based on debian bullseye)."
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4288 from pan3793/image.
Closes#4288
b7b619018 [Cheng Pan] Use eclipse-temurin:8-jdk-focal as default base image
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
`build/mvn` should download the maven when the local `mvn` version does not match, this corrects the change in #4261
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4291 from pan3793/mvn-version.
Closes#4247
42717a73c [Cheng Pan] [KYUUBI #4247][FOLLOWUP] Fix build/mvn version comparison
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
As mentioned in https://github.com/apache/kyuubi/issues/4139, we are not able to use Travis for AMR testing in the future.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4287 from pan3793/rm-travis.
Closes#4287
efe2a4854 [Cheng Pan] nit
c72a8b00c [Cheng Pan] Remove travis
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
As mentioned in #4247, it seems to be better that using the same maven version with master. Thus, modify the version check logic in `build/mvn` from equal or greater to equal.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4261 from edddddy/mvn_ver_cp.
Closes#4247
11f19eca1 [1456173400] change the if condition to equal
Lead-authored-by: edddddy <rmrfall@qq.com>
Co-authored-by: 1456173400 <1456173400@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- Skip Maven dependency snapshot update fetching to building `dependencyList` faster without waiting for fetching daily snapshots checking (only snapshots of submodules used).
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4191 from bowenliang123/dependency-resolve.
Closes#4191
0bf36815 [liangbowen] apply `--no-snapshot-updates` option in build_classpath
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
- commit date is more readable and helpful to determine earlier/later version, especially for snapshot branch
- adding revision commit date from git to `kyuubi-version-info.properties`
- Print revision date on server startup
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4187 from bowenliang123/print-revision-date.
Closes#4187
b0b0a6c8 [liangbowen] print revision date on server startup
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
After changes in #4034, we should modify `./build/dist` to build kyuubi binary with charts
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4143 from zwangsheng/helm/build_dist_with_charts.
Closes#4034
78edac4e [zwangsheng] [BUILD] Should build binary with charts dir
Authored-by: zwangsheng <2213335496@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Fix#2062
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4078 from xiaoyuandajian/fix-#2062.
Closes#4078
a5a01945 [Cheng Pan] Update .github/workflows/style.yml
3c08e692 [袁福元] fix the shellcheck found error file
a33cbb22 [袁福元] fix the shellcheck found error file
da195782 [袁福元] fix the shellcheck found error file
b24e6adf [袁福元] fix the shellcheck found error file
90ef0712 [袁福元] fix the shellcheck found error file
42acb31f [袁福元] fixup! test1
31c1783d [袁福元] fixup! test
c40b5f38 [袁福元] Revert "test"
d97801fa [袁福元] Merge remote-tracking branch 'origin/fix-#2062' into fix-#2062
39a9ff31 [袁福元] test
fc7198da [xiaoyuandajian] Update style.yml
62e80726 [袁福元] test
0abdef36 [xiaoyuandajian] Delete main.yml
1e252e2a [xiaoyuandajian] Create main.yml
c19728ec [xiaoyuandajian] Delete main.yml
5db61e5a [xiaoyuandajian] Delete main1.yml
9e27a363 [xiaoyuandajian] Create main1.yml
69540652 [xiaoyuandajian] Create main.yml
Lead-authored-by: 袁福元 <yuanfuyuan@mafengwo.com>
Co-authored-by: xiaoyuandajian <51512358+xiaoyuandajian@users.noreply.github.com>
Co-authored-by: Cheng Pan <pan3793@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
The changelogs are exported from the git command, which is quite duplicated and does not help much, after removing, users still can get them from GitHub or git history easily.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4094 from pan3793/changelog.
Closes#4094
2d0bfe07 [Cheng Pan] [INFRA] Remove changelogs and generate_changelog.sh
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
This is part of https://github.com/apache/kyuubi/issues/4020, and should finalize the update for repo https://github.com/apache/kyuubi
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4089 from pan3793/graduate.
Closes#4089
60ece0c1 [Cheng Pan] [INFRA] Update release scripts and templates because of graduation
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- DISCLAIMER file on top-level folder of project was already removed after graduation in (https://github.com/apache/kyuubi/issues/4020)
- fix build/dist for `cp` non-existed DISCLAIMER file
- remove DISCLAIMER file reference in dev/dist script, doc
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4086 from bowenliang123/desclaimer-ref.
Closes#4086
18be7489 [liangbowen] remove DISCLAIMER file reference in dev/dist script, doc, docker , labeler.yml
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4069 from lightning-L/kyuubi-4020.
Closes#4020
97406ca0 [Tianlin Liao] [KYUUBI #4020] remove incubating from kyuubi source code
Authored-by: Tianlin Liao <tiliao@ebay.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
#3847
Windows users need to generate `kyuubi-version-info.properties` to debug.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3903 from cxzl25/win_gen_version_info.
Closes#3903
089cf9eb [sychen] support windows generate kyuubi-version-info.properties
Authored-by: sychen <sychen@ctrip.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3816 from cxzl25/release_typo.
Closes#3816
614d60fe [sychen] typo
Authored-by: sychen <sychen@ctrip.com>
Signed-off-by: Shaoyun Chen <csy@apache.org>
### _Why are the changes needed?_
Fix https://github.com/apache/incubator-kyuubi/issues/3441;
This pr aims to two points as follow:
- Bump Spark 3.3.0 to Spark 3.3.1 in spark-3.3 profile
- Change default Spark version to 3.3.1
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3507 from Yikf/pr/3471.
Closes#3441
7449280b [yikf] Change spark 3.3.0 as default version
Authored-by: yikf <yikaifei1@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
By default, on CentOS 7 w/ openJDK 8 installed via YUM/RPM, the `/usr/bin/java` links to `/usr/lib/jvm/java-1.8.0-openjdk/jre/bin/java`, the current detection logic will think the `JAVA_HOME` is `/usr/lib/jvm/java-1.8.0-openjdk/jre`, and then cause the issue.
```
[ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:4.6.1:compile (scala-compile-first) on project kyuubi-common_2.12: wrap: java.io.IOException: Cannot run program "/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.322.b06-1.el7_9.x86_64/jre/bin/javac" (in directory "/home/pancheng/apache-kyuubi"): error=2, No such file or directory -> [Help 1]
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
w/ this patch, the `build/dist` works expected on CentOS w/ openJDK 8 installed via `yum`, the debug info shows it detected the right `JAVA_HOME`
```
+ [[ -z '' ]]
++ command -v rpm
+ '[' /usr/bin/rpm ']'
++ rpm -E %java_home
+ RPM_JAVA_HOME=/usr/lib/jvm/java
+ '[' /usr/lib/jvm/java '!=' %java_home ']'
+ JAVA_HOME=/usr/lib/jvm/java
+ echo 'No JAVA_HOME set, proceeding with '\''/usr/lib/jvm/java'\'' learned from rpm'
No JAVA_HOME set, proceeding with '/usr/lib/jvm/java' learned from rpm
+ '[' -z /usr/lib/jvm/java ']'
+ [[ -z /usr/lib/jvm/java ]]
+ echo 'JAVA_HOME is set to /usr/lib/jvm/java'
JAVA_HOME is set to /usr/lib/jvm/java
```
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3619 from pan3793/java-home.
Closes#3619
b6e10a78 [Cheng Pan] nit
2a59d299 [Cheng Pan] [BUILD] Detect JAVA_HOME from rpm
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
The 1.6.0 binary tarball is nearly 300MB, we can reduce the size by sharing jars between engine and server, as beeline does.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
```
-rw-r--r-- 1 chengpan staff 155M Sep 9 14:21 apache-kyuubi-1.7.0-SNAPSHOT-bin.tgz
```
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3459 from pan3793/dist.
Closes#3459
3a995e77 [Cheng Pan] fix
b86d1a0b [Cheng Pan] [BUILD] Share jars to reduce binary release tarball size
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Some users encounter errors on building, enable -x to help user identify the failure reason
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3372 from pan3793/dist.
Closes#3372
7bdfb39f [Cheng Pan] Enable -x on build/dist
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
close https://github.com/apache/incubator-kyuubi/issues/3138#issue-1316764571
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#3146 from zhaomin1423/dist_doris.
Closes#3138
8edfb174 [Min] [KYUUBI #3138] [Subtask] DorisSQLEngine - Add jdbc engine to dist
Authored-by: Min <zhaomin1423@163.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2862 from pan3793/release-3.3.
Closes#2862
8c03a59f [Cheng Pan] [BUILD] Release script support Spark 3.3
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
refactor the changelog script to fix:
- the nit of Apache Kyuubi(Incubating)
- the format of the output changelog
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2839 from ulysses-you/changelog.
Closes#2839
d9e2b6d0 [ulysses-you] comments
b966dc30 [ulysses-you] comments
5f94c52d [ulysses-you] refactor
18cdde22 [ulysses-you] refactor
009699c3 [ulysses-you] Merge branch 'master' of https://github.com/apache/incubator-kyuubi into changelog
76419a92 [ulysses-you] fix
1a4090fc [ulysses-you] nit
b6154b58 [ulysses-you] changelog
Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
Maintain notice-binary manually.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2828 from ulysses-you/notice.
Closes#2746
52a0652f [ulysses-you] netty
c92695ec [ulysses-you] provided
318a7f5c [ulysses-you] maintain notice-binary
04cc02da [ulysses-you] init
Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
Add generate_changelog.sh script to simple generate changelog for Apache Incubating Kyuubi.
I've searched there are generaly three ways to generate changelog
1. add change log manually in ervery pr ([Apache Skywalking](https://github.com/apache/skywalking/pull/8881/files#diff-e4d10d406c447edb82b7e7958bebc5ff8ca9123076cf138b4e0ba037fa873055))
2. use github action to generate ([sqlfluff](https://github.com/sqlfluff/sqlfluff/blob/main/.github/release-drafter.yml))
3. use script to generate changelog
I choose to write a script to generate changelog, it does not rely on github and we can use it more conviniently as there maybe mutiple release candidates in the release process.
I add a script to generate changelog once we prepare a new release version, it allows release manager to easy generate changelog between last release tag and current release tag.
And the changelog doc structure is:
$KYUUBI_DIR
    /docs
        /changelog
            /v1.5.1-incubating.md
            ...
### _How was this patch tested?_
use `./generate_changelog.sh v1.5.0-incubating v1.5.1-incubating-rc0` to generate v1.5.1-incubating changelog
Closes#2501 from yangrong688/KYUUBI-2211.
Closes#2211
b3174b4d [yangrong688] Add generate_changelog.sh script to simple generate changelog for Apache Incubating Kyuubi.
Authored-by: yangrong688 <yangrong.jxufe@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
…xecutable
### _Why are the changes needed?_
fix#2346
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2418 from jiaoqingbo/kyuubi2346.
Closes#2346
eda5d383 [jiaoqingbo] after code review
720fede3 [jiaoqingbo] execute source config.sh and fix ut failed and delete debug log
1b53a9ba [jiaoqingbo] Merge branch 'master' into kyuubi2346
0c7eb5e7 [jiaoqingbo] add hadoop classpath to UT
ff2bb14c [jiaoqingbo] Merge branch 'master' into kyuubi2346
c9e8019d [jiaoqingbo] change ut
308ae5fd [jiaoqingbo] code review
f3eb068b [jiaoqingbo] fix ui failed
4e6d168d [jiaoqingbo] spoltless apply
9eae5576 [jiaoqingbo] fix ut failed
bc00e690 [jiaoqingbo] delete flink-sql-engine.sh
d6b87b9c [jiaoqingbo] delete childProcEnv
3aa8738a [jiaoqingbo] [KYUUBI #2346] [Improvement] Simplify FlinkProcessBuilder with java executable
Authored-by: jiaoqingbo <1178404354@qq.com>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
fix#2424
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2425 from jiaoqingbo/2424.
Closes#2424
9949d9a5 [jiaoqingbo] Update UtilsSuite.scala
ac9afb46 [jiaoqingbo] Update package.scala
e415345d [jiaoqingbo] Update kyuubi-common/src/main/scala/org/apache/kyuubi/package.scala
081b061f [jiaoqingbo] Update kyuubi-server/src/main/scala/org/apache/kyuubi/server/KyuubiServer.scala
c1ade020 [jiaoqingbo] Update build/kyuubi-build-info
6af3dd08 [jiaoqingbo] [KYUUBI #2424] [Improvement] add Flink compile version and Trino client compile version
Authored-by: jiaoqingbo <1178404354@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Build the java command programlly instead of a static shell script
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2336 from yaooqinn/trino.
Closes#2336
c906fcc4 [Kent Yao] add comments
7048f971 [Kent Yao] fix tests
49e582fc [Kent Yao] fix tests
444de58e [Kent Yao] Simplify TrinoProcessBuilder with java executable
9f2d1467 [Kent Yao] Simplify TrinoProcessBuilder with java executable
d4fdd467 [Kent Yao] Simplify TrinoProcessBuilder with java executable
48da4e0a [Kent Yao] Simplify TrinoProcessBuilder with java executable
de4884c0 [Kent Yao] Simplify TrinoProcessBuilder with java executable
6ac315a2 [Kent Yao] Simplify TrinoProcessBuilder with java executable
Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
Make the HiveProcBuilder actually work.
- hive 2.3.9 has scala 2.11 deps, so not compatible with kyuubi common, thus, upgrade 3.1.x
- hive 3.1 does not work with java 11 as https://issues.apache.org/jira/browse/HIVE-21237 bug
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2290 from yaooqinn/2024.
Closes#2024
83e07fa5 [Kent Yao] fix
1e7167f8 [Kent Yao] import
ac7853ac [Kent Yao] jdk11
ba9f6c33 [Kent Yao] temp
44049736 [Kent Yao] temp
fd624307 [Kent Yao] temp
a4a5e42f [Kent Yao] temp
b01bb226 [Kent Yao] temp
298fc478 [Kent Yao] temp
5630857a [Kent Yao] temp
98457b6e [Kent Yao] temp
18d8f5cb [Kent Yao] [KYUUBI #2024][FOLLOWUP] Hive Backend Engine - ProcBuilder for HiveEngine
246681dd [Kent Yao] [KYUUBI #2024][FOLLOWUP] Hive Backend Engine - ProcBuilder for HiveEngine
890579b9 [Kent Yao] [KYUUBI #2084][FOLLOWUP] Support arbitrary parameters for KyuubiConf
105b70b6 [Kent Yao] tmp
bcc4a0e2 [Kent Yao] tmp
Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
Move `dev/kyuubi-spark-extension*` to `extension/spark/`
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2291 from ulysses-you/unify-spark-extension.
Closes#2292
4e6f4ca6 [ulysses-you] fix
c2cc018d [ulysses-you] unify
2375af0d [ulysses-you] unify
Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
much code about maven options only has `-Pflink-provided,spark-provided` no `hive-provided`.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2219 from deadwind4/KYUUBI-2218.
Closes#2218
5e369978 [Ada Wang] fix build dist help
a5de3ba8 [Ada Wang] [KYUUBI #2218] Fix maven options about hive-provided
Authored-by: Ada Wang <wang4luning@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
### _Why are the changes needed?_
`build/dist` supports hive.
close https://github.com/apache/incubator-kyuubi/issues/2035
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#2169 from cxzl25/KYUUBI-2035.
Closes#2035
69558bb6 [sychen] build/dist supports hive
Authored-by: sychen <sychen@trip.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
