Commit Graph

253 Commits

Author SHA1 Message Date
Cheng Pan
216dd66a47
[KYUUBI #1849] Bump Iceberg 0.13.0 and enable Iceberg test with Spark 3.2
### _Why are the changes needed?_

Bump Iceberg 0.13.0 and enable Iceberg test with Spark 3.2.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1849 from pan3793/iceberg-0.13.0.

Closes #1849

39b156b5 [Cheng Pan] update comments
a94eb4ca [Cheng Pan] work around for namespace issue
1c3a4237 [Cheng Pan] RC2 vote passed
f266c0cc [Cheng Pan] Iceberg 0.13.0 RC2
1f32af24 [Cheng Pan] Bump Iceberg 0.13.0 and enable Iceberg test with Spark 3.2

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-02-04 15:00:48 +08:00
Cheng Pan
839c5c5bf4
[KYUUBI #1861] Migrate bind-api and activation-api from javax to jakarta
### _Why are the changes needed?_

The Java world is migrating JavaEE components from `javax` to `jakarta`, we should follow it.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1861 from pan3793/jaxb.

Closes #1861

3563b773 [Cheng Pan] Migrate bind-api activation-api to jakarta

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-02-04 14:59:28 +08:00
Cheng Pan
dd4ae3f410
[KYUUBI #1848] Bump Spark 3.2.1
### _Why are the changes needed?_

Close #1848

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1862 from pan3793/spark-3.2.1.

Closes #1848

306da100 [Cheng Pan] [KYUUBI #1848] Bump Spark 3.2.1

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-02-04 14:58:07 +08:00
Cheng Pan
b2bad04d75 [KYUUBI #1769][FOLLOWUP] Exclude log4j 12 deps from spark-hive and fix detect deps change workflow
### _Why are the changes needed?_

The previous PR did not handle log4j 1.2 deps of `spark-hive`, and the current detect deps change workflow has the dirty cache issue, this PR also fix it.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1856 from pan3793/log4j2.

Closes #1769

2785acf4 [Cheng Pan] nit
58affcb5 [Cheng Pan] update workflow
301acb6f [Cheng Pan] update check deps script
7611c7e0 [Cheng Pan] revert exclusion
239e7cc4 [Cheng Pan] Remove duplicated scalatest
e143fecc [Cheng Pan] dependencyList
a7254771 [Cheng Pan] [KYUUBI #1769][FOLLOWUP] Exclude log4j 12 deps from test deps

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Fei Wang <fwang12@ebay.com>
2022-02-01 23:59:17 +08:00
Paul Lin
8b389f27fd
[KYUUBI #1858] Rename test log4j2 properties to log4j2-test.properties
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

Avoid test log4j2 configurations overtaken by the ones in 3rd party dependencies.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1860 from link3280/bug/KYUUBI-1858.

Closes #1858

614e7937 [Paul Lin] [KYUUBI #1858] Update log2j configuration path in pom.xml
71c95613 [Paul Lin] [KYUUBI #1858] Rename test log4j2 properties to log4j2-test.properties

Authored-by: Paul Lin <paullin3280@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-02-01 01:22:58 +08:00
SteNicholas
463f851351
[KYUUBI #1850] Bump Prometheus 0.14.1
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
Bump Prometheus 0.14.1.

There are some breaking changes, please go to https://github.com/prometheus/client_java/releases to learn details.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1857 from SteNicholas/KYUUBI-1850.

Closes #1850

41c6f682 [SteNicholas] [KYUUBI #1850] Bump Prometheus 0.14.1

Authored-by: SteNicholas <programgeek@163.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-01-31 13:33:27 +08:00
SteNicholas
0e45a4b953
[KYUUBI #1847] Bump Flink 1.14.3
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
Bump Flink 1.14.3.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1851 from SteNicholas/KYUUBI-1847.

Closes #1847

79d5cdc7 [SteNicholas] [KYUUBI #1847] Bump Flink 1.14.3

Authored-by: SteNicholas <programgeek@163.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-01-29 14:37:04 +08:00
Cheng Pan
cebc03af1a
[KYUUBI #1852] Fix UT property of log4j2 conf
### _Why are the changes needed?_

We need use `log4j2.configurationFile` for log4j2 instead of `log4j.configuration`.
https://logging.apache.org/log4j/2.x/manual/configuration.html

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1852 from pan3793/log4j2.

Closes #1852

9e4b3cd4 [Cheng Pan] Fix UT property of log4j2 conf

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-01-29 00:47:54 +08:00
Fei Wang
885ace06c8 [KYUUBI #1769][FOLLOWUP] Some cleanups for log4j properties
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

This is a followup of #1769. Cleanup the log4j properties files and using log4j2.properties for UT.

### _How was this patch tested?_

Passed UT.

Closes #1842 from turboFei/remove_log4j1.

Closes #1769

b6757555 [Fei Wang] unused change
365fbec7 [Fei Wang] revert license
14c64ec9 [Fei Wang] exclude log4j 1.2.17 in slf4j-log4j12
034b04db [Fei Wang] recover jcl-over-slf4j and slf4j-api for k8s it
2beae75e [Fei Wang] remove log4j 1.2.17
b00f07b5 [Fei Wang] remove from license
e55fd2ec [Fei Wang] remove unused dependencies
ab86f023 [Fei Wang] [KYUUBI #1769][FOLLOWUP] Some cleanups for log4j properties

Authored-by: Fei Wang <fwang12@ebay.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
2022-01-28 00:58:25 +08:00
Fei Wang
53d59a02bf [KYUUBI #1769] [BUILD] Migrate from log4j1 to log4j2
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
This patch proposes to migrate from log4j1 to log4j2 in Kyuubi.
Refer the spark patch: https://github.com/apache/spark/pull/34895

### Does this PR introduce any user-facing change?
Yes. Users may need to rewrite their log4j properties file for log4j2. As of version 2.4, log4j now supports configuration via properties files. Note that the property syntax is not the same as the syntax used in log4j 1, but during the migration I found the syntax is pretty close so the migration should be straightforward.

### _How was this patch tested?_
Passed all existing tests.

Closes #1769 from turboFei/log4j2.

Closes #1769

8613779c [Fei Wang] remove log4j dependencies from spark-sql-engine
b2fe6dba [Fei Wang] Use String to present default log level
762e2d03 [Fei Wang] only remove org.apache.logging.log4j:log4j-slf4j-impl
8a912086 [Fei Wang] remove dependencies from spark-sql-engine
7e3a4980 [Fei Wang] address commments
051f49f5 [Fei Wang] address comments
85316a0b [Fei Wang] Keep compatible with log4j12
01d1a84e [Fei Wang] for log4j1
b9e17e1b [Fei Wang] refactor
e24391ed [Fei Wang] revert log count
38803002 [Fei Wang] add log4j2.properties.template
4f0b22fc [Fei Wang] save
7ce84119 [Fei Wang] modify log level
1ea5ca53 [Fei Wang] add log4j to engine
c4a86d4d [Fei Wang] use AbstractFilter
27b08b6a [Fei Wang] remove more
8cc15ae7 [Fei Wang] reformat
c13ec29e [Fei Wang] save temporarily
33a38e2e [Fei Wang] exclude log4j12 from spark-sql
9129a64a [Fei Wang] refactor
5362b43d [Fei Wang] make it run at first
7f27f519 [Fei Wang] more
56f4f1ff [Fei Wang] fix logging
a74b6d37 [Fei Wang] start appender
dea964aa [Fei Wang] fix build erorr at first
e20b7500 [Fei Wang] address comments
2ec02b4d [Fei Wang] fix LogDivertAppender
dded1290 [Fei Wang] more
c63e0008 [Fei Wang] add log4j2.properties

Authored-by: Fei Wang <fwang12@ebay.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
2022-01-26 10:53:16 +08:00
ulysses-you
a533b2fa37
[KYUUBI #1738] Unify kubernetes integration test
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
- Move kubernetes integration test into the new module `integration-test`
- move the kubernetes profile  into `integration-test` module since it is only used in test

### _How was this patch tested?_
Pass CI

Closes #1738 from ulysses-you/unify-it.

Closes #1738

2582823a [ulysses-you] Unify kubernetes integration test

Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
2022-01-13 17:29:30 +08:00
hongdongdong
ab4184dbe3
[KYUUBI #1740] Use testcontainers scalatest in trino module
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
Use testcontainers scalatest instead testcontainers

### _How was this patch tested?_
- [X] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1741 from hddong/use-testcontainers-scala-trino.

Closes #1740

3a12747f [hongdongdong] [KYUUBI #1740] Use testcontainers scalatest in trino module

Authored-by: hongdongdong <hongdongdong@cmss.chinamobile.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-01-13 11:39:18 +08:00
Kent Yao
c1a6cd587c
[KYUUBI #1716] Unify Hive deps for server and client
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

This PR make the hive deps be consistent between server and clients for a single release, we shall not play client-server compatibility tricky in one release

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1716 from yaooqinn/hive.

Closes #1716

2eb3bdf7 [Kent Yao] Merge branch 'master' into hive
c1bf6369 [Kent Yao] dep
c7f0f26f [Kent Yao] dep
645d76ab [Kent Yao] fix test
3ac4f55c [Kent Yao] Merge branch 'master' into hive
f1ca475a [Kent Yao] delta test
01d30291 [Kent Yao] refine
1e93f33a [Kent Yao] Merge branch 'master' of github.com:apache/incubator-kyuubi into hive
5bd96313 [Kent Yao] Unify Hive deps for server and client

Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
2022-01-12 16:36:40 +08:00
yanghua
60ea99225b
[KYUUBI #1732] Reduce spark driver memory configuration to protect fa…
…iling from OOM

<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1734 from yanghua/KYUUBI-1732.

Closes #1732

49f88df9 [yanghua] [KYUUBI #1732] Reduce spark driver memory configuration to protect failing from OOM

Authored-by: yanghua <yanghua1127@gmail.com>
Signed-off-by: Kent Yao <yao@apache.org>
2022-01-12 00:12:14 +08:00
sundapeng
89689c0221 [KYUUBI #1719] javax.activation:activation:jar duplicates
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1725 from SunDapeng1/issues-1719.

Closes #1719

d31d4e49 [sundapeng] [KYUUBI #1719] javax.activation:activation:jar duplicates

Authored-by: sundapeng <sundapeng@cmss.chinamobile.com>
Signed-off-by: hongdongdong <hongdongdong@cmss.chinamobile.com>
2022-01-11 17:14:01 +08:00
yanghua
d63eed2749
[KYUUBI #1709] Introduce a IT module for Kyuubi and FlinkSQL Engine t…
…o decouple with kyuubi server

<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1711 from yanghua/KYUUBI-1709.

Closes #1709

95466130 [yanghua] refactor code
89f9c572 [yanghua] Reorder module list in root pom file
b664a4bb [yanghua] Rename IT module names
6b7fa88f [yanghua] Rename IT module names
b133e8e9 [yanghua] Reformatted pom indent
38444416 [yanghua] [KYUUBI #1709] Introduce a IT module for Kyuubi and FlinkSQL Engine to decouple with kyuubi server

Authored-by: yanghua <yanghua1127@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-01-11 12:45:41 +08:00
Kent Yao
32990b707f
[KYUUBI #1717] Extract rat configurations from pom to file
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

Extract rat configurations from pom to file

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1717 from yaooqinn/rat.

Closes #1717

3f6836fe [Kent Yao] Extract rat configuation from pom to file
cb267b37 [Kent Yao] Extract rat configuation from pom to file

Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
2022-01-11 11:52:18 +08:00
Kent Yao
a9d8fb2e7c
[KYUUBI #1712] Bump up maven.version to 3.8.4
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1712 from yaooqinn/mvnv.

Closes #1712

93793468 [Kent Yao] Bump up maven.version to 3.8.4

Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
2022-01-10 19:21:10 +08:00
hongdongdong
e886524c74
[KYUUBI #1641] Add Trino client
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
Add trino client to communicate with trino cluster.

### _How was this patch tested?_
- [X] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1642 from hddong/add-trino-runner.

Closes #1641

67c2f1f4 [hongdongdong] fix
3e4f7782 [hongdongdong] use testcontainers
3cd4d944 [hongdongdong] [KYUUBI #1641] Add Trino client

Authored-by: hongdongdong <hongdongdong@cmss.chinamobile.com>
Signed-off-by: Kent Yao <yao@apache.org>
2022-01-07 19:58:41 +08:00
Kent Yao
d1de9ca529
[KYUUBI #1658] Revamp swagger ui dependencies
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

This PR mainly removes the swagger-scala-module which is currently fine to remove. It also introduces transitive deps with
`-jakarta` which could introduce jar conflicts like ,

```scala
Caused by: java.lang.NoClassDefFoundError: jakarta/xml/bind/annotation/XmlElement
	at io.swagger.v3.core.jackson.SwaggerAnnotationIntrospector.hasRequiredMarker(SwaggerAnnotationIntrospector.java:29)
	at com.fasterxml.jackson.databind.introspect.AnnotationIntrospectorPair.hasRequiredMarker(AnnotationIntrospectorPair.java:319)
	at com.fasterxml.jackson.databind.introspect.AnnotationIntrospectorPair.hasRequiredMarker(AnnotationIntrospectorPair.java:319)
	at com.fasterxml.jackson.databind.introspect.AnnotationIntrospectorPair.hasRequiredMarker(AnnotationIntrospectorPair.java:319)
	at com.fasterxml.jackson.databind.introspect.POJOPropertyBuilder.getMetadata(POJOPropertyBuilder.java:229)
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1658 from yaooqinn/dep.

Closes #1658

d5ab0765 [Kent Yao] Merge branch 'master' into dep
d1ddd5fd [Kent Yao] Revamp swagger ui dependencies
ecfcc92a [Kent Yao] Revamp swagger ui dependencies
59196b94 [Kent Yao] Revamp swagger ui dependencies
d7ca6a5f [Kent Yao] Revamp swagger ui dependencies
215b1b53 [Kent Yao] Revamp swagger ui dependencies

Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
2021-12-31 09:22:24 +08:00
rayliu
2af105a417 [KYUUBI #1556] Upgrade swagger-ui version from 4.1.0 to 4.1.3
### _Why are the changes needed?_
Close https://github.com/apache/incubator-kyuubi/issues/1612

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1630 from ldzhjn/1612.

Closes #1556

71fe05bc [rayliu] Upgrade swagger-ui version from 4.1.0 to 4.1.3

Authored-by: rayliu <rayliu@cisco.com>
Signed-off-by: yanghua <yanghua1127@gmail.com>
2021-12-25 20:50:56 +08:00
yanghua
3673399329
[KYUUBI #1579] Implement basic ability of executing statement in Flink engine
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1603 from yanghua/KYUUBI-1579.

Closes #1579

48db76b3 [Cheng Pan] cleanup
36707516 [Cheng Pan] Address comments
25ca5ae2 [yanghua] reduce code
6f18a4a0 [yanghua] [KYUUBI #1579] Implement basic ability of executing statement

Lead-authored-by: yanghua <yanghua1127@gmail.com>
Co-authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2021-12-22 22:55:43 +08:00
Cheng Pan
d529402168
[KYUUBI #1587] [SUB-TASK][KPIP-2] Bump Flink 1.14.2
### _Why are the changes needed?_

Bump the latest version of 1.14.x to suppress log4-shell issue

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1587 from pan3793/flink-version.

Closes #1587

9d7ca949 [Cheng Pan] Bump Flink 1.14.2

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
2021-12-20 18:19:30 +08:00
yanghua
01364e589f
[KYUUBI #1578] Introduce flink-sql-client dependency to reduce some duplicated code
…uplicated code

<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1584 from yanghua/KYUUBI-1578.

Closes #1578

90c2f89e [yanghua] [KYUUBI #1578] Introduce flink-sql-client dependency to reduce some duplicated code

Authored-by: yanghua <yanghua1127@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2021-12-17 16:30:33 +08:00
yanghua
4b58b0ab9b
[KYUUBI #1555][FLINK] Bump flink version to 1.14.0
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1573 from yanghua/KYUUBI-1555.

Closes #1555

4f5ce746 [yanghua] [KYUUBI #1555] [SUB-TASK][KPIP-2] Bump flink version to 1.14.1

Authored-by: yanghua <yanghua1127@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2021-12-16 16:59:38 +08:00
Cheng Pan
022f52bc7e
[KYUUBI #1460] Bump Hudi 0.10.0
### _Why are the changes needed?_

https://hudi.apache.org/releases/release-0.10.0

Switch test from Hudi 0.9.0 & Spark 3.0 to Hudi 1.10.0 & Spark 3.1. Drop test of Spark 3.0 because

1. Hudi 1.10.0 claims to support Spark 3.0 but requires to build by self with specific maven profile, the published jar only support Spark 3.1
2. Hudi has lots of transitive deps and there are different in different versions, it's hard to maintain multi hudi versions.

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1460 from pan3793/hudi.

Closes #1460

bac633a7 [Cheng Pan] Update comments
0ab8784d [Cheng Pan] Pin hadoop-common
c811e679 [Cheng Pan] Exclude scala-library
63cad3b9 [Cheng Pan] Exclude hudi-aws
a13708f9 [Cheng Pan] Disable Hudi test on Spark 3.0
4b179bae [Cheng Pan] Enable Hudi test in Spark 3.1
79dca54c [Cheng Pan] Bump Hudi 0.10.0
71982890 [Cheng Pan] Add default Pk col

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
2021-12-16 10:13:42 +08:00
Cheng Pan
77a278707f
[KYUUBI #1559] Bump Jersey 2.34
### _Why are the changes needed?_

#1561 and align Spark 3.2.0

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1559 from pan3793/jersey.

Closes #1559

a86d4d9f [Cheng Pan] Bump Jersey 2.34

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
2021-12-16 10:12:46 +08:00
Cheng Pan
f841da4b4e
[KYUUBI #1558] Bump Jackson 2.12.5
### _Why are the changes needed?_

Because Spark 3.2, Trino 363, Jersey 2.34 (we use Jersey 2.30 now, will update to 2.34 soon) use Jackson 2.12.x

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1558 from pan3793/jackson.

Closes #1558

7bdd65b0 [Cheng Pan] Bump Jackson 2.12.5

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2021-12-15 21:21:01 +08:00
Cheng Pan
6fa1327f4e
[KYUUBI #1560] Align all Jackson libs version
### _Why are the changes needed?_

Fix missing version of `jackson-module-jaxb-annotations` in `dependencyManagement`

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1560 from pan3793/jackson-1.

Closes #1560

49529d47 [Cheng Pan] Update deps
b3610007 [Cheng Pan] fix
67044840 [Cheng Pan] Algin all Jackson libs version

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2021-12-15 12:02:10 +08:00
hongdongdong
c1a0ce8e01
[KYUUBI #1515] Add RowSet for trino engine
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
Add RowSet for trino engine

### _How was this patch tested?_
- [X] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1549 from hddong/add-rowset-for-trino.

Closes #1515

cc885130 [hongdongdong] remove timeZone
2fb10934 [hongdongdong] fix toHiveString
9066caff [hongdongdong] rm optionalStart
491b7682 [hongdongdong] Reset
27895ebe [hongdongdong] fix RowSetUtils
7ce4edeb [hongdongdong] fix
34dd01b8 [hongdongdong] move functions to Utils
79e0621a [hongdongdong] fix
2dcd91e5 [hongdongdong] fix
71bdcfd2 [hongdongdong] [KYUUBI #1515] Add RowSet for trino engine

Authored-by: hongdongdong <hongdongdong@cmss.chinamobile.com>
Signed-off-by: Kent Yao <yao@apache.org>
2021-12-14 20:52:20 +08:00
yanghua
53ff56be29
[KYUUBI #1525][FLINK] Implement GetCatalogs operation and do the further initialization
…

<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1529 from yanghua/KYUUBI-1525.

Closes #1525

e166965d [yanghua] Address review suggestion
9929aa29 [yanghua] Remove some comment
eafb200b [yanghua] refactor code
58df34c7 [yanghua] refactor code
675e14e7 [yanghua] pruning unnecessary code
15e97283 [yanghua] pruning unnecessary code
c6d4f549 [yanghua] pruning unnecessary code
b099a455 [yanghua] pruning unnecessary code
316086bb [yanghua] pruning unnecessary code
fe7af8ef [yanghua] pruning unnecessary code
ecf3ccf4 [yanghua] pruning unnecessary code
4c174517 [yanghua] pruning unnecessary code
b9226c3e [yanghua] pruning unnecessary code
fb2f6a62 [yanghua] pruning unnecessary code
21e21a50 [yanghua] pruning unnecessary code
a58076ff [yanghua] pruning unnecessary code
acd24fc5 [yanghua] pruning unnecessary code
333d4275 [yanghua] pruning unnecessary code
7416b3de [yanghua] pruning unnecessary code
84d0db91 [yanghua] pruning unnecessary code and refactor code
a55db41a [yanghua] pruning unnecessary code
b9909308 [yanghua] [KYUUBI #1525] Implement GetCatalogs operation and do the further initialization

Authored-by: yanghua <yanghua1127@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2021-12-14 17:40:11 +08:00
Fei Wang
609bb5a878 [KYUUBI #1535] Remove duplicate hive client version property
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

Remove duplicate hive.cleint.version property, it should be involved when adding kyuubi-hive-beeline module.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1535 from turboFei/typo.

Closes #1535

56e734fa [Fei Wang] Remove duplicate hive client version property

Authored-by: Fei Wang <fwang12@ebay.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
2021-12-09 18:37:13 +08:00
hongdongdong
6ea83e040e
[KYUUBI #1511] Initialize kyuubi trino engine module
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
1. Initialize kyuubi trino engine module
2. Add `TrinoThriftBinaryFrontendService`

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1514 from hddong/init-trino-engine.

Closes #1511

9dd0e55f [hongdongdong] [KYUUBI #1511] Initialize kyuubi trino engine module

Authored-by: hongdongdong <hongdongdong@cmss.chinamobile.com>
Signed-off-by: Kent Yao <yao@apache.org>
2021-12-07 13:33:35 +08:00
yanghua
339fa7682a
[KYUUBI #1492] Initialize kyuubi flink sql engine module
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1497 from yanghua/KYUUBI-1492.

Closes #1492

ba9f221a [yanghua] [KYUUBI #1492] Initialize kyuubi flink sql engine module

Authored-by: yanghua <yanghua1127@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2021-12-06 15:45:44 +08:00
Kent Yao
97c3835738
[KYUUBI #1490] Introduce the basic framework for running scala
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

Introduce the basic framework for running scala, see #1490 for the detail
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [x] Add screenshots for manual tests if appropriate
```
Beeline version 1.5.0-SNAPSHOT by Apache Kyuubi (Incubating)
0: jdbc:hive2://10.242.189.214:10009/> spark.version;
2021-12-03 13:47:07.556 INFO operation.ExecuteStatement: Processing kent's query[08b8b6da-d434-4296-b613-2027e3518441]: INITIALIZED_STATE -> PENDING_STATE, statement: spark.version
2021-12-03 13:47:07.560 INFO operation.ExecuteStatement: Processing kent's query[08b8b6da-d434-4296-b613-2027e3518441]: PENDING_STATE -> RUNNING_STATE, statement: spark.version
2021-12-03 13:47:07.558 INFO operation.ExecuteStatement: Processing kent's query[321dc15d-68d0-4f91-9216-1e08f09842df]: INITIALIZED_STATE -> PENDING_STATE, statement: spark.version
2021-12-03 13:47:07.559 INFO operation.ExecuteStatement: Processing kent's query[321dc15d-68d0-4f91-9216-1e08f09842df]: PENDING_STATE -> RUNNING_STATE, statement: spark.version
2021-12-03 13:47:07.560 INFO operation.ExecuteStatement:
           Spark application name: kyuubi_USER_SPARK_SQL_kent_default_61cff9fb-7035-4435-b509-80c1730876ed
                 application ID: local-1638510289918
                 application web UI: http://10.242.189.214:65027
                 master: local[*]
                 deploy mode: client
                 version: 3.1.2
           Start time: 2021-12-03T13:44:49.313
           User: kent
2021-12-03 13:47:07.562 INFO scheduler.DAGScheduler: Asked to cancel job group 321dc15d-68d0-4f91-9216-1e08f09842df
2021-12-03 13:47:07.565 INFO operation.ExecuteStatement: Processing kent's query[321dc15d-68d0-4f91-9216-1e08f09842df]: RUNNING_STATE -> ERROR_STATE, statement: spark.version, time taken: 0.006 seconds
2021-12-03 13:47:07.565 ERROR operation.ExecuteStatement: Error operating EXECUTE_STATEMENT: org.apache.spark.sql.catalyst.parser.ParseException:
mismatched input 'spark' expecting {'(', 'ADD', 'ALTER', 'ANALYZE', 'CACHE', 'CLEAR', 'COMMENT', 'COMMIT', 'CREATE', 'DELETE', 'DESC', 'DESCRIBE', 'DFS', 'DROP', 'EXPLAIN', 'EXPORT', 'FROM', 'GRANT', 'IMPORT', 'INSERT', 'LIST', 'LOAD', 'LOCK', 'MAP', 'MERGE', 'MSCK', 'REDUCE', 'REFRESH', 'REPLACE', 'RESET', 'REVOKE', 'ROLLBACK', 'SELECT', 'SET', 'SHOW', 'START', 'TABLE', 'TRUNCATE', 'UNCACHE', 'UNLOCK', 'UPDATE', 'USE', 'VALUES', 'WITH'}(line 1, pos 0)

== SQL ==
spark.version
^^^

	at org.apache.spark.sql.catalyst.parser.ParseException.withCommand(ParseDriver.scala:255)
	at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parse(ParseDriver.scala:124)
	at org.apache.spark.sql.execution.SparkSqlParser.parse(SparkSqlParser.scala:49)
	at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parsePlan(ParseDriver.scala:75)
	at org.apache.spark.sql.SparkSession.$anonfun$sql$2(SparkSession.scala:616)
	at org.apache.spark.sql.catalyst.QueryPlanningTracker.measurePhase(QueryPlanningTracker.scala:111)
	at org.apache.spark.sql.SparkSession.$anonfun$sql$1(SparkSession.scala:616)
	at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:775)
	at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:613)
	at org.apache.kyuubi.engine.spark.operation.ExecuteStatement.$anonfun$executeStatement$1(ExecuteStatement.scala:98)
	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
	at org.apache.kyuubi.engine.spark.operation.ExecuteStatement.withLocalProperties(ExecuteStatement.scala:157)
	at org.apache.kyuubi.engine.spark.operation.ExecuteStatement.org$apache$kyuubi$engine$spark$operation$ExecuteStatement$$executeStatement(ExecuteStatement.scala:92)
	at org.apache.kyuubi.engine.spark.operation.ExecuteStatement$$anon$1.run(ExecuteStatement.scala:125)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

org.apache.spark.sql.catalyst.parser.ParseException:
mismatched input 'spark' expecting {'(', 'ADD', 'ALTER', 'ANALYZE', 'CACHE', 'CLEAR', 'COMMENT', 'COMMIT', 'CREATE', 'DELETE', 'DESC', 'DESCRIBE', 'DFS', 'DROP', 'EXPLAIN', 'EXPORT', 'FROM', 'GRANT', 'IMPORT', 'INSERT', 'LIST', 'LOAD', 'LOCK', 'MAP', 'MERGE', 'MSCK', 'REDUCE', 'REFRESH', 'REPLACE', 'RESET', 'REVOKE', 'ROLLBACK', 'SELECT', 'SET', 'SHOW', 'START', 'TABLE', 'TRUNCATE', 'UNCACHE', 'UNLOCK', 'UPDATE', 'USE', 'VALUES', 'WITH'}(line 1, pos 0)

== SQL ==
spark.version
^^^

	at org.apache.spark.sql.catalyst.parser.ParseException.withCommand(ParseDriver.scala:255)
	at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parse(ParseDriver.scala:124)
	at org.apache.spark.sql.execution.SparkSqlParser.parse(SparkSqlParser.scala:49)
	at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parsePlan(ParseDriver.scala:75)
	at org.apache.spark.sql.SparkSession.$anonfun$sql$2(SparkSession.scala:616)
	at org.apache.spark.sql.catalyst.QueryPlanningTracker.measurePhase(QueryPlanningTracker.scala:111)
	at org.apache.spark.sql.SparkSession.$anonfun$sql$1(SparkSession.scala:616)
	at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:775)
	at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:613)
	at org.apache.kyuubi.engine.spark.operation.ExecuteStatement.$anonfun$executeStatement$1(ExecuteStatement.scala:98)
	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
	at org.apache.kyuubi.engine.spark.operation.ExecuteStatement.withLocalProperties(ExecuteStatement.scala:157)
	at org.apache.kyuubi.engine.spark.operation.ExecuteStatement.org$apache$kyuubi$engine$spark$operation$ExecuteStatement$$executeStatement(ExecuteStatement.scala:92)
	at org.apache.kyuubi.engine.spark.operation.ExecuteStatement$$anon$1.run(ExecuteStatement.scala:125)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
2021-12-03 13:47:07.569 INFO operation.ExecuteStatement: Query[08b8b6da-d434-4296-b613-2027e3518441] in ERROR_STATE
2021-12-03 13:47:07.569 INFO operation.ExecuteStatement: Processing kent's query[08b8b6da-d434-4296-b613-2027e3518441]: RUNNING_STATE -> ERROR_STATE, statement: spark.version, time taken: 0.009 seconds
Error: Error operating EXECUTE_STATEMENT: org.apache.spark.sql.catalyst.parser.ParseException:
mismatched input 'spark' expecting {'(', 'ADD', 'ALTER', 'ANALYZE', 'CACHE', 'CLEAR', 'COMMENT', 'COMMIT', 'CREATE', 'DELETE', 'DESC', 'DESCRIBE', 'DFS', 'DROP', 'EXPLAIN', 'EXPORT', 'FROM', 'GRANT', 'IMPORT', 'INSERT', 'LIST', 'LOAD', 'LOCK', 'MAP', 'MERGE', 'MSCK', 'REDUCE', 'REFRESH', 'REPLACE', 'RESET', 'REVOKE', 'ROLLBACK', 'SELECT', 'SET', 'SHOW', 'START', 'TABLE', 'TRUNCATE', 'UNCACHE', 'UNLOCK', 'UPDATE', 'USE', 'VALUES', 'WITH'}(line 1, pos 0)

== SQL ==
spark.version
^^^

	at org.apache.spark.sql.catalyst.parser.ParseException.withCommand(ParseDriver.scala:255)
	at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parse(ParseDriver.scala:124)
	at org.apache.spark.sql.execution.SparkSqlParser.parse(SparkSqlParser.scala:49)
	at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parsePlan(ParseDriver.scala:75)
	at org.apache.spark.sql.SparkSession.$anonfun$sql$2(SparkSession.scala:616)
	at org.apache.spark.sql.catalyst.QueryPlanningTracker.measurePhase(QueryPlanningTracker.scala:111)
	at org.apache.spark.sql.SparkSession.$anonfun$sql$1(SparkSession.scala:616)
	at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:775)
	at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:613)
	at org.apache.kyuubi.engine.spark.operation.ExecuteStatement.$anonfun$executeStatement$1(ExecuteStatement.scala:98)
	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
	at org.apache.kyuubi.engine.spark.operation.ExecuteStatement.withLocalProperties(ExecuteStatement.scala:157)
	at org.apache.kyuubi.engine.spark.operation.ExecuteStatement.org$apache$kyuubi$engine$spark$operation$ExecuteStatement$$executeStatement(ExecuteStatement.scala:92)
	at org.apache.kyuubi.engine.spark.operation.ExecuteStatement$$anon$1.run(ExecuteStatement.scala:125)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748) (state=,code=0)
0: jdbc:hive2://10.242.189.214:10009/> set kyuubi.operation.language=scala;
2021-12-03 13:47:11.982 INFO operation.ExecuteStatement: Processing kent's query[e4e4dc55-4e05-460e-9d9d-86b2cf5c054b]: INITIALIZED_STATE -> PENDING_STATE, statement: set kyuubi.operation.language=scala
2021-12-03 13:47:11.985 INFO operation.ExecuteStatement: Processing kent's query[e4e4dc55-4e05-460e-9d9d-86b2cf5c054b]: PENDING_STATE -> RUNNING_STATE, statement: set kyuubi.operation.language=scala
2021-12-03 13:47:11.983 INFO operation.ExecuteStatement: Processing kent's query[2b6ec68c-2ae9-4d93-a35b-9e2319df4d4f]: INITIALIZED_STATE -> PENDING_STATE, statement: set kyuubi.operation.language=scala
2021-12-03 13:47:11.984 INFO operation.ExecuteStatement: Processing kent's query[2b6ec68c-2ae9-4d93-a35b-9e2319df4d4f]: PENDING_STATE -> RUNNING_STATE, statement: set kyuubi.operation.language=scala
2021-12-03 13:47:11.985 INFO operation.ExecuteStatement:
           Spark application name: kyuubi_USER_SPARK_SQL_kent_default_61cff9fb-7035-4435-b509-80c1730876ed
                 application ID: local-1638510289918
                 application web UI: http://10.242.189.214:65027
                 master: local[*]
                 deploy mode: client
                 version: 3.1.2
           Start time: 2021-12-03T13:44:49.313
           User: kent
2021-12-03 13:47:11.995 INFO operation.ExecuteStatement: Processing kent's query[2b6ec68c-2ae9-4d93-a35b-9e2319df4d4f]: RUNNING_STATE -> RUNNING_STATE, statement: set kyuubi.operation.language=scala
2021-12-03 13:47:11.995 INFO operation.ExecuteStatement: Execute in full collect mode
2021-12-03 13:47:12.006 INFO operation.ExecuteStatement: Processing kent's query[2b6ec68c-2ae9-4d93-a35b-9e2319df4d4f]: RUNNING_STATE -> FINISHED_STATE, statement: set kyuubi.operation.language=scala, time taken: 0.022 seconds
2021-12-03 13:47:12.007 INFO operation.ExecuteStatement: Query[e4e4dc55-4e05-460e-9d9d-86b2cf5c054b] in FINISHED_STATE
2021-12-03 13:47:12.007 INFO operation.ExecuteStatement: Processing kent's query[e4e4dc55-4e05-460e-9d9d-86b2cf5c054b]: RUNNING_STATE -> FINISHED_STATE, statement: set kyuubi.operation.language=scala, time taken: 0.022 seconds
+----------------------------+--------+
|            key             | value  |
+----------------------------+--------+
| kyuubi.operation.language  | scala  |
+----------------------------+--------+
1 row selected (0.052 seconds)
0: jdbc:hive2://10.242.189.214:10009/> spark.version;
2021-12-03 13:47:13.685 INFO operation.ExecuteStatement: Processing kent's query[178ff72d-b870-44d4-99d8-08a0fa0e8efa]: INITIALIZED_STATE -> PENDING_STATE, statement: spark.version
2021-12-03 13:47:15.541 INFO operation.ExecuteStatement: Processing kent's query[178ff72d-b870-44d4-99d8-08a0fa0e8efa]: PENDING_STATE -> RUNNING_STATE, statement: spark.version
2021-12-03 13:47:15.543 INFO operation.ExecuteStatement: Query[178ff72d-b870-44d4-99d8-08a0fa0e8efa] in FINISHED_STATE
2021-12-03 13:47:15.544 INFO operation.ExecuteStatement: Processing kent's query[178ff72d-b870-44d4-99d8-08a0fa0e8efa]: RUNNING_STATE -> FINISHED_STATE, statement: spark.version, time taken: 0.003 seconds
+-----------------------+
|        output         |
+-----------------------+
| res0: String = 3.1.2  |
+-----------------------+
1 row selected (1.871 seconds)
0: jdbc:hive2://10.242.189.214:10009/> spark.sql("select current_date()")
. . . . . . . . . . . . . . . . . . .> ;
2021-12-03 13:47:36.512 INFO operation.ExecuteStatement: Processing kent's query[602c2d88-7e8a-4175-9b53-e34adfff007a]: INITIALIZED_STATE -> PENDING_STATE, statement: spark.sql("select current_date()")
2021-12-03 13:47:36.689 INFO operation.ExecuteStatement: Processing kent's query[602c2d88-7e8a-4175-9b53-e34adfff007a]: PENDING_STATE -> RUNNING_STATE, statement: spark.sql("select current_date()")
2021-12-03 13:47:36.692 INFO operation.ExecuteStatement: Query[602c2d88-7e8a-4175-9b53-e34adfff007a] in FINISHED_STATE
2021-12-03 13:47:36.692 INFO operation.ExecuteStatement: Processing kent's query[602c2d88-7e8a-4175-9b53-e34adfff007a]: RUNNING_STATE -> FINISHED_STATE, statement: spark.sql("select current_date()"), time taken: 0.003 seconds
+----------------------------------------------------+
|                       output                       |
+----------------------------------------------------+
| res1: org.apache.spark.sql.DataFrame = [current_date(): date] |
+----------------------------------------------------+
1 row selected (0.187 seconds)
0: jdbc:hive2://10.242.189.214:10009/> results += spark.range(1, 5, 2, 3);
Error: Error operating EXECUTE_STATEMENT: org.apache.kyuubi.KyuubiSQLException: Interpret error:
results += spark.range(1, 5, 2, 3)
 <console>:26: error: type mismatch;
 found   : org.apache.spark.sql.Dataset[Long]
 required: org.apache.spark.sql.Dataset[org.apache.spark.sql.Row]
       results += spark.range(1, 5, 2, 3)
                             ^
	at org.apache.kyuubi.KyuubiSQLException$.apply(KyuubiSQLException.scala:69)
	at org.apache.kyuubi.engine.spark.operation.ExecuteScala.runInternal(ExecuteScala.scala:70)
	at org.apache.kyuubi.operation.AbstractOperation.run(AbstractOperation.scala:130)
	at org.apache.kyuubi.session.AbstractSession.runOperation(AbstractSession.scala:93)
	at org.apache.kyuubi.engine.spark.session.SparkSessionImpl.runOperation(SparkSessionImpl.scala:62)
	at org.apache.kyuubi.session.AbstractSession.$anonfun$executeStatement$1(AbstractSession.scala:121)
	at org.apache.kyuubi.session.AbstractSession.withAcquireRelease(AbstractSession.scala:75)
	at org.apache.kyuubi.session.AbstractSession.executeStatement(AbstractSession.scala:118)
	at org.apache.kyuubi.service.AbstractBackendService.executeStatement(AbstractBackendService.scala:61)
	at org.apache.kyuubi.service.ThriftBinaryFrontendService.ExecuteStatement(ThriftBinaryFrontendService.scala:265)
	at org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1557)
	at org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1542)
	at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
	at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
	at org.apache.kyuubi.service.authentication.TSetIpAddressProcessor.process(TSetIpAddressProcessor.scala:36)
	at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:310)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748) (state=,code=0)
0: jdbc:hive2://10.242.189.214:10009/> results += spark.range(1, 5, 2, 3).toDF;
2021-12-03 13:48:30.327 INFO operation.ExecuteStatement: Processing kent's query[daccd163-978a-4d66-af85-af466086d1db]: INITIALIZED_STATE -> PENDING_STATE, statement: results += spark.range(1, 5, 2, 3).toDF
2021-12-03 13:48:31.700 INFO operation.ExecuteStatement: Processing kent's query[daccd163-978a-4d66-af85-af466086d1db]: PENDING_STATE -> RUNNING_STATE, statement: results += spark.range(1, 5, 2, 3).toDF
2021-12-03 13:48:31.702 INFO operation.ExecuteStatement: Query[daccd163-978a-4d66-af85-af466086d1db] in FINISHED_STATE
2021-12-03 13:48:31.702 INFO operation.ExecuteStatement: Processing kent's query[daccd163-978a-4d66-af85-af466086d1db]: RUNNING_STATE -> FINISHED_STATE, statement: results += spark.range(1, 5, 2, 3).toDF, time taken: 0.003 seconds
+-----+
| id  |
+-----+
| 1   |
| 3   |
+-----+
2 rows selected (1.387 seconds)
```

** Session level isolated **

![image](https://user-images.githubusercontent.com/8326978/144553902-fc390c30-06de-453b-af3d-cf8def577aa9.png)

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1491 from yaooqinn/scala.

Closes #1490

af4d0a13 [Kent Yao] Merge branch 'master' into scala
2ebdf6a3 [Kent Yao] provided scala dep
d58b2bfe [Kent Yao] [KYUUBI #1490] Introduce the basic framework for running scala
d256bfde [Kent Yao] [KYUUBI #1490] Introduce the basic framework for running scala
52090434 [Kent Yao] init

Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
2021-12-06 11:17:59 +08:00
Cheng Pan
5dba80ccfd
[KYUUBI #1486] Enable delta test on Spark 3.2 with delta 1.1.0
### _Why are the changes needed?_

Delta 1.1.0 is out, compatible with Spark 3.2

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1486 from pan3793/delta.

Closes #1486

8dce5373 [Cheng Pan] Enable delta test on Spark 3.2 with delta 1.1.0

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2021-12-03 13:36:14 +08:00
fwang12
119e7f2c2c [KYUUBI #1441] [BUILD] Bump 1.5.0-SNAPSHOT
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1441 from turboFei/bump_1.5.0.

Closes #1441

0c03c75d [fwang12] [BUILD] Bump 1.5.0-SNAPSHOT

Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: fwang12 <fwang12@ebay.com>
2021-11-23 22:22:26 +08:00
Cheng Pan
6bb5af609c
[KYUUBI #1433] Enable scalafmt check in spotless
### _Why are the changes needed?_

fail compile if violating scalafmt rules.

```
[ERROR] Failed to execute goal com.diffplug.spotless:spotless-maven-plugin:2.17.4:check (default) on project kyuubi-common_2.12: The following files had format violations:
[ERROR]     src/test/scala/org/apache/kyuubi/operation/JDBCTestHelper.scala
[ERROR]          -52,7 +52,7
[ERROR]          ··}
[ERROR]
[ERROR]          ··def·withMultipleConnectionJdbcStatement(
[ERROR]         -····tableNames:·String*)(fs:·(Statement·=>·Unit)*):·Unit·=·{
[ERROR]         +······tableNames:·String*)(fs:·(Statement·=>·Unit)*):·Unit·=·{
[ERROR]          ····val·connections·=·fs.map·{·_·=>·DriverManager.getConnection(jdbcUrlWithConf,·user,·password)·}
[ERROR]          ····val·statements·=·connections.map(_.createStatement())
[ERROR]
[ERROR] Run 'mvn spotless:apply' to fix these violations.
```
Can auto-fix by `dev/reformat` or `build/mvn spotless:apply`

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1433 from pan3793/scalafmt.

Closes #1433

26188a54 [Cheng Pan] Enable scalafmt check in spotless

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
2021-11-23 15:23:36 +08:00
fwang12
55a7347b6f
[KYUUBI #1425] Remove hardcode skipTests for maven-surefire-plugin
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

So that we can transfer skipTests variable when build kyuubi binary.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1425 from turboFei/java_tst.

Closes #1425

094904d2 [fwang12] typo
af447e8c [fwang12] prevent hardcode

Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
2021-11-22 09:50:46 +08:00
fwang12
21cc217281 [KYUUBI #1419] Refactor hive client related property names
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
Refactor the kyuubi client related property names from `hive.cli` to `hive.client`.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1419 from turboFei/refactor_hive_cli.

Closes #1419

de532387 [fwang12] cli -> client

Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: fwang12 <fwang12@ebay.com>
2021-11-19 16:09:43 +08:00
fwang12
e5e49b5b1d
[KYUUBI #1414] Add kyuubi-hive-beeline module to support KyuubiConnection and KyuubiStatement
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
Add kyuubi-hive-beeline module to support KyuubiConnection and KyuubiStatement.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [x] Add screenshots for manual tests if appropriate
<img width="1410" alt="Screen Shot 2021-11-19 at 1 56 21 PM" src="https://user-images.githubusercontent.com/6757692/142572735-409afe25-b96a-49da-8191-e9a8ced3947b.png">

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1414 from turboFei/rebase-kyuubi_beeline.

Closes #1414

4694bbca [fwang12] return engine launch handle anyway
20ca1400 [fwang12] address comments
2999c30a [fwang12] fix ut
4d3abf4f [fwang12] jars.target.dir to target
4659f1f6 [fwang12] add readme
7e74c5c8 [fwang12] integrate beeline and server
969af045 [fwang12] fix beeline shell
90395ade [fwang12] BeeLine version By Apache Kyuubi
4e63695e [fwang12] fix ut
026e41ef [fwang12] refactor properties
a49677af [fwang12] refactor build and beeline script
ca68eba1 [fwang12] use beeline mode property instead of static method
4a1932e7 [fwang12] cli -> beeline
624281e3 [fwang12] nit
d7d27a09 [fwang12] refactor kyuubi cli assembly
9d9b25a9 [fwang12] simplify the waitLaunchEngineToComplete
cc025ecf [fwang12] Rebase PR_1407

Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
2021-11-19 15:05:10 +08:00
sunfangbin
e54fabab68
[KYUUBI #1416] Refine jline dependency
### _Why are the changes needed?_
Follows on #1413.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1416 from murong00/branch-1415.

Closes #1416

05335c45 [sunfangbin] Refine jline dependency

Authored-by: sunfangbin <sunfangbin@cmss.chinamobile.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2021-11-19 10:40:40 +08:00
sunfangbin
fb8bf49dbc
[KYUUBI #1413] Improve usage of zookeeper cli
### _Why are the changes needed?_
This change enabled the JLine support so that the `kyuubi-zookeeper-shell` command can be easier to use with history, prompt and completion, etc. Note that the `jline` version `0.9.94` is dependent on `zookeeper` version `3.4.14` which is a lower version. BTW, users can consider upgrading the `zookeeper` version for the sake of performance improvements and new features.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1413 from murong00/branch-1413.

Closes #1413

2d1ec084 [sunfangbin] Rename shell script to zk-cli
d09f64dc [sunfangbin] Fixes failed ci
ce88357a [sunfangbin] Improve usage of zookeeper cli

Authored-by: sunfangbin <sunfangbin@cmss.chinamobile.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2021-11-18 19:17:11 +08:00
yanghua
2baabf46ca
[KYUUBI #1384] Introduce spotless and google java format for java code
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1403 from yanghua/KYUUBI-1384.

Closes #1384

e4e12088 [yanghua] address review suggestion
17a4cb86 [yanghua] reformat all the java files based on the offical google java code style
10a2ca04 [yanghua] [KYUUBI #1384] Introduce Java code style and formatter

Authored-by: yanghua <yanghua1127@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2021-11-18 10:45:51 +08:00
Fu Chen
a5ef94accc
[KYUUBI #1354] Expose OpenAPI Specifications
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

1. add a new endpoint `/openapi.json` which describes the entire restful API of Kyuubi.
2. add a static web page `/api/v1/swagger-ui`. see [swagger-ui](https://github.com/swagger-api/swagger-ui)

```
curl http://localhost:10099/openapi.json | jq .
```

```json
{
  "openapi": "3.0.1",
  "info": {
    "title": "Apache Kyuubi RESTful Api Documentation",
    "description": "Apache Kyuubi RESTful Api Documentation",
    "contact": {
      "name": "Apache Kyuubi Community",
      "url": "https://kyuubi.apache.org/issue_tracking.html",
      "email": "devkyuubi.apache.org"
    },
    "license": {
      "name": "Apache 2.0",
      "url": "https://www.apache.org/licenses/LICENSE-2.0.html"
    },
    "version": "1.4.0"
  },
  "tags": [
    {
      "name": "Session"
    }
  ],
  "paths": {
    "/api/v1/ping": {
      "get": {
        "operationId": "ping",
        "responses": {
          "default": {
            "description": "default response",
            "content": {
              "text/plain": {
                "schema": {
                  "type": "string"
                }
              }
            }
          }
        }
      }
    },
    "/api/v1/sessions/{sessionHandle}": {
      "get": {
        "tags": [
          "Session"
        ],
        "operationId": "sessionInfo",
        "parameters": [
          {
            "name": "sessionHandle",
            "in": "path",
            "required": true,
            "schema": {
              "type": "string"
            }
          }
        ],
        "responses": {
          "200": {
            "description": "get a session via session handle identifier",
            "content": {
              "application/json": {}
            }
          }
        }
      },
      "delete": {
        "tags": [
          "Session"
        ],
        "operationId": "closeSession",
        "parameters": [
          {
            "name": "sessionHandle",
            "in": "path",
            "required": true,
            "schema": {
              "type": "string"
            }
          }
        ],
        "responses": {
          "200": {
            "description": "close a Session",
            "content": {
              "application/json": {}
            }
          }
        }
      }
    },
    "/api/v1/sessions/execPool/statistic": {
      "get": {
        "tags": [
          "Session"
        ],
        "operationId": "execPoolStatistic",
        "responses": {
          "200": {
            "description": "get some statistic info of sessions",
            "content": {
              "application/json": {}
            }
          }
        }
      }
    },
    "/api/v1/sessions/{sessionHandle}/info/{infoType}": {
      "get": {
        "tags": [
          "Session"
        ],
        "operationId": "getInfo",
        "parameters": [
          {
            "name": "sessionHandle",
            "in": "path",
            "required": true,
            "schema": {
              "type": "string"
            }
          },
          {
            "name": "infoType",
            "in": "path",
            "required": true,
            "schema": {
              "type": "integer",
              "format": "int32"
            }
          }
        ],
        "responses": {
          "200": {
            "description": "get a information detail via session handle identifier and a specific information type",
            "content": {
              "application/json": {}
            }
          }
        }
      }
    },
    "/api/v1/sessions": {
      "get": {
        "tags": [
          "Session"
        ],
        "operationId": "sessionInfoList",
        "responses": {
          "200": {
            "description": "get all the session list hosted in SessionManager",
            "content": {
              "application/json": {}
            }
          }
        }
      },
      "post": {
        "tags": [
          "Session"
        ],
        "operationId": "openSession",
        "requestBody": {
          "content": {
            "application/json": {
              "schema": {
                "$ref": "#/components/schemas/SessionOpenRequest"
              }
            }
          }
        },
        "responses": {
          "200": {
            "description": "Open(create) a Session",
            "content": {
              "application/json": {}
            }
          }
        }
      }
    },
    "/api/v1/sessions/count": {
      "get": {
        "tags": [
          "Session"
        ],
        "operationId": "sessionCount",
        "responses": {
          "200": {
            "description": "get open session count",
            "content": {
              "application/json": {}
            }
          }
        }
      }
    },
    "/api/v1/exception": {
      "get": {
        "operationId": "test",
        "responses": {
          "default": {
            "description": "default response",
            "content": {
              "text/plain": {}
            }
          }
        }
      }
    },
    "/application.wadl/{path}": {
      "get": {
        "operationId": "getExternalGrammar",
        "parameters": [
          {
            "name": "path",
            "in": "path",
            "required": true,
            "schema": {
              "type": "string"
            }
          }
        ],
        "responses": {
          "default": {
            "description": "default response",
            "content": {
              "application/xml": {}
            }
          }
        }
      }
    },
    "/application.wadl": {
      "get": {
        "operationId": "getWadl",
        "responses": {
          "default": {
            "description": "default response",
            "content": {
              "application/vnd.sun.wadl+xml": {},
              "application/xml": {}
            }
          }
        }
      }
    }
  },
  "components": {
    "schemas": {
      "ExecPoolStatistic": {
        "required": [
          "execPoolSize"
        ],
        "type": "object",
        "properties": {
          "execPoolSize": {
            "type": "integer",
            "format": "int32"
          },
          "execPoolActiveCount": {
            "type": "integer",
            "format": "int32"
          }
        }
      },
      "InfoDetail": {
        "required": [
          "infoType"
        ],
        "type": "object",
        "properties": {
          "infoType": {
            "type": "string"
          },
          "infoValue": {
            "type": "string"
          }
        }
      },
      "HandleIdentifier": {
        "required": [
          "publicId"
        ],
        "type": "object",
        "properties": {
          "publicId": {
            "type": "string",
            "format": "uuid"
          },
          "secretId": {
            "type": "string",
            "format": "uuid"
          }
        }
      },
      "SessionHandle": {
        "required": [
          "identifier",
          "protocol"
        ],
        "type": "object",
        "properties": {
          "identifier": {
            "$ref": "#/components/schemas/HandleIdentifier"
          },
          "protocol": {
            "type": "string",
            "enum": [
              "HIVE_CLI_SERVICE_PROTOCOL_V1",
              "HIVE_CLI_SERVICE_PROTOCOL_V2",
              "HIVE_CLI_SERVICE_PROTOCOL_V3",
              "HIVE_CLI_SERVICE_PROTOCOL_V4",
              "HIVE_CLI_SERVICE_PROTOCOL_V5",
              "HIVE_CLI_SERVICE_PROTOCOL_V6",
              "HIVE_CLI_SERVICE_PROTOCOL_V7",
              "HIVE_CLI_SERVICE_PROTOCOL_V8",
              "HIVE_CLI_SERVICE_PROTOCOL_V9",
              "HIVE_CLI_SERVICE_PROTOCOL_V10"
            ]
          }
        }
      },
      "MapStringString": {
        "type": "object",
        "additionalProperties": {
          "type": "string"
        }
      },
      "SessionOpenRequest": {
        "required": [
          "configs",
          "protocolVersion",
          "user"
        ],
        "type": "object",
        "properties": {
          "protocolVersion": {
            "type": "integer",
            "format": "int32"
          },
          "user": {
            "type": "string"
          },
          "password": {
            "type": "string"
          },
          "ipAddr": {
            "type": "string"
          },
          "configs": {
            "type": "object",
            "additionalProperties": {
              "type": "string"
            }
          }
        }
      },
      "SessionOpenCount": {
        "required": [
          "openSessionCount"
        ],
        "type": "object",
        "properties": {
          "openSessionCount": {
            "type": "integer",
            "format": "int32"
          }
        }
      },
      "SessionDetail": {
        "required": [
          "configs",
          "createTime",
          "user"
        ],
        "type": "object",
        "properties": {
          "user": {
            "type": "string"
          },
          "ipAddr": {
            "type": "string"
          },
          "createTime": {
            "type": "integer",
            "format": "int64"
          },
          "sessionHandle": {
            "$ref": "#/components/schemas/SessionHandle"
          },
          "lastAccessTime": {
            "type": "integer",
            "format": "int64"
          },
          "lastIdleTime": {
            "type": "integer",
            "format": "int64"
          },
          "noOperationTime": {
            "type": "integer",
            "format": "int64"
          },
          "configs": {
            "type": "object",
            "additionalProperties": {
              "type": "string"
            }
          }
        }
      },
      "SeqSessionOverview": {
        "type": "array",
        "items": {
          "$ref": "#/components/schemas/SessionOverview"
        }
      },
      "SessionList": {
        "required": [
          "sessionList"
        ],
        "type": "object",
        "properties": {
          "sessionList": {
            "type": "array",
            "items": {
              "$ref": "#/components/schemas/SessionOverview"
            }
          }
        }
      },
      "SessionOverview": {
        "required": [
          "createTime",
          "user"
        ],
        "type": "object",
        "properties": {
          "user": {
            "type": "string"
          },
          "ipAddr": {
            "type": "string"
          },
          "createTime": {
            "type": "integer",
            "format": "int64"
          },
          "sessionHandle": {
            "$ref": "#/components/schemas/SessionHandle"
          }
        }
      }
    }
  }
}
```
http://localhost:10099/api/v1/swagger-ui

> Note that this request will redirect to http://localhost:10099/swagger-ui-redirected/index.html?url=http://localhost:10099/openapi.json

![截屏2021-11-09 上午10 43 49](https://user-images.githubusercontent.com/8537877/141281596-e47f5552-75b9-47d1-8a36-67337cb6eb3a.png)

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1363 from cfmcgrady/kyuubi-1354.

Closes #1354

8895d7fc [Fu Chen] typo
8f5dea9a [Fu Chen] addressed comment
4d325a45 [Fu Chen] constant MediaType.APPLICATION_JSON
0d96951e [Fu Chen] RESTful Api -> REST API
be7d02dc [Fu Chen] add LIABILITY-edl.txt
4ff1053e [Fu Chen] update dev/dependencyList
b126f7e6 [Fu Chen] swagger-ui 4.1.0
a35c2f29 [Fu Chen] addressed comment
7bde0534 [Fu Chen] update LICENSE-binary
38397dc2 [Fu Chen] align jackson version
923f3e7e [Fu Chen] update dev/dependencyList
7d5338c8 [Fu Chen] remove kyuubi-server static files
5cb66b33 [Fu Chen] update dev/dependencyList
e8a8c421 [Fu Chen] fix style
5b13f69c [Fu Chen] add license and remove .map files
c04d7a75 [Fu Chen] rebase master
2388dae8 [Fu Chen] fix style
8dcbd3db [Fu Chen] add static resource
4b0d6d0a [Fu Chen] expose OpenAPI

Authored-by: Fu Chen <cfmcgrady@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
2021-11-17 12:17:58 +08:00
fwang12
ed43391f8c [KYUUBI #1381] Rewrite kyuubi hive jdbc module based on hive branch-2.3 code base
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

Hive code base:
8e7f23f34b

commit 8e7f23f34b2ce7328c9d571a13c336f0c8cdecb6 (HEAD, origin/branch-2.3)
Author: Neelesh Srinivas Salian <nssalianusers.noreply.github.com>
Date:   Tue Oct 19 12:50:31 2021 -0700

    HIVE-25616: Hive-24741 backport to 2.3 (Neelesh Srinivas Salian reviewed by Vihang Karajgaonkar)

    Closes (#2730)

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1381 from turboFei/kyuubi_hive_jdbc.

Closes #1381

f39f1a1d [fwang12] adreess comments
5612f490 [fwang12] remove kyuubi-hive-jdbc from code cov
f5a6376e [fwang12] do not fail if no tests
4e99ffcb [fwang12] check kyuubi-hive-jdbc code coverage
1c5f2e16 [fwang12] refactor the test classes
d7f9c7d7 [fwang12] copy test classes from hive branch-2.3
a28c077a [fwang12] rename classname in comments
42f7c5ea [fwang12] try to remove hive-jdbc dependency
fcf5953a [fwang12] use kyuubi hive jdbc utils
17402f4d [fwang12] replace licenses
5cbca43d [fwang12] add missing class ignored by .gitignore
b2c19334 [fwang12] revert docs change
bdc860b4 [fwang12] backport kyuubiHiveDriver
04731e72 [fwang12] make some rename
9d57076b [fwang12] remove from org.apache.hive.jdbc to org.apache.kyuubi.jdbc.hive
86813e6b [fwang12] Backport KyuubiDriver
42f748d6 [fwang12] Backport KyuubiDatabaseMetaData.getTables and for KyuubiConnection.getMetaData, it is not needed
f0d2aedb [fwang12] rename hive classes to kyuubi version
3ea8ba73 [fwang12] copy the source code from hive branch-2.3
7db4e5ac [fwang12] remove at first

Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: fwang12 <fwang12@ebay.com>
2021-11-15 15:38:38 +08:00
Cheng Pan
c122a23a0d
[KYUUBI #1348] [DEPS] Add netty and mysql jdbc driver to kyuubi-server module
### _Why are the changes needed?_

This PR is one part of #1334.

Add netty to `kyuubi-server` module compile scope, and mysql jdbc driver to test scope, it's required for implementing MySQL frontend protocol

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1348 from pan3793/mysql-fe-deps.

Closes #1348

288ad9c0 [Cheng Pan] [DEPS] Add netty to kyuubi-server module

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2021-11-09 20:31:09 +08:00
Cheng Pan
0b43a8bb78
[KYUUBI #1338] Bump Iceberg 0.12.1
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
Test Iceberg 0.12.1 RC0

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1338 from pan3793/iceberg-0.12.1.

Closes #1338

d2dd4c1f [Cheng Pan] Remove staging repo
a16103e8 [Cheng Pan] Test Iceberg 0.12.1 RC0

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2021-11-09 12:28:36 +08:00
yanghua
aad575ce3e
[KYUUBI #1329] Support download Flink binary package in kyuubi-download module
…

<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1339 from yanghua/KYUUBI-1329.

Closes #1329

895815fb [yanghua] address review suggestion
eddf0605 [yanghua] [KYUUBI #1329] Support download Flink binary package in kyuubi-download module

Authored-by: yanghua <yanghua1127@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
2021-11-05 18:03:57 +08:00
Brian Yue
7665339244
[KYUUBI #460] Upgrade Hive dependency to 2.3.9
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
Close #460.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1179 from byyue/feature-460.

Closes #460

92eb71b8 [Brian Yue] Upgrade Hive dependency to 2.3.9

Authored-by: Brian Yue <code.byyue@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2021-10-23 13:47:22 +08:00