Commit Graph

13 Commits

Author SHA1 Message Date
Cheng Pan
aa6197cdf3
[RELEASE] Bump 1.3.0-incubating 2021-08-17 21:03:13 +08:00
Kent Yao
6dd8dfb70c
[KYUUBI #914] Use isEmpty to trigger default init queries
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

1 use isEmpty instead show for the init SQLs
2 add job desc for SQL tab of Spark UI
3 make it simple.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #914 from yaooqinn/isempty.

Closes #914

70785da7 [Kent Yao] Use isEmpty to trigger default init queries
438009e4 [Kent Yao] Use isEmpty to trigger default init queries

Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
2021-08-10 17:27:47 +08:00
Cheng Pan
4fecf13902 [KYUUBI #888] Add scala version suffix on scala modules
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
- Add scala version suffix on scala modules to match the name pattern.
- Enable `kyuubi-extension-spark-3-1` in daily publish snapshot

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request

Closes #888 from pan3793/scala.

Closes #888

571eb9c3 [Cheng Pan] comments
16cb263a [Cheng Pan] remove hardcode scala version in pom
6090c798 [Cheng Pan] Fix shade
5178847f [Cheng Pan] Fixup
a1547753 [Cheng Pan] Fix engine jar name
2cf4793e [Cheng Pan] fix
cac96531 [Cheng Pan] Enable kyuubi-extension-spark-3-1 in daily publish snapshot
1ca31de5 [Cheng Pan] Add scala version suffix on scala modules

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: ulysses-you <ulyssesyou18@gmail.com>
2021-08-04 09:31:19 +08:00
Cheng Pan
ce5fb97036
[KYUUBI #849] ASF Nexus SNAPSHOT publish
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
https://infra.apache.org/publishing-maven-artifacts.html
https://issues.apache.org/jira/browse/INFRA-22114

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request

Closes #849 from pan3793/asf.

Closes #849

c609f6ba [Cheng Pan] update pom
31f1f2e1 [Cheng Pan] rename root project name to kyuubi-parent
9c5616d8 [Cheng Pan] [INFRA] Update POM for Apache release

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
2021-07-23 13:15:05 +08:00
Cheng Pan
a1cc06ce97
[KYUUBI #689] Fix MetricsSystemSuite on JDK 11
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/NetEase/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
Fix #689.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request

Closes #690 from pan3793/689.

Closes #689

bd64a666 [Cheng Pan] Revert "[GA] Enable JDK 11 in GA"
ab103cf3 [Cheng Pan] [GA] Enable JDK 11 in GA
1762f92b [Cheng Pan] Fix MetricsSystemSuite on G1

Authored-by: Cheng Pan <379377944@qq.com>
Signed-off-by: Kent Yao <yao@apache.org>
2021-06-13 23:15:51 +08:00
Cheng Pan
d20a92e5ae
[KYUUBI #637] [METRICS] Remove unnecessary Guava Cache
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/NetEase/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
`MetricRegistry` hold counters in `ConcurrentHashMap`, it's unnecessary to use Guava Cache.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request

Closes #637 from pan3793/metrics.

Closes #637

aee29f1 [Cheng Pan] remove synchronized
a8d607b [Cheng Pan] [METRICS] Remove unnecessary Guava Cache

Authored-by: Cheng Pan <379377944@qq.com>
Signed-off-by: Kent Yao <yao@apache.org>
2021-05-14 17:56:19 +08:00
Kent Yao
bb9e8a50e5
Prepare Kyuubi v1.3.0 2021-05-13 13:58:20 +08:00
Cheng Pan
0b4f4f242c
[KYUUBI #630] [METRICS] Support expose Prometheus metrics
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/NetEase/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
Support expose Prometheus metrics

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request

Closes #630 from pan3793/prometheus.

Closes #630

fff68b0 [Cheng Pan] nit
47c19eb [Cheng Pan] simplify
203e776 [Cheng Pan] address comments
10a877c [Cheng Pan] synchronized
26eb26e [Cheng Pan] simplify random port
827427c [Cheng Pan] fix import
476ee3a [Cheng Pan] address comments
9787c77 [Cheng Pan] update ut
c8e1b0e [Cheng Pan] update license
2387a97 [Cheng Pan] [METRICS] Support expose Prometheus metrics

Authored-by: Cheng Pan <379377944@qq.com>
Signed-off-by: Kent Yao <yao@apache.org>
2021-05-13 12:32:27 +08:00
Cheng Pan
09c5f9123e [KYUUBI #601] [METRICS] Change metrics config keys
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/NetEase/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
Change the config keys of metrics to get a consistent name style, `metrics.{reporter_type}.{property}`.

The change breaks the compatibility with master, but consider v1.2.0 is not out yet, so changes should be safe.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request

Closes #601 from pan3793/metrics.

Closes #601

89d1974f [Cheng Pan] address comments
361ae6e2 [Cheng Pan] revert default metrics reporter change
1c493e64 [Cheng Pan] rat exclude report.json
fcdb4252 [Cheng Pan] exclude report.json
337e09ae [Cheng Pan] Default enable SFL4J metrics reporter
6b9f3963 [Cheng Pan] [METRICS] Change metrics config keys

Authored-by: Cheng Pan <379377944@qq.com>
Signed-off-by: Cheng Pan <379377944@qq.com>
2021-04-25 18:30:06 +08:00
Kent Yao
c6585b4708
[KYUUBI #471] Separate Embedded Zookeeper server from ha
![yaooqinn](https://badgen.net/badge/Hello/yaooqinn/green) [![Closes #471](https://badgen.net/badge/Preview/Closes%20%23471/blue)](https://github.com/yaooqinn/kyuubi/pull/471) ![386](https://badgen.net/badge/%2B/386/red) ![152](https://badgen.net/badge/-/152/green) ![8](https://badgen.net/badge/commits/8/yellow) ![Target Issue](https://badgen.net/badge/Missing/Target%20Issue/ff0000) [<img width="16" alt="Powered by Pull Request Badge" src="https://user-images.githubusercontent.com/1393946/111216524-d2bb8e00-85d4-11eb-821b-ed4c00989c02.png">](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info)<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->

<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

1. Separated embedded zk from ha module, as it's only used for setup a zk from Kyuubi server for now if there is not an external one
2. Support more configuration for the embedded zookeeper, for example, HOSTNAME, then it's more close to prod usage.
3. TODO: Make the embedded zookeeper of our own, which should support as quorum peers

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request

Closes #471 from yaooqinn/qp.

Closes #471

1c24d02 [Kent Yao] address comments
9666a40 [Kent Yao] refine
f069fc0 [Kent Yao] refine
8bb2f3a [Kent Yao] refine
c20593a [Kent Yao] refine
8fa9a40 [Kent Yao] Merge branch 'master' into qp
1e2e1a3 [Kent Yao] tmp
7d66d05 [Kent Yao] tmp

Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
2021-03-30 18:34:01 +08:00
Kent Yao
392a10ce71
[KYUUBI #458] Mute Console Appender for CI
![yaooqinn](https://badgen.net/badge/Hello/yaooqinn/green) [![Closes #458](https://badgen.net/badge/Preview/Closes%20%23458/blue)](https://github.com/yaooqinn/kyuubi/pull/458) ![66](https://badgen.net/badge/%2B/66/red) ![12](https://badgen.net/badge/-/12/green) ![1](https://badgen.net/badge/commits/1/yellow) ![Target Issue](https://badgen.net/badge/Missing/Target%20Issue/ff0000) [<img width="16" alt="Powered by Pull Request Badge" src="https://user-images.githubusercontent.com/1393946/111216524-d2bb8e00-85d4-11eb-821b-ed4c00989c02.png">](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info)<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->

<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
Rely on the unit-tests.log is better for us to reduce the overhead of CIs

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request

Closes #458 from yaooqinn/log.

7eb1521 [Kent Yao] logbetter

Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
2021-03-26 16:41:56 +08:00
watermen
4fe409562a
[KYUUBI #454] Fix metrics Versions
![watermen](https://badgen.net/badge/Hello/watermen/green) [![Closes #454](https://badgen.net/badge/Preview/Closes%20%23454/blue)](https://github.com/yaooqinn/kyuubi/pull/454) ![27](https://badgen.net/badge/%2B/27/red) ![27](https://badgen.net/badge/-/27/green) ![1](https://badgen.net/badge/commits/1/yellow) ![Target Issue](https://badgen.net/badge/Missing/Target%20Issue/ff0000) ![Test Plan](https://badgen.net/badge/Missing/Test%20Plan/ff0000) [<img width="16" alt="Powered by Pull Request Badge" src="https://user-images.githubusercontent.com/1393946/111216524-d2bb8e00-85d4-11eb-821b-ed4c00989c02.png">](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info)<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->

Closes #454 from watermen/version.

c141344 [watermen] Fix metrics Versions

Authored-by: watermen <qiyadong2010@gmail.com>
Signed-off-by: Kent Yao <yao@apache.org>
2021-03-26 11:56:20 +08:00
Kent Yao
9e7649296d
[KYUUBI #449] Add Basic Metrics Service framework
![yaooqinn](https://badgen.net/badge/Hello/yaooqinn/green) [![Closes #450](https://badgen.net/badge/Preview/Closes%20%23450/blue)](https://github.com/yaooqinn/kyuubi/pull/450) ![847](https://badgen.net/badge/%2B/847/red) ![73](https://badgen.net/badge/-/73/green) ![8](https://badgen.net/badge/commits/8/yellow) [<img width="16" alt="Powered by Pull Request Badge" src="https://user-images.githubusercontent.com/1393946/111216524-d2bb8e00-85d4-11eb-821b-ed4c00989c02.png">](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info)<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->

<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

This PR adds a future-proofing Metrics Service framework close #499

add some reporters, like json file, jmx, etc..
add some basic metrics, like statement.error

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

WIP
- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request

Closes #450 from yaooqinn/449.

ea21a43 [Kent Yao] add tests
6513d07 [Kent Yao] update codecov
7cfe318 [Kent Yao] update docs
a8454d0 [Kent Yao] update docs
91c33f4 [Kent Yao] update deps
05b13fb [Kent Yao] fix dep issues
babc739 [Kent Yao] Add Basic Metrics Service Platform
acc716a [Kent Yao] Add Basic Metrics Service Platform

Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
2021-03-24 18:56:56 +08:00