Commit Graph

59 Commits

Author SHA1 Message Date
Cheng Pan
cdd478400e
[RELEASE] Bump 1.3.1-SNAPSHOT 2021-09-06 15:14:05 +08:00
Cheng Pan
ff9bab6b7a
[RELEASE] Bump 1.3.0-incubating 2021-09-06 14:26:35 +08:00
Cheng Pan
92cc4cd058
[RELEASE] Bump 1.3.1-SNAPSHOT 2021-09-05 00:50:18 +08:00
Cheng Pan
fb3990c647
[RELEASE] Bump 1.3.0-incubating 2021-09-05 00:27:17 +08:00
Cheng Pan
328d5b2d47
[RELEASE] Bump 1.3.1-SNAPSHOT 2021-08-30 15:28:36 +08:00
Cheng Pan
f1319e5098
[RELEASE] Bump 1.3.0-incubating 2021-08-30 15:04:29 +08:00
Cheng Pan
4c47981626
[RELEASE] Bump 1.3.1-SNAPSHOT 2021-08-30 12:49:28 +08:00
Cheng Pan
2536309355
[RELEASE] Bump 1.3.0-incubating 2021-08-30 10:04:28 +08:00
Cheng Pan
0f6e76cd2d
[RELEASE] Bump 1.3.1-SNAPSHOT 2021-08-27 15:09:37 +08:00
Cheng Pan
1ec75b3ef2
[RELEASE] Bump 1.3.0-incubating 2021-08-27 14:54:31 +08:00
Cheng Pan
57dd860721
[RELEASE] Bump 1.3.1-SNAPSHOT 2021-08-25 20:23:40 +08:00
Cheng Pan
95557b5302
[RELEASE] Bump 1.3.0-incubating 2021-08-25 20:02:43 +08:00
Cheng Pan
6ccc00a791
[RELEASE] Bump 1.3.1-SNAPSHOT 2021-08-25 13:59:33 +08:00
Cheng Pan
504b10384b
[RELEASE] Bump 1.3.0-incubating 2021-08-25 13:47:58 +08:00
Cheng Pan
ad3112ab84
[RELEASE] Bump 1.3.1-SNAPSHOT 2021-08-23 09:22:38 +08:00
Cheng Pan
217b00aff5
[RELEASE] Bump 1.3.0-incubating 2021-08-22 16:36:03 +08:00
Cheng Pan
4a436d885d
[RELEASE] Bump 1.3.1-SNAPSHOT 2021-08-22 09:31:28 +08:00
Cheng Pan
8cd28504ac
[RELEASE] Bump 1.3.0-incubating 2021-08-22 08:08:14 +08:00
Cheng Pan
8e11244101
[RELEASE] Bump 1.3.1-SNAPSHOT 2021-08-17 21:34:11 +08:00
Cheng Pan
aa6197cdf3
[RELEASE] Bump 1.3.0-incubating 2021-08-17 21:03:13 +08:00
timothy65535
5526898f7b [KYUUBI #894] Enhance kyuubi-zookeeper module
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

### 1. Describe the feature

Kyuubi use `TestingServer` as the EmbeddedZookeeper service in codebase, and require many args to init the service. These `electionPort`, `quorumPort`, `serverId` parameters only work in cluster mode, these ports are not bound to the host.

From hadoop and bookkeeper project, they use `ZooKeeperServer` as a built-in zk service, especially the bookeeper project.
- https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-registry/src/main/java/org/apache/hadoop/registry/server/services/MicroZookeeperService.java
- https://github.com/apache/bookkeeper/blob/master/bookkeeper-server/src/main/java/org/apache/bookkeeper/shims/zk/ZooKeeperServerShimImpl.java

### 2. Describe the solution

Improve
- Remove `electionPort`, `quorumPort`, `serverId` which are nerver be used, hadoop and bookkeer do the same
- Replace `TestingServer` with `ZooKeeperServer`
- Remove `curator-test` dependency

### 3. `deleteDataDirectoryOnClose`

The name style for `deleteDataDirectoryOnClose` option, refer to hadoop `core-default.xml`
https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/core-default.xml
```
hadoop.security.dns.log-slow-lookups.enabled
hadoop.security.dns.log-slow-lookups.threshold.ms
dfs.client.block.write.replace-datanode-on-failure.policy
```

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request

Closes #907 from timothy65535/ky-894.

Closes #894

7d7fde9f [timothy65535] remove useless config
ce06a0fd [timothy65535] limit travis build only master branch
ff825c3f [timothy65535] revert deleteDataDirectoryOnClose config
a336544a [timothy65535] remove javassist
3da0bd55 [timothy65535] remove javassist
0abb44df [timothy65535] [KYUUBI #894] Enhance kyuubi-zookeeper module

Authored-by: timothy65535 <timothy65535@163.com>
Signed-off-by: ulysses-you <ulyssesyou18@gmail.com>
2021-08-11 10:06:44 +08:00
Cheng Pan
4fecf13902 [KYUUBI #888] Add scala version suffix on scala modules
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
- Add scala version suffix on scala modules to match the name pattern.
- Enable `kyuubi-extension-spark-3-1` in daily publish snapshot

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request

Closes #888 from pan3793/scala.

Closes #888

571eb9c3 [Cheng Pan] comments
16cb263a [Cheng Pan] remove hardcode scala version in pom
6090c798 [Cheng Pan] Fix shade
5178847f [Cheng Pan] Fixup
a1547753 [Cheng Pan] Fix engine jar name
2cf4793e [Cheng Pan] fix
cac96531 [Cheng Pan] Enable kyuubi-extension-spark-3-1 in daily publish snapshot
1ca31de5 [Cheng Pan] Add scala version suffix on scala modules

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: ulysses-you <ulyssesyou18@gmail.com>
2021-08-04 09:31:19 +08:00
ulysses-you
f4fb822743
[KYUUBI #871] Change module name kyuubi-main to kyuubi-server
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
As we will release soon, it's better to format the module name and artifactId.

The name of kyuubi-main sounds like a bootstrap of Kyuubi but not a server.

### _How was this patch tested?_
Pass CI

Closes #872 from ulysses-you/kyuubi-server.

Closes #871

1437f9b5 [ulysses-you] address comment
117f2c7c [ulysses-you] server
86986baa [ulysses-you] master
eba660a2 [ulysses-you] Merge branch 'master' of https://github.com/NetEase/kyuubi into kyuubi-server
e649012b [ulysses-you] change module name

Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2021-07-29 13:41:43 +08:00
Kent Yao
91f622f94f Remove Kyuubi 0.8 and before 2020-11-16 20:12:08 +08:00
Kent Yao
53205a43cc Revert "Revert "fix""
This reverts commit 9f4ccdd70c.
2020-09-23 05:08:26 +08:00
Kent Yao
9f4ccdd70c Revert "fix"
This reverts commit d97f71456b.
2020-09-22 18:33:11 +08:00
Kent Yao
d97f71456b fix 2020-09-22 14:15:58 +08:00
Kent Yao
e22dda7dbd Add more build infomation 2020-09-11 09:55:11 +08:00
Kent Yao
0468fd4d4e Implementaton of KyuubiOperationManager 2020-08-21 10:51:55 +08:00
Kent Yao
86e1a7aac3 SparkProcessBuilder 2020-08-18 17:22:08 +08:00
Kent Yao
7682225145
Add github action for pull request and commit (#240)
* Add github action for pull request and commit

* fix

* ut
2020-08-11 10:23:53 +08:00
Kent Yao
eea65272e2 Add Operation for spark engine 2020-07-31 11:09:11 +08:00
Kent Yao
934a9d3158 Support RowSet 2020-07-08 11:38:48 +08:00
Kent Yao
1a3e886627 drop module kyuubi-hive-thrift 2020-06-23 17:32:43 +08:00
Kent Yao
c2271ac05b Bumpup Spark to 3.0.0 from 3.0.0-preview2 2020-06-18 17:41:43 +08:00
Kent Yao
5100d8a519 Support EmbeddedZkServer 2020-06-12 17:18:46 +08:00
Kent Yao
15d179507f Create Kyuubi Project Spark SQL Engine 2020-06-09 10:34:47 +08:00
Kent Yao
651db99866
HIVE_CLI_SERVICE_PROTOCOL_V11 (#235)
* HIVE_CLI_SERVICE_PROTOCOL_V11

* pom

* style

* tests

* travis

* code cover
2020-05-26 16:07:41 +08:00
Kent Yao
191b8171d6
[KYUUBI-216]Make thrift IDL a separate sub module fix #216 (#217)
* [KYUUBI-216]Make thrift IDL a separate sub module fix #216

* typo
2019-09-25 11:38:35 +08:00
Kent Yao
5edf961a44
[KYUUBI-204]Apache Kudu Integration Guide fix #204 (#205)
* [KYUUBI-204]Apache Kudu Integration Guide fix #204

* references
2019-07-09 16:47:20 +08:00
Kent Yao
e295995959
[KYUUBI-194]Add a server side metrics system (#195)
* fix #194 add server side metrics system

* add spark session cache size

* add ut

* add ut

* add doc
2019-06-12 11:12:18 +08:00
Kent Yao
d7f95775de [BUILD]Bump up to v0.7.0-SNAPSHOT 2019-05-10 11:58:22 +08:00
Kent Yao
1f0bc742e4
[KYUUBI-167][FOLLOWUP]populate tokens via spark session cache mgr (#183)
* Prepare releasing v0.6.0

* fix #KYUUBI-167 populate token via spark session cache mgr

* typo

* fix ut

* code cov
2019-05-07 10:32:04 +08:00
Kent Yao
13c68b5550
[KYUUBI-167]Handling ApplicationMaster token expiry to support long caching sparkcontext with dynamic executor allocation (#168)
* [KYUUBI-167]Handling ApplicationMaster token expiry to support long caching sparkcontext with dynamic executor allocation - fix #167

* fix ut

* add ut

* fix ut

* add doc

* fix https://github.com/apache/spark/pull/24120

* logger

* typo

* comment/dist jar

* fix ut

* typo
2019-03-19 13:51:08 +08:00
Kent Yao
7401ca3c6e
[KYUUBI-135][KYUUBI-142]add support for spark 2.4.0 release (#157)
* fix #135 spark 2.4.0 support

* fix #142

* import

* add ut

* fix ut

* fix ut

* fix ut

* fix ut
2019-03-02 01:16:50 +08:00
Kent Yao
51e7a5d480
Prepare for developing v0.6.0 (#154) 2019-02-27 13:40:29 +08:00
Kent Yao
ae85c245d2
Prepare releasing v0.5.0 (#152)
* Prepare releasing 0.5.0

* update documentations
2019-02-25 12:33:48 +08:00
Kent Yao
694e3ef791 localize remote jars to add them into classpath 2019-02-20 10:21:18 +08:00
Kent Yao
5fe0a34ea4 Preparing v0.5.0 developing 2018-12-21 14:48:05 +08:00
Kent Yao
5a957ef5d8 ignore SparkEnv code cov 2018-11-07 17:13:39 +08:00