Commit Graph

14 Commits

Author SHA1 Message Date
ulysses-you
42d57e9bd8 [KYUUBI #636] Add kyuubi-extension-spark_3.1 in release script
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/NetEase/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
Make `kyuubi-extension-spark_3.1` module packaged during release.

### _How was this patch tested?_

Closes #636 from ulysses-you/release-script.

Closes #636

363f74b5 [ulysses-you] fix

Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou18@gmail.com>
2021-05-14 09:07:51 +08:00
Cheng Pan
8fb6ffdf84 [KYUUBI #535] [BUILD] In favor using profile in maven command
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/NetEase/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
Recap the background of Hadoop dependencies in Kyuubi.

1. Kyuubi Server depends on few Hadoop UGI classes to support Kerbose, and those are pretty stable APIs through decade. So we decided use hadoop shaded client instead of hadoop vanilla client to avoid transitive deps, see #398.
2. Current Kyuubi build against Spark 3.0, 3.1 and 3.2-SNOTSHOT(master), all of them support and dist with both hadoop-2.7 and hadoop-3.2.

The property `hadoop.binary.version` only affect which `spark-bin-hadoop-{hadoop-version}.tgz` we will use to run integration tests in `kyuubi-main` module and include into Kyuubi binary dist bundle, and kyuubi server itself always depends on hadoop shaded client 3.2.2, thus we named the maven profile `spark-hadoop-2.7/3.2`.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request

Closes #535 from pan3793/maven.

Closes #535

8959c797 [Cheng Pan] [BUILD] In favor using profile in maven command

Authored-by: Cheng Pan <379377944@qq.com>
Signed-off-by: Cheng Pan <379377944@qq.com>
2021-04-17 11:07:07 +08:00
ulysses-you
e971732e1d [KYUUBI #407] Auto update release note
![ulysses-you](https://badgen.net/badge/Hello/ulysses-you/green) [![Closes #407](https://badgen.net/badge/Preview/Closes%20%23407/blue)](https://github.com/yaooqinn/kyuubi/pull/407) ![5](https://badgen.net/badge/%2B/5/red) ![0](https://badgen.net/badge/-/0/green) ![6](https://badgen.net/badge/commits/6/yellow) ![Target Issue](https://badgen.net/badge/Missing/Target%20Issue/ff0000) ![Test Plan](https://badgen.net/badge/Missing/Test%20Plan/ff0000) [&#10088;?&#10089;](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info)<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->

<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

### _How was this patch tested?_

Closes #407 from ulysses-you/auto-update-release-note.

4dc09ae [ulysses-you] Release Changelog Builder
cfbe23c [ulysses-you] remove
8fcb56c [ulysses-you] changelog
28e8618 [ulysses-you] release
e953874 [ulysses-you] config
d792eea [ulysses-you] init

Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou18@gmail.com>
2021-03-09 10:36:25 +08:00
Kent Yao
016277d183
[KYUUBI #406] Create GitHub Release only once
![yaooqinn](https://badgen.net/badge/Hello/yaooqinn/green) [![Closes #406](https://badgen.net/badge/Preview/Closes%20%23406/blue)](https://github.com/yaooqinn/kyuubi/pull/406) ![10](https://badgen.net/badge/%2B/10/red) ![10](https://badgen.net/badge/-/10/green) ![1](https://badgen.net/badge/commits/1/yellow) ![Target Issue](https://badgen.net/badge/Missing/Target%20Issue/ff0000) [&#10088;?&#10089;](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info)<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->

<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request

Closes #406 from yaooqinn/release.

2fc8754 [Kent Yao] Create GitHub Release only once

Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
2021-03-08 20:08:26 +08:00
Kent Yao
183e18f1bc
[KYUUBI #404] How To Use Spark Dynamic Resource Allocation (DRA) in Kyuubi
![yaooqinn](https://badgen.net/badge/Hello/yaooqinn/green) [![Closes #404](https://badgen.net/badge/Preview/Closes%20%23404/blue)](https://github.com/yaooqinn/kyuubi/pull/404) ![518](https://badgen.net/badge/%2B/518/red) ![443](https://badgen.net/badge/-/443/green) ![3](https://badgen.net/badge/commits/3/yellow) ![Target Issue](https://badgen.net/badge/Missing/Target%20Issue/ff0000) [&#10088;?&#10089;](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info)<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->

<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request

Closes #404 from yaooqinn/dra.

b8b755c [Kent Yao] How To Use Spark Dynamic Resource Allocation (DRA) in Kyuubi
9bb47b1 [Kent Yao] How To Use Spark Dynamic Resource Allocation (DRA) in Kyuubi
e786787 [Kent Yao] How To Use Spark Dynamic Resource Allocation (DRA) in Kyuubi

Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
2021-03-08 18:27:49 +08:00
Cheng Pan
8e83245d7a [KYUUBI #402] Remove hadoop suffix from tarball name when dist —without-spark
![pan3793](https://badgen.net/badge/Hello/pan3793/green) [![Closes #402](https://badgen.net/badge/Preview/Closes%20%23402/blue)](https://github.com/yaooqinn/kyuubi/pull/402) ![3](https://badgen.net/badge/%2B/3/red) ![4](https://badgen.net/badge/-/4/green) ![2](https://badgen.net/badge/commits/2/yellow) ![Target Issue](https://badgen.net/badge/Missing/Target%20Issue/ff0000) [&#10088;?&#10089;](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info)<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->

<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [x] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request

`build/dist --tgz --spark-provided`

previous:
```
kyuubi-1.1.0-SNAPSHOT-bin-without-spark-hadoop3.2.tar.gz
```

after patch:
```
kyuubi-1.1.0-SNAPSHOT-bin-without-spark.tar.gz
```

Closes #402 from pan3793/build.

a534426 [Cheng Pan] update release workflow
1629489 [Cheng Pan] Remove hadoop suffix from tarball name when dist —without-spark

Authored-by: Cheng Pan <379377944@qq.com>
Signed-off-by: Cheng Pan <379377944@qq.com>
2021-03-05 13:49:34 +08:00
Cheng Pan
4abc3ff6a0
[KYUUBI #398] Move to hadoop shaded client
![pan3793](https://badgen.net/badge/Hello/pan3793/green) [![Closes #399](https://badgen.net/badge/Preview/Closes%20%23399/blue)](https://github.com/yaooqinn/kyuubi/pull/399) ![101](https://badgen.net/badge/%2B/101/red) ![41](https://badgen.net/badge/-/41/green) ![5](https://badgen.net/badge/commits/5/yellow) [&#10088;?&#10089;](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info)<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->

<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
close #398

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [x] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request

manual test release workflow
```
➜  kyuubi git:(KYUUBI-398) ll | grep tar.gz
-rw-r--r--   1 chengpan  staff   265M Mar  4 21:38 kyuubi-1.1.0-SNAPSHOT-bin-spark-3.0-hadoop2.7.tar.gz
-rw-r--r--   1 chengpan  staff   269M Mar  4 21:40 kyuubi-1.1.0-SNAPSHOT-bin-spark-3.0-hadoop3.2.tar.gz
-rw-r--r--   1 chengpan  staff   269M Mar  4 21:46 kyuubi-1.1.0-SNAPSHOT-bin-spark-3.1-hadoop2.7.tar.gz
-rw-r--r--   1 chengpan  staff   273M Mar  4 21:44 kyuubi-1.1.0-SNAPSHOT-bin-spark-3.1-hadoop3.2.tar.gz
```

Closes #399 from pan3793/KYUUBI-398.

a9294a1 [Cheng Pan] fix ut
d1d816d [Cheng Pan] fix dist script
2e3bc20 [Cheng Pan] update release workflow and dist script
0428b1b [Cheng Pan] update travis.yml
4a9bc1b [Cheng Pan] [KYUUBI #398] move to hadoop shaded client

Authored-by: Cheng Pan <379377944@qq.com>
Signed-off-by: Kent Yao <yao@apache.org>
2021-03-04 23:33:31 +08:00
Kent Yao
25d2d587b0
[KYUUBI #393] Add Support for Hadoop 3.2
![yaooqinn](https://badgen.net/badge/Hello/yaooqinn/green) [![Closes #393](https://badgen.net/badge/Preview/Closes%20%23393/blue)](https://github.com/yaooqinn/kyuubi/pull/393) ![42](https://badgen.net/badge/%2B/42/red) ![41](https://badgen.net/badge/-/41/green) ![3](https://badgen.net/badge/commits/3/yellow) ![Target Issue](https://badgen.net/badge/Missing/Target%20Issue/ff0000) [&#10088;?&#10089;](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info)<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->

<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

This PR Add Support for Hadoop 3.2 Profile to work with spark3.1.1

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request

Closes #393 from yaooqinn/guava2.

1f56082 [Kent Yao] nit
c8c576c [Kent Yao] nit
ccdfd03 [Kent Yao] Add Support for Hadoop 3.2

Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
2021-03-04 18:25:11 +08:00
Cheng Pan
08dbe24d28
[KYUUBI #380] [FOLLOWUP #377] change default distribution bundle name, and distribute without spark bundle
![pan3793](https://badgen.net/badge/Hello/pan3793/green) [![Closes #380](https://badgen.net/badge/Preview/Closes%20%23380/blue)](https://github.com/yaooqinn/kyuubi/pull/380) ![4](https://badgen.net/badge/%2B/4/red) ![2](https://badgen.net/badge/-/2/green) ![1](https://badgen.net/badge/commits/1/yellow) ![Target Issue](https://badgen.net/badge/Missing/Target%20Issue/ff0000) [&#10088;?&#10089;](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info)<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->

<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

This is a followup of #377, change list:

- change the spark version suffix in default distribution bundle name from `spark-{major}.{minor}.{patch}` to `spark-{major}.{minor}`
- release distribution bundles both with spark and without spark

Consider Kyuubi 1.1.x has not released, the available distribution still in old style, so now just keep the doc as is.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request

- [x] Manual test

`build/dist --tgz`
```
➜  kyuubi git:(master) ✗ ls | grep tar.gz
kyuubi-1.1.0-SNAPSHOT-bin-spark-3.0.tar.gz
```

Closes #380 from pan3793/build.

36fa006 [Cheng Pan] [FOLLOWUP #377] change default distribution bundle name, and distribute without spark bundle

Authored-by: Cheng Pan <379377944@qq.com>
Signed-off-by: Kent Yao <yao@apache.org>
2021-02-28 23:05:36 +08:00
Kent Yao
15820a2b09 fix 2020-11-16 19:42:06 +08:00
Kent Yao
7bfc4703f3 add github action for release last 2020-11-13 18:51:17 +08:00
Kent Yao
4b0e0ee1fe add github action for release 2020-11-13 17:39:55 +08:00
Kent Yao
99abbe4389 add github action for release 2020-11-13 17:37:46 +08:00
Kent Yao
218dd045db add github action for release 2020-11-13 17:32:36 +08:00