Cheng Pan
b7ac3cb48c
Revert "[KYUUBI #488 ] Add kyuubi-shaded module to avoid conflicts with external jars"
...
This reverts commit e058b87dd2
2021-04-10 00:16:04 +08:00
fwang12
e058b87dd2
[KYUUBI #488 ] Add kyuubi-shaded module to avoid conflicts with external jars
...
 [](https://github.com/yaooqinn/kyuubi/pull/488 )      [<img width="16" alt="Powered by Pull Request Badge" src="https://user-images.githubusercontent.com/1393946/111216524-d2bb8e00-85d4-11eb-821b-ed4c00989c02.png ">](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info )<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues , add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
Shade curator dependency in kyuubi-ha module to avoid impaction by external jars.
### _How was this patch tested?_
Existing UT.
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests ) locally before make a pull request
Closes #488 from turboFei/shade_curator.
Closes #488
c88414c [fwang12] Use wildcards when selecting more than 6 elements
a2dfbf6 [fwang12] add doc for install shaded jars
224cc38 [fwang12] remove transactions
c7b9df3 [fwang12] address comments
81fc123 [fwang12] scala style
408c978 [fwang12] create kyuubi-shaded module to shade dependencies
6b15b45 [fwang12] shade hive service rpc
b4b3b1a [fwang12] include curator-client
65086ed [fwang12] address comments
a46f00f [fwang12] Shade curator dependencies to avoid impaction by external jars
Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: Kent Yao <yao@apache.org>
2021-04-09 11:29:24 +08:00
Cheng Pan
63817634fb
[KYUUBI #432 ] [LICENSE] Introduce Apache Rat
...
 [](https://github.com/yaooqinn/kyuubi/pull/432 )     [❨?❩](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info )<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues , add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests ) locally before make a pull request
Closes #432 from pan3793/rat.
c7cad27 [Cheng Pan] simplify rat rule
24f0688 [Cheng Pan] use rat maven plugin and improve workflow
891ccc1 [Cheng Pan] enable check-license in GH workflow
0bb1c09 [Cheng Pan] [LICENSE] enable apache rat
Authored-by: Cheng Pan <379377944@qq.com>
Signed-off-by: Kent Yao <yao@apache.org>
2021-03-14 00:57:25 +08:00
Kent Yao
354f55a0a5
[KYUUBI #416 ] Detected Dependency List Change for better dependency management
...
 [](https://github.com/yaooqinn/kyuubi/pull/416 )     [❨?❩](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info )<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues , add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
Detect Dependency List Change, to let us and users know what actually release with Kyuubi
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests ) locally before make a pull request
Closes #416 from yaooqinn/dep.
5938423 [Kent Yao] update doc
037c897 [Kent Yao] nit
2b06bf9 [Kent Yao] Detected Dependency List Change for better dependency management
Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
2021-03-11 11:48:14 +08:00
Kent Yao
183e18f1bc
[KYUUBI #404 ] How To Use Spark Dynamic Resource Allocation (DRA) in Kyuubi
...
 [](https://github.com/yaooqinn/kyuubi/pull/404 )     [❨?❩](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info )<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues , add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests ) locally before make a pull request
Closes #404 from yaooqinn/dra.
b8b755c [Kent Yao] How To Use Spark Dynamic Resource Allocation (DRA) in Kyuubi
9bb47b1 [Kent Yao] How To Use Spark Dynamic Resource Allocation (DRA) in Kyuubi
e786787 [Kent Yao] How To Use Spark Dynamic Resource Allocation (DRA) in Kyuubi
Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
2021-03-08 18:27:49 +08:00
Cheng Pan
4abc3ff6a0
[KYUUBI #398 ] Move to hadoop shaded client
...
 [](https://github.com/yaooqinn/kyuubi/pull/399 )    [❨?❩](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info )<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues , add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
close #398
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests ) locally before make a pull request
manual test release workflow
```
➜ kyuubi git:(KYUUBI-398) ll | grep tar.gz
-rw-r--r-- 1 chengpan staff 265M Mar 4 21:38 kyuubi-1.1.0-SNAPSHOT-bin-spark-3.0-hadoop2.7.tar.gz
-rw-r--r-- 1 chengpan staff 269M Mar 4 21:40 kyuubi-1.1.0-SNAPSHOT-bin-spark-3.0-hadoop3.2.tar.gz
-rw-r--r-- 1 chengpan staff 269M Mar 4 21:46 kyuubi-1.1.0-SNAPSHOT-bin-spark-3.1-hadoop2.7.tar.gz
-rw-r--r-- 1 chengpan staff 273M Mar 4 21:44 kyuubi-1.1.0-SNAPSHOT-bin-spark-3.1-hadoop3.2.tar.gz
```
Closes #399 from pan3793/KYUUBI-398.
a9294a1 [Cheng Pan] fix ut
d1d816d [Cheng Pan] fix dist script
2e3bc20 [Cheng Pan] update release workflow and dist script
0428b1b [Cheng Pan] update travis.yml
4a9bc1b [Cheng Pan] [KYUUBI #398 ] move to hadoop shaded client
Authored-by: Cheng Pan <379377944@qq.com>
Signed-off-by: Kent Yao <yao@apache.org>
2021-03-04 23:33:31 +08:00
Kent Yao
874c71c6fe
[KYUUBI #396 ] Add Backward Compatibility Checker for Compile with Spark 3.0 but Run on 3.1
...
 [](https://github.com/yaooqinn/kyuubi/pull/396 )     [❨?❩](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info )<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues , add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
new ci rule to check Backward Compatibility
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests ) locally before make a pull request
Closes #396 from yaooqinn/bc.
ae55ac9 [Kent Yao] ga
bf854bd [Kent Yao] Add Backward Compatibility Checker for Compile with Spark 3.0 but Run on 3.1
Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
2021-03-04 20:57:58 +08:00
Kent Yao
25d2d587b0
[KYUUBI #393 ] Add Support for Hadoop 3.2
...
 [](https://github.com/yaooqinn/kyuubi/pull/393 )     [❨?❩](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info )<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues , add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
This PR Add Support for Hadoop 3.2 Profile to work with spark3.1.1
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests ) locally before make a pull request
Closes #393 from yaooqinn/guava2.
1f56082 [Kent Yao] nit
c8c576c [Kent Yao] nit
ccdfd03 [Kent Yao] Add Support for Hadoop 3.2
Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
2021-03-04 18:25:11 +08:00
Kent Yao
2e3315867b
[KYUUBI #298 ]Add Support for Apache Spark 3.1.1
...
 [](https://github.com/yaooqinn/kyuubi/pull/366 )     [❨?❩](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info )<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues , add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
Spark 3.1.1 released, it looks more stable than the first minor release of Spark 3.x, and a lot of great features were involved.
In this PR, I add a profile for `spark-3.1.1` support, and temporary we only support `hadoop2.7`
TODOS:
- Add Hadoop 3.2 support with some dependency issues
- Enable Iceberg tests for spark-3.1.1 after Apache iceberg‘s next release
- Using spark-3.1.1 as default
- Restore HiveResult string?
fix #298
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests ) locally before make a pull request
Closes #366 from yaooqinn/spark3.1.
28ddf93 [Kent Yao] ga
37cb49e [Kent Yao] Merge branch 'master' into spark3.1
7b332ec [Kent Yao] typo
b6b33d5 [Kent Yao] set up ga
00005c0 [Kent Yao] set up travis
2c09fe4 [Kent Yao] test fix
68c7edb [Kent Yao] deps
2c65e98 [Kent Yao] deps
236c7e9 [Kent Yao] nit
8c8a783 [Kent Yao] Merge branch 'master' into spark3.1
46c318a [Kent Yao] nit
32f6511 [Kent Yao] Merge branch 'master' into spark3.1
27c836b [Kent Yao] Merge branch 'master' into spark3.1
5574a8e [Kent Yao] ga
d2ec2d5 [Kent Yao] [WIP]Spark3.1
49313cc [Kent Yao] [WIP]Travis
ccb1a97 [Kent Yao] [WIP]Spark3.1
Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
2021-03-04 13:30:05 +08:00
Cheng Pan
81e36a1389
[KYUUBI #372 ] Auto upload unit-tests.log
...
 [](https://github.com/yaooqinn/kyuubi/pull/373 )     [❨?❩](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info )<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues , add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
close #372
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests ) locally before make a pull request
Closes #373 from pan3793/KYUUBI-372.
a560827 [Cheng Pan] [KYUUBI #372 ] Auto upload unit-tests.log
Authored-by: Cheng Pan <379377944@qq.com>
Signed-off-by: Kent Yao <yao@apache.org>
2021-02-25 20:55:06 +08:00
Kent Yao
7157370ef6
Add Building Doc
2020-11-05 19:05:26 +08:00
Kent Yao
bd319f4216
remove most tests in kyuubi-server module
2020-11-05 14:14:11 +08:00
Kent Yao
65830f30d8
travis and git action
2020-09-25 17:08:43 +08:00
Kent Yao
1b891ceed5
flags action
2020-09-23 05:00:07 +08:00
Kent Yao
ff15958f8d
action
2020-09-23 03:09:50 +08:00
Kent Yao
b391116175
action
2020-09-23 03:02:23 +08:00
Kent Yao
5481380a37
action
2020-09-23 02:41:28 +08:00
Kent Yao
d1cdb5ebbf
github action code cov
2020-09-22 19:01:32 +08:00
Kent Yao
4f46307d2e
github action code cove
2020-09-22 18:43:43 +08:00
Kent Yao
52e5f4a783
build
2020-09-22 18:08:23 +08:00
Kent Yao
efa78a92f1
improve get schemas test
2020-09-22 17:43:41 +08:00
Kent Yao
269f0ad7cc
fix
2020-09-22 15:12:45 +08:00
Kent Yao
a49660e769
fix
2020-08-21 14:58:35 +08:00
Kent Yao
7682225145
Add github action for pull request and commit ( #240 )
...
* Add github action for pull request and commit
* fix
* ut
2020-08-11 10:23:53 +08:00
