Commit Graph

14 Commits

Author SHA1 Message Date
Cheng Pan
98f38ba517
[BUILD] Bump 1.4.0-SNAPSHOT 2021-08-17 01:39:06 +08:00
Cheng Pan
ce5fb97036
[KYUUBI #849] ASF Nexus SNAPSHOT publish
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
https://infra.apache.org/publishing-maven-artifacts.html
https://issues.apache.org/jira/browse/INFRA-22114

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request

Closes #849 from pan3793/asf.

Closes #849

c609f6ba [Cheng Pan] update pom
31f1f2e1 [Cheng Pan] rename root project name to kyuubi-parent
9c5616d8 [Cheng Pan] [INFRA] Update POM for Apache release

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
2021-07-23 13:15:05 +08:00
Kent Yao
bb9e8a50e5
Prepare Kyuubi v1.3.0 2021-05-13 13:58:20 +08:00
Kent Yao
5e2a1290c3
Prepare Kyuubi v1.2.0 2021-03-08 19:50:12 +08:00
Kent Yao
2e3315867b
[KYUUBI #298]Add Support for Apache Spark 3.1.1
![yaooqinn](https://badgen.net/badge/Hello/yaooqinn/green) [![Closes #366](https://badgen.net/badge/Preview/Closes%20%23366/blue)](https://github.com/yaooqinn/kyuubi/pull/366) ![359](https://badgen.net/badge/%2B/359/red) ![58](https://badgen.net/badge/-/58/green) ![17](https://badgen.net/badge/commits/17/yellow) ![Feature](https://badgen.net/badge/Label/Feature/) [&#10088;?&#10089;](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info)<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->

<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

Spark 3.1.1 released, it looks more stable than the first minor release of Spark 3.x, and a lot of great features were involved.

In this PR, I add a profile for `spark-3.1.1` support, and temporary we only support `hadoop2.7`

TODOS:
- Add Hadoop 3.2 support with some dependency issues
- Enable Iceberg tests for spark-3.1.1 after Apache iceberg‘s next release
- Using spark-3.1.1 as default
- Restore HiveResult string?

fix #298
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request

Closes #366 from yaooqinn/spark3.1.

28ddf93 [Kent Yao] ga
37cb49e [Kent Yao] Merge branch 'master' into spark3.1
7b332ec [Kent Yao] typo
b6b33d5 [Kent Yao] set up ga
00005c0 [Kent Yao] set up travis
2c09fe4 [Kent Yao] test fix
68c7edb [Kent Yao] deps
2c65e98 [Kent Yao] deps
236c7e9 [Kent Yao] nit
8c8a783 [Kent Yao] Merge branch 'master' into spark3.1
46c318a [Kent Yao] nit
32f6511 [Kent Yao] Merge branch 'master' into spark3.1
27c836b [Kent Yao] Merge branch 'master' into spark3.1
5574a8e [Kent Yao] ga
d2ec2d5 [Kent Yao] [WIP]Spark3.1
49313cc [Kent Yao] [WIP]Travis
ccb1a97 [Kent Yao] [WIP]Spark3.1

Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
2021-03-04 13:30:05 +08:00
Cheng Pan
2f8df2355d [KYUUBI #377] build/dist support --spark-provided
![pan3793](https://badgen.net/badge/Hello/pan3793/green) [![Closes #378](https://badgen.net/badge/Preview/Closes%20%23378/blue)](https://github.com/yaooqinn/kyuubi/pull/378) ![55](https://badgen.net/badge/%2B/55/red) ![13](https://badgen.net/badge/-/13/green) ![3](https://badgen.net/badge/commits/3/yellow) [&#10088;?&#10089;](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info)<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->

<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
close #377

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request

- [x] Manual test

`build/dist --spark-provided`

```
(kyuubi) ➜  kyuubi git:(master) ✗ tree dist
dist
├── LICENSE
├── RELEASE
├── bin
│   ├── kyuubi
│   ├── kyuubi-logo
│   └── load-kyuubi-env.sh
├── conf
│   ├── kyuubi-defaults.conf.template
│   ├── kyuubi-env.sh.template
│   └── log4j.properties.template
├── externals
│   └── engines
│       └── spark
│           └── kyuubi-spark-sql-engine-1.1.0-SNAPSHOT.jar
├── jars
│   ├── apacheds-i18n-2.0.0-M15.jar
│   ├── apacheds-kerberos-codec-2.0.0-M15.jar
│   ├── api-asn1-api-1.0.0-M20.jar
│   ├── api-util-1.0.0-M20.jar
│   ├── audience-annotations-0.5.0.jar
│   ├── commons-beanutils-1.7.0.jar
│   ├── commons-beanutils-core-1.8.0.jar
│   ├── commons-codec-1.4.jar
│   ├── commons-collections-3.2.2.jar
│   ├── commons-configuration-1.6.jar
│   ├── commons-digester-1.8.jar
│   ├── commons-io-2.4.jar
│   ├── commons-lang-2.6.jar
│   ├── commons-lang3-3.10.jar
│   ├── commons-math-2.2.jar
│   ├── curator-client-2.7.1.jar
│   ├── curator-framework-2.7.1.jar
│   ├── curator-recipes-2.7.1.jar
│   ├── curator-test-2.7.1.jar
│   ├── guava-11.0.2.jar
│   ├── hadoop-annotations-2.7.4.jar
│   ├── hadoop-auth-2.7.4.jar
│   ├── hadoop-common-2.7.4.jar
│   ├── hive-service-rpc-2.3.7.jar
│   ├── httpclient-4.5.6.jar
│   ├── httpcore-4.4.12.jar
│   ├── javassist-3.18.1-GA.jar
│   ├── jcl-over-slf4j-1.7.30.jar
│   ├── jsr305-3.0.2.jar
│   ├── kyuubi-common-1.1.0-SNAPSHOT.jar
│   ├── kyuubi-ha-1.1.0-SNAPSHOT.jar
│   ├── kyuubi-main-1.1.0-SNAPSHOT.jar
│   ├── libfb303-0.9.3.jar
│   ├── libthrift-0.9.3.jar
│   ├── log4j-1.2.17.jar
│   ├── netty-3.7.0.Final.jar
│   ├── scala-library-2.12.10.jar
│   ├── slf4j-api-1.7.30.jar
│   ├── slf4j-log4j12-1.7.30.jar
│   ├── spotbugs-annotations-3.1.9.jar
│   └── zookeeper-3.4.14.jar
├── logs
├── pid
└── work
```

Bundle size
```
(kyuubi) ➜  kyuubi git:(KYUUBI-377) ll | grep tar.gz
-rw-r--r--   1 chengpan  staff   230M Feb 26 12:59 kyuubi-1.1.0-SNAPSHOT-bin-spark-3.0.2.tar.gz
-rw-r--r--   1 chengpan  staff    20M Feb 26 12:56 kyuubi-1.1.0-SNAPSHOT-bin-without-spark.tar.gz
```

Closes #378 from pan3793/KYUUBI-377.

442569e [Cheng Pan] add profile spark-provided for skip download spark binary
0d8da24 [Cheng Pan] update doc
5b7c41f [Cheng Pan] [KYUUBI #377] build/dist support --spark-provided

Authored-by: Cheng Pan <379377944@qq.com>
Signed-off-by: ulysses-you <ulyssesyou18@gmail.com>
2021-02-26 15:00:42 +08:00
Kent Yao
eadca7589c
[KYUUBI #296] Add spark 2.4 profile and shade hive-sevice-rpc to engine jar to fix thrift build err (#296)
* add spark 2.4.7 profile

* nit

* nit

* scala versions for profile spark-2.4
2021-01-18 14:23:31 +08:00
Cheng Pan
90c120759e
Improve pom.xml
fixes #268
Squashed commit of the following:

commit d40d0831d578af23559ea2a2544ceebc47f6df50
Author: Cheng Pan <379377944@qq.com>
Date:   Fri Jan 8 18:30:51 2021 +0800

    remove unused mockito

commit 386e7e4eb1c93006e302c41ffbc04d0acfaf9a9b
Author: Cheng Pan <379377944@qq.com>
Date:   Fri Jan 8 17:35:44 2021 +0800

    extract versions to properties
2021-01-08 19:04:48 +08:00
Kent Yao
ca1b452e9a
Prepare Kyuubi v1.1.0 2020-12-31 15:51:12 +08:00
Kent Yao
7157370ef6 Add Building Doc 2020-11-05 19:05:26 +08:00
Kent Yao
83d1ab7450 bump up spark to 3.0.1 2020-09-10 10:58:52 +08:00
Kent Yao
e8ab5ab4a0 Some build staff 2020-08-21 14:39:33 +08:00
Kent Yao
0468fd4d4e Implementaton of KyuubiOperationManager 2020-08-21 10:51:55 +08:00
Kent Yao
86e1a7aac3 SparkProcessBuilder 2020-08-18 17:22:08 +08:00