apache/kyuubi
1
0
Fork 0
Code Issues Pull Requests Actions 5 Packages Projects Releases Wiki Activity

[KYUUBI #2974][FEATURE] EOL Support for Spark 3.0

Browse Source 
### _Why are the changes needed?_
This PR is to drop support for spark 3.0 from compiling/building and test.  [#2974](https://github.com/apache/incubator-kyuubi/issues/2974)
Following the discussion on dev list https://lists.apache.org/thread/3mw8zz9r0tzg2dpj6717xy59610bhkp4

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [X] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3035 from huage1994/2974.

Closes #2974

add45cfe [guanhua.lgh] [FEATURE] EOL Support for Spark 3.0

Authored-by: guanhua.lgh <guanhua.lgh@alibaba-inc.com>
Signed-off-by: Kent Yao <yao@apache.org>
This commit is contained in:
guanhua.lgh 2022-07-12 11:01:50 +08:00 committed by Kent Yao
parent 1bc3916d99
commit c1158accca
No known key found for this signature in database
GPG Key ID: F7051850A0AF904D
5 changed files with 4 additions and 24 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
.github/workflows/master.yml vendored
View File

@ -45,7 +45,6 @@ jobs:
- 8
- 11
spark:
- '3.0'
- '3.1'
- '3.2'
- '3.3'
@ -53,11 +52,6 @@ jobs:
exclude-tags: [""]
comment: ["normal"]
include:
- java: 8
spark: '3.2'
spark-archive: '-Dspark.archive.mirror=https://archive.apache.org/dist/spark/spark-3.0.3 -Dspark.archive.name=spark-3.0.3-bin-hadoop2.7.tgz'
exclude-tags: '-Dmaven.plugin.scalatest.exclude.tags=org.scalatest.tags.Slow,org.apache.kyuubi.tags.DeltaTest,org.apache.kyuubi.tags.HudiTest,org.apache.kyuubi.tags.IcebergTest'
comment: 'verify-spark-3.0'
- java: 8
spark: '3.2'
spark-archive: '-Dspark.archive.mirror=https://archive.apache.org/dist/spark/spark-3.1.3 -Dspark.archive.name=spark-3.1.3-bin-hadoop3.2.tgz'

4
.github/workflows/style.yml vendored
View File

@ -46,7 +46,7 @@ jobs:
cache: 'maven'
check-latest: false
- name: Scalastyle with maven
run: build/mvn scalastyle:check -Pflink-provided,hive-provided,spark-provided,spark-block-cleaner,spark-3.3,spark-3.2,spark-3.1,spark-3.0,tpcds
run: build/mvn scalastyle:check -Pflink-provided,hive-provided,spark-provided,spark-block-cleaner,spark-3.3,spark-3.2,spark-3.1,tpcds
- name: Print scalastyle error report
if: failure()
run: >-
@ -57,4 +57,4 @@ jobs:
fi
done
- name: JavaStyle with maven
run: build/mvn spotless:check -Pflink-provided,hive-provided,spark-provided,spark-block-cleaner,spark-3.3,spark-3.2,spark-3.1,spark-3.0,tpcds
run: build/mvn spotless:check -Pflink-provided,hive-provided,spark-provided,spark-block-cleaner,spark-3.3,spark-3.2,spark-3.1,tpcds

2
dev/reformat
View File

@ -20,6 +20,6 @@ set -x
KYUUBI_HOME="$(cd "`dirname "$0"`/.."; pwd)"
PROFILES="-Pflink-provided,hive-provided,spark-provided,spark-block-cleaner,spark-3.3,spark-3.2,spark-3.1,spark-3.0,tpcds"
PROFILES="-Pflink-provided,hive-provided,spark-provided,spark-block-cleaner,spark-3.3,spark-3.2,spark-3.1,tpcds"
${KYUUBI_HOME}/build/mvn spotless:apply $PROFILES

2
docs/develop_tools/building.md
View File

@ -64,8 +64,8 @@ Since v1.1.0, Kyuubi support building with different Spark profiles,
Profile | Default | Since
--- | --- | ---
-Pspark-3.0 | Yes | 1.0.0
-Pspark-3.1 | No | 1.1.0
-Pspark-3.2 | Yes | 1.4.0
## Building with Apache dlcdn site

14
pom.xml
View File

@ -1985,20 +1985,6 @@
</properties>
</profile>
<profile>
<id>spark-3.0</id>
<properties>
<spark.version>3.0.3</spark.version>
<delta.version>0.8.0</delta.version>
<iceberg.name>iceberg-spark3-runtime</iceberg.name>
<!--
https://hudi.apache.org/releases/release-0.11.0/#bundle-usage-updates
Hudi 0.11.0 Spark bundle for 3.0.x is no longer officially supported.
-->
<maven.plugin.scalatest.exclude.tags>org.scalatest.tags.Slow,org.apache.kyuubi.tags.HudiTest</maven.plugin.scalatest.exclude.tags>
</properties>
</profile>
<profile>
<id>spark-3.1</id>
<properties>