[KYUUBI #6305][FOLLOWUP] Improve package Spark SQL engine both Scala 2.12 and 2.13

# 🔍 Description
## Issue References 🔗

This pull request fixes #6305

## Describe Your Solution 🔧

### Solution 1 use `<profile>` - Inappropriate
I found a way to use <profiles>, roughly as follows:
```xml
<profile>
    <id>scala-2.12</id>
    <properties>
        <scala.binary.version>2.12</scala.binary.version>
    </properties>
</profile>
```
After specifying, I attempted to use
1. `build/mvn install -DskipTests -Dmaven.javadoc.skip=true -Dmaven.scaladoc.skip=true - Dmaven.source.skip -Pscala-2.12 -Pscala-2.13 -pl: kyuubi-spark-sql-engine -am`
2. `build/mvn install -DskipTests -Dmaven.javadoc.skip=true -Dmaven.scaladoc.skip=true - Dmaven.source.skip -Pscala-2.13 -Pscala-2.12 -pl: kyuubi-spark-sql-engine -am`

#### Problem
But in the end, it was found that if both '-Pscala-2.12' and '-Pscala-2.13' are used at the same time, '-Pscala2.13' will be selected by default, which may not be a good solution to this problem.

### Solution2
Later, I thought about whether it was possible to filter the parameter '$' internally. It is effective.

## Types of changes 🔖

- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪
1. scala2.12
![scala_2 12版本spark-sql-engine](https://github.com/apache/kyuubi/assets/96274454/a9f82433-8f73-4bb0-b0f7-7d2725435a0e)
![scala_2 12_select 测试](https://github.com/apache/kyuubi/assets/96274454/fe1cb601-8d19-4dc0-a248-c045176b87a6)
2. scala2.13
![image](https://github.com/apache/kyuubi/assets/96274454/d79ae1cc-64fe-4079-b706-37a21b00b627)
![scala_2 13版本测试](https://github.com/apache/kyuubi/assets/96274454/a4084499-12ea-4150-982a-67aacd16c184)
---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6330 from PorterZhang2021/followup-6297.

Closes #6305

5524b13f5 [Cheng Pan] nit
cccb94d98 [PorterZhang2021] [# 6297] imporve Package Spark SQL engine both Scala 2.12 and 2.13 Follow Up
fb03d60c2 [Porter Zhang] Merge branch 'apache:master' into followup-6297
393435ea9 [PorterZhang2021] [# 6297] imporve Package Spark SQL engine both Scala 2.12 and 2.13 Follow Up
0b49f60e6 [Porter Zhang] Merge branch 'apache:master' into followup-6297
f7c7a65c8 [PorterZhang2021] [# 6297] Package Spark SQL engine both Scala 2.12 and 2.13 Follow Up
3d2926afc [Porter Zhang] Merge branch 'apache:master' into followup-6297
eb6406148 [PorterZhang2021] [followup-issue6297] improve issue6297
956ac4955 [PorterZhang2021] [# 6297] Package Spark SQL engine both Scala 2.12 and 2.13 Follow Up

Lead-authored-by: PorterZhang2021 <PorterZhang2021@outlook.com>
Co-authored-by: Porter Zhang <96274454+PorterZhang2021@users.noreply.github.com>
Co-authored-by: Cheng Pan <chengpan@apache.org>
Co-authored-by: PorterZhang2021 <porterzhang2021@outlook.com>
Co-authored-by: Porter Zhang <porterzhang2021@outlook.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
This commit is contained in:
PorterZhang2021 2024-05-15 12:43:09 +08:00 committed by Cheng Pan
parent c8e645734b
commit bc394a933b
No known key found for this signature in database
GPG Key ID: 8001952629BCC75D
2 changed files with 21 additions and 3 deletions

View File

@ -247,18 +247,28 @@ BUILD_COMMAND=("$MVN" clean install $MVN_DIST_OPT $@)
echo -e "\nBuilding with..."
# shellcheck disable=SC2145
echo -e "\$ ${BUILD_COMMAND[@]}\n"
"${BUILD_COMMAND[@]}"
FILTERED_ARGS=()
# shellcheck disable=SC2045
for arg in "$@"; do
if [[ $arg == *scala-2.12* ]]; then
FILTERED_ARGS+=("${arg//scala-2.12/scala-2.13}")
elif [[ $arg == *scala-2.13* ]]; then
FILTERED_ARGS+=("${arg//scala-2.13/scala-2.12}")
fi
done
# shellcheck disable=SC2050
if [ "$SCALA_VERSION" = "2.12" ]; then
EXTRA_SPARK_ENGINE_BUILD_COMMAND=("$MVN" install $MVN_DIST_OPT $@ -Pscala-2.13 -pl :kyuubi-spark-sql-engine_2.13 -am)
EXTRA_SPARK_ENGINE_BUILD_COMMAND=("$MVN" install $MVN_DIST_OPT ${FILTERED_ARGS[@]} -Pscala-2.13 -pl :kyuubi-spark-sql-engine_2.13 -am)
else
EXTRA_SPARK_ENGINE_BUILD_COMMAND=("$MVN" install $MVN_DIST_OPT $@ -Pscala-2.12 -pl :kyuubi-spark-sql-engine_2.12 -am)
EXTRA_SPARK_ENGINE_BUILD_COMMAND=("$MVN" install $MVN_DIST_OPT ${FILTERED_ARGS[@]} -pl :kyuubi-spark-sql-engine_2.12 -am)
fi
# shellcheck disable=SC2145
echo -e "\$ ${EXTRA_SPARK_ENGINE_BUILD_COMMAND[@]}\n"
"${BUILD_COMMAND[@]}"
"${EXTRA_SPARK_ENGINE_BUILD_COMMAND[@]}"
# Make directories
@ -307,6 +317,7 @@ done
cp "$KYUUBI_HOME/externals/kyuubi-flink-sql-engine/target/kyuubi-flink-sql-engine_${SCALA_VERSION}-${VERSION}.jar" "$DISTDIR/externals/engines/flink/"
# Copy spark engines
# shellcheck disable=SC2045
for scala_version in 2.12 2.13; do
cp "$KYUUBI_HOME/externals/kyuubi-spark-sql-engine/target/kyuubi-spark-sql-engine_${scala_version}-${VERSION}.jar" "$DISTDIR/externals/engines/spark/"
done

View File

@ -1950,6 +1950,13 @@
</properties>
</profile>
<profile>
<id>scala-2.12</id>
<properties>
<scala.binary.version>2.12</scala.binary.version>
</properties>
</profile>
<!-- For development only, not generally applicable for all modules -->
<profile>
<id>scala-2.13</id>