kyuubi/integration-tests/kyuubi-trino-it
Bowen Liang e33df9ce66 [KYUUBI #5196] Enable CI Test on Scala 2.13 and support custom or spark-core extracted Scala version for Spark's engine
### _Why are the changes needed?_

- enable CI test on Scala-2.13 for all modules except Flink SQL engine
- For testing, choose available Spark engine home in `download` module by `SCALA_COMPILE_VERSION` of Kyuubi server
- Choose the Scala version of Spark engine main resource Jar  in the following order:
  1. `SPARK_SCALA_VERSION` system env
  2. Extract Scala version from Spark home's `spark-core` jar filename
- Fixed 1 assertion error of kyuubi-spark-lineage module, as Spark on Scala 2.12 and 2.13 show different order of column linage output in `MergeIntoTable` ut
```
SparkSQLLineageParserHelperSuite:
- columns lineage extract - MergeIntoTable *** FAILED ***
  inputTables(List(v2_catalog.db.source_t))
  outputTables(List(v2_catalog.db.target_t))
  columnLineage(List(ColumnLineage(v2_catalog.db.target_t.name,Set(v2_catalog.db.source_t.name)), ColumnLineage(v2_catalog.db.target_t.price,Set(v2_catalog.db.source_t.price)), ColumnLineage(v2_catalog.db.target_t.id,Set(v2_catalog.db.source_t.id)))) did not equal inputTables(List(v2_catalog.db.source_t))
  outputTables(List(v2_catalog.db.target_t))
  columnLineage(List(ColumnLineage(v2_catalog.db.target_t.id,Set(v2_catalog.db.source_t.id)), ColumnLineage(v2_catalog.db.target_t.name,Set(v2_catalog.db.source_t.name)), ColumnLineage(v2_catalog.db.target_t.price,Set(v2_catalog.db.source_t.price)))) (SparkSQLLineageParserHelperSuite.scala:182)
```
- Fixed other tests relying on Scala scripting results

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request

### _Was this patch authored or co-authored using generative AI tooling?_

Closes #5196 from bowenliang123/scala213-test.

Closes #5196

97fafacd3 [liangbowen] prevent repeated compilation for regrex pattern
76b99d423 [Bowen Liang] test on scala-2.13

Lead-authored-by: Bowen Liang <liangbowen@gf.com.cn>
Co-authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
2023-10-10 08:42:40 +08:00
..
src/test [KYUUBI #4959] [MINOR] Code improvements for Scala 2023-06-16 21:20:17 +08:00
pom.xml [KYUUBI #5196] Enable CI Test on Scala 2.13 and support custom or spark-core extracted Scala version for Spark's engine 2023-10-10 08:42:40 +08:00