apache/kyuubi
1
0
Fork 0
Code Issues Pull Requests Actions 5 Packages Projects Releases Wiki Activity
83b2f8326b
kyuubi/extensions
History
Fu Chen 83b2f8326b [KYUUBI #5257] [LINEAGE] Adapt CreateTableAsSelect plan to Spark 3.5 changes 
### _Why are the changes needed?_

to fix
```
- columns lineage extract - CreateTableAsSelect *** FAILED ***
  java.util.NoSuchElementException: None.get
  at scala.None$.get(Option.scala:529)
  at scala.None$.get(Option.scala:527)
  at org.apache.kyuubi.plugin.lineage.helper.SparkSQLLineageParserHelperSuite.extractLineage(SparkSQLLineageParserHelperSuite.scala:1452)
  at org.apache.kyuubi.plugin.lineage.helper.SparkSQLLineageParserHelperSuite.$anonfun$new$24(SparkSQLLineageParserHelperSuite.scala:354)
  at org.apache.kyuubi.plugin.lineage.helper.SparkSQLLineageParserHelperSuite.$anonfun$new$24$adapted(SparkSQLLineageParserHelperSuite.scala:353)
  at org.apache.spark.sql.SparkListenerExtensionTest.withTable(SparkListenerExtensionTest.scala:48)
  at org.apache.spark.sql.SparkListenerExtensionTest.withTable$(SparkListenerExtensionTest.scala:46)
  at org.apache.kyuubi.plugin.lineage.helper.SparkSQLLineageParserHelperSuite.withTable(SparkSQLLineageParserHelperSuite.scala:34)
  at org.apache.kyuubi.plugin.lineage.helper.SparkSQLLineageParserHelperSuite.$anonfun$new$23(SparkSQLLineageParserHelperSuite.scala:353)
  at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
  ...
```

the method `CreateTableAsSelect#left` was removed since Spark 3.5, for more details, see https://github.com/apache/spark/pull/40734

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request

### _Was this patch authored or co-authored using generative AI tooling?_

No

Closes #5257 from cfmcgrady/fix-SparkSQLLineageParserHelperSuite.

Closes #5257

aa5e730fc [Fu Chen] fix

Authored-by: Fu Chen <cfmcgrady@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2023-09-06 07:26:30 +00:00
..
server/kyuubi-server-plugin Bump 1.9.0-SNAPSHOT 2023-09-04 14:23:12 +08:00
spark [KYUUBI #5257] [LINEAGE] Adapt CreateTableAsSelect plan to Spark 3.5 changes 2023-09-06 07:26:30 +00:00
README.md [KYUUBI #4312] [DOCS] Include **/README.md in markdown style check 2023-02-14 02:23:32 +08:00

README.md

For developers

This folder contains plugins/extension for kyuubi server and different engine types.

  • ext
    • kyuubi-server
    • spark
    • flink
    • trino
    • hive
    • others
    • ...