apache/kyuubi
1
0
Fork 0
Code Issues Pull Requests Actions 5 Packages Projects Releases Wiki Activity
586f6008bd
kyuubi/extensions
History
Cheng Pan 522a28e1d5
[KYUUBI #6398] Fix lineage plugin UT for Spark 4.0 
# 🔍 Description

```
build/mvn clean test -Pscala-2.13 -Pspark-master -pl :kyuubi-spark-lineage_2.13
```

```
- test group by *** FAILED ***
  org.apache.spark.sql.catalyst.ExtendedAnalysisException: [DATATYPE_MISMATCH.BINARY_OP_WRONG_TYPE] Cannot resolve "(b + c)" due to data type mismatch: the binary operator requires the input type ("NUMERIC" or "INTERVAL DAY TO SECOND" or "INTERVAL YEAR TO MONTH" or "INTERVAL"), not "STRING". SQLSTATE: 42K09; line 1 pos 59;
'InsertIntoStatement RelationV2[a#546, b#547, c#548] v2_catalog.db.t1 v2_catalog.db.t1, false, false, false
+- 'Aggregate [a#543], [a#543, unresolvedalias('count(distinct (b#544 + c#545))), (count(distinct b#544) * count(distinct c#545)) AS (count(DISTINCT b) * count(DISTINCT c))#551L]
   +- SubqueryAlias v2_catalog.db.t2
      +- RelationV2[a#543, b#544, c#545] v2_catalog.db.t2 v2_catalog.db.t2
  at org.apache.spark.sql.catalyst.analysis.package$AnalysisErrorAt.dataTypeMismatch(package.scala:73)
  at org.apache.spark.sql.catalyst.analysis.CheckAnalysis.$anonfun$checkAnalysis0$7(CheckAnalysis.scala:315)
  at org.apache.spark.sql.catalyst.analysis.CheckAnalysis.$anonfun$checkAnalysis0$7$adapted(CheckAnalysis.scala:302)
  at org.apache.spark.sql.catalyst.trees.TreeNode.foreachUp(TreeNode.scala:244)
  at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$foreachUp$1(TreeNode.scala:243)
  at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$foreachUp$1$adapted(TreeNode.scala:243)
  at scala.collection.immutable.Vector.foreach(Vector.scala:1856)
  at org.apache.spark.sql.catalyst.trees.TreeNode.foreachUp(TreeNode.scala:243)
  at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$foreachUp$1(TreeNode.scala:243)
  at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$foreachUp$1$adapted(TreeNode.scala:243)
  ...
```

## Types of changes 🔖

- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

Pass UT.

---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6398 from pan3793/lineage-fix.

Closes #6398

afce6b880 [Cheng Pan] Fix lineage plugin UT for Spark 4.0

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2024-05-20 22:03:48 +08:00
..
server/kyuubi-server-plugin [RELEASE] Bump 1.10.0-SNAPSHOT 2024-03-13 14:24:49 +08:00
spark [KYUUBI #6398] Fix lineage plugin UT for Spark 4.0 2024-05-20 22:03:48 +08:00
README.md [KYUUBI #4312] [DOCS] Include **/README.md in markdown style check 2023-02-14 02:23:32 +08:00

README.md

For developers

This folder contains plugins/extension for kyuubi server and different engine types.

  • ext
    • kyuubi-server
    • spark
    • flink
    • trino
    • hive
    • others
    • ...