apache/kyuubi
1
0
Fork 0
Code Issues Pull Requests Actions 5 Packages Projects Releases Wiki Activity

[KYUUBI #6424] TPC-H/DS connector support Spark 4.0

Browse Source 
# 🔍 Description

Adapt changes in SPARK-45857

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

```
build/mvn -pl ':kyuubi-spark-connector-tpch_2.13,:kyuubi-spark-connector-tpcds_2.13' \
    -Pscala-2.13 -Pspark-master -am clean install -DskipTests
build/mvn -pl ':kyuubi-spark-connector-tpch_2.13,:kyuubi-spark-connector-tpcds_2.13' \
    -Pscala-2.13 -Pspark-master test
```

```
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Kyuubi Spark TPC-DS Connector 1.10.0-SNAPSHOT:
[INFO]
[INFO] Kyuubi Spark TPC-DS Connector ...................... SUCCESS [ 53.699 s]
[INFO] Kyuubi Spark TPC-H Connector ....................... SUCCESS [ 30.511 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  01:24 min
[INFO] Finished at: 2024-05-27T06:01:58Z
[INFO] ------------------------------------------------------------------------
```

---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6424 from pan3793/tpc-conn-4.

Closes #6424

9012a177f [Cheng Pan] TPC-H/DS connector support Spark 4.0

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
This commit is contained in:
Cheng Pan 2024-05-27 07:02:52 +00:00
parent f003efd61f
commit 82441671a5
2 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
extensions/spark/kyuubi-spark-connector-tpcds/src/main/scala/org/apache/kyuubi/spark/connector/tpcds/TPCDSCatalog.scala
View File

@ -58,7 +58,7 @@ class TPCDSCatalog extends TableCatalog with SupportsNamespaces with Logging {
override def listTables(namespace: Array[String]): Array[Identifier] = namespace match {
case Array(db) if databases contains db => tables.map(Identifier.of(namespace, _))
case _ => throw new NoSuchNamespaceException(namespace.mkString("."))
case _ => throw new NoSuchNamespaceException(namespace)
}
override def loadTable(ident: Identifier): SparkTable = (ident.namespace, ident.name) match {

2
extensions/spark/kyuubi-spark-connector-tpch/src/main/scala/org/apache/kyuubi/spark/connector/tpch/TPCHCatalog.scala
View File

@ -58,7 +58,7 @@ class TPCHCatalog extends TableCatalog with SupportsNamespaces with Logging {
override def listTables(namespace: Array[String]): Array[Identifier] = namespace match {
case Array(db) if databases contains db => tables.map(Identifier.of(namespace, _))
case _ => throw new NoSuchNamespaceException(namespace.mkString("."))
case _ => throw new NoSuchNamespaceException(namespace)
}
override def loadTable(ident: Identifier): SparkTable = (ident.namespace, ident.name) match {