apache/kyuubi
1
0
Fork 0
Code Issues Pull Requests Actions 5 Packages Projects Releases Wiki Activity
ceaa2bf3cb
kyuubi/extensions
History
odone 4277710174 [KYUUBI #4393] [Kyuubi #4332] Fix some bugs with Groupby and CacheTable 
close #4332
### _Why are the changes needed?_

For the case where the table name has been resolved and an `Expand` logical plan exists
```
InsertIntoHiveTable `default`.`t1`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, false, false, [a, b]
+- Aggregate [a#0], [a#0, ansi_cast((count(if ((gid#9 = 1)) spark_catalog.default.t2.`b`#10 else null) * count(if ((gid#9 = 2)) spark_catalog.default.t2.`c`#11 else null)) as string) AS b#8]
   +- Aggregate [a#0, spark_catalog.default.t2.`b`#10, spark_catalog.default.t2.`c`#11, gid#9], [a#0, spark_catalog.default.t2.`b`#10, spark_catalog.default.t2.`c`#11, gid#9]
      +- Expand [ArrayBuffer(a#0, b#1, null, 1), ArrayBuffer(a#0, null, c#2, 2)], [a#0, spark_catalog.default.t2.`b`#10, spark_catalog.default.t2.`c`#11, gid#9]
         +- HiveTableRelation [`default`.`t2`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: [a#0, b#1, c#2], Partition Cols: []]
```
For the case `CacheTable` with `window` function
```
InsertIntoHiveTable `default`.`t1`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, true, false, [a, b]
+- Project [a#98, b#99]
   +- InMemoryRelation [a#98, b#99, rank#100], StorageLevel(disk, memory, deserialized, 1 replicas)
         +- *(2) Filter (isnotnull(rank#4) AND (rank#4 = 1))
            +- Window [row_number() windowspecdefinition(a#9, b#10 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS rank#4], [a#9], [b#10 ASC NULLS FIRST]
               +- *(1) Sort [a#9 ASC NULLS FIRST, b#10 ASC NULLS FIRST], false, 0
                  +- Exchange hashpartitioning(a#9, 200), ENSURE_REQUIREMENTS, [id=#38]
                     +- Scan hive default.t2 [a#9, b#10], HiveTableRelation [`default`.`t2`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: [a#9, b#10], Partition Cols: []]

```

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4393 from iodone/kyuubi-4332.

Closes #4393

d2afdabd [odone] fix cache table bug
443af798 [odone] fix some bugs with groupby

Authored-by: odone <odone.zhang@gmail.com>
Signed-off-by: ulyssesyou <ulyssesyou@apache.org>
2023-02-24 15:37:08 +08:00
..
server/kyuubi-server-plugin Bump 1.8.0-SNAPSHOT 2023-02-10 15:25:49 +08:00
spark [KYUUBI #4393] [Kyuubi #4332] Fix some bugs with Groupby and CacheTable 2023-02-24 15:37:08 +08:00
README.md [KYUUBI #4312] [DOCS] Include **/README.md in markdown style check 2023-02-14 02:23:32 +08:00

README.md

For developers

This folder contains plugins/extension for kyuubi server and different engine types.

  • ext
    • kyuubi-server
    • spark
    • flink
    • trino
    • hive
    • others
    • ...