238c893f39
# 🔍 Description
## Issue References 🔗
Now, MaxScanStrategy can be adopted to limit max scan file size in some datasources, such as Hive. Hopefully we can enhance MaxScanStrategy to include support for the datasourcev2.
## Describe Your Solution 🔧
get the statistics about files scanned through datasourcev2 API
## Types of changes 🔖
- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request ⚰️
#### Behavior With This Pull Request 🎉
#### Related Unit Tests
---
# Checklists
## 📝 Author Self Checklist
- [x] My code follows the [style guidelines](https://kyuubi.readthedocs.io/en/master/contributing/code/style.html) of this project
- [x] I have performed a self-review
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my feature works
- [x] New and existing unit tests pass locally with my changes
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
## 📝 Committer Pre-Merge Checklist
- [x] Pull request title is okay.
- [x] No license issues.
- [x] Milestone correctly set?
- [x] Test coverage is ok
- [x] Assignees are selected.
- [ ] Minimum number of approvals
- [ ] No changes are requested
**Be nice. Be informative.**
Closes #5852 from zhaohehuhu/dev-1213.
Closes #6315
3c5b0c276 [hezhao2] reformat
fb113d625 [hezhao2] disable the rule that checks the maxPartitions for dsv2
acc358732 [hezhao2] disable the rule that checks the maxPartitions for dsv2
c8399a021 [hezhao2] fix header
70c845bee [hezhao2] add UTs
3a0739686 [hezhao2] add ut
4d26ce131 [hezhao2] reformat
f87cb072c [hezhao2] reformat
b307022b8 [hezhao2] move code to Spark 3.5
73258c2ae [hezhao2] fix unused import
cf893a0e1 [hezhao2] drop reflection for loading iceberg class
dc128bc8e [hezhao2] refactor code
661834cce [hezhao2] revert code
6061f42ab [hezhao2] delete IcebergSparkPlanHelper
5f1c3c082 [hezhao2] fix
b15652f05 [hezhao2] remove iceberg dependency
fe620ca92 [hezhao2] enable MaxScanStrategy when accessing iceberg datasource
Authored-by: hezhao2 <hezhao2@cisco.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
(cherry picked from commit
|
||
|---|---|---|
| .. | ||
| server/kyuubi-server-plugin | ||
| spark | ||
| README.md | ||
For developers
This folder contains plugins/extension for kyuubi server and different engine types.
- ext
- kyuubi-server
- spark
- flink
- trino
- hive
- others
- ...