9e01aac501
### What changes were proposed in this pull request? As title ### Why are the changes needed? See https://issues.apache.org/jira/browse/SPARK-42689 ### Does this PR introduce _any_ user-facing change? Yes. User need to set `spark.shuffle.sort.io.plugin.class` to `org.apache.spark.shuffle.celeborn.CelebornShuffleDataIO` to enable this feature. ### How was this patch tested? Add a new matrix dimension, shuffle-plugin-class, in github ci, to run spark tests over `LocalDiskShuffleDataIO` and `CelebornShuffleDataIO` respectively. Closes #1884 from zhouyifan279/spark-driver-component. Authored-by: zhouyifan279 <zhouyifan279@gmail.com> Signed-off-by: Cheng Pan <chengpan@apache.org> |
||
|---|---|---|
| .. | ||
| ISSUE_TEMPLATE | ||
| workflows | ||
| PULL_REQUEST_TEMPLATE.md | ||