9e01aac501
### What changes were proposed in this pull request? As title ### Why are the changes needed? See https://issues.apache.org/jira/browse/SPARK-42689 ### Does this PR introduce _any_ user-facing change? Yes. User need to set `spark.shuffle.sort.io.plugin.class` to `org.apache.spark.shuffle.celeborn.CelebornShuffleDataIO` to enable this feature. ### How was this patch tested? Add a new matrix dimension, shuffle-plugin-class, in github ci, to run spark tests over `LocalDiskShuffleDataIO` and `CelebornShuffleDataIO` respectively. Closes #1884 from zhouyifan279/spark-driver-component. Authored-by: zhouyifan279 <zhouyifan279@gmail.com> Signed-off-by: Cheng Pan <chengpan@apache.org> |
||
|---|---|---|
| .. | ||
| benchmark_manual.yml | ||
| benchmark.yml | ||
| check_stale_prs.yml | ||
| deps.yml | ||
| integration.yml | ||
| license.yml | ||
| maven.yml | ||
| regression_manual.yml | ||
| regression.yml | ||
| sbt.yml | ||
| style.yml | ||