### What changes were proposed in this pull request? `CelebornShuffleFallbackPolicyRunner` could not only check quota, but also check whether cluster has available workers. If there is no available workers, fallback to external shuffle. ### Why are the changes needed? `CelebornShuffleFallbackPolicyRunner` adds a check for available workers. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? - `SparkShuffleManagerSuite#testClusterNotAvailableWithAvailableWorkers` Closes #1814 from SteNicholas/CELEBORN-830. Authored-by: SteNicholas <programgeek@163.com> Signed-off-by: zky.zhoukeyong <zky.zhoukeyong@alibaba-inc.com> |
||
|---|---|---|
| .. | ||
| src | ||
| pom.xml | ||