### What changes were proposed in this pull request? Ignore the getReducerFileGroup timeout before shuffle stage end. ### Why are the changes needed? 1. if the getReducerFileGroup timeout is caused by lifecycle manager commitFiles timeout(stage not ended) 2. maybe many tasks failed and would not report fetch failure 3. then it cause the spark application failed eventually. The shuffle client should ignore the getReducerFileGroup timeout before LifeCycleManager commitFiles complete. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? UT Closes #3263 from turboFei/is_stage_end. Lead-authored-by: Wang, Fei <fwang12@ebay.com> Co-authored-by: Fei Wang <cn.feiwang@gmail.com> Signed-off-by: mingji <fengmingxiao.fmx@alibaba-inc.com> |
||
|---|---|---|
| .. | ||
| flink-it | ||
| kubernetes-it | ||
| mr-it | ||
| spark-it | ||
| tez-it | ||