d659e06d45
### What changes were proposed in this pull request? Add logic to support avoid sorting shuffle files for Reduce mode when optimize skew partitions ### Why are the changes needed? Current logic need sorting shuffle files when read Reduce mode skew partition shuffle files, we found some shuffle sorting timeout and performance issue ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Cluster test and uts Closes #2373 from wangshengjie123/optimize-skew-partition. Lead-authored-by: wangshengjie <wangshengjie3@xiaomi.com> Co-authored-by: wangshengjie3 <wangshengjie3@xiaomi.com> Co-authored-by: Fu Chen <cfmcgrady@gmail.com> Co-authored-by: Shuang <lvshuang.tb@gmail.com> Co-authored-by: wangshengjie3 <soldier.sj.wang@gmail.com> Co-authored-by: Fei Wang <fwang12@ebay.com> Co-authored-by: Wang, Fei <fwang12@ebay.com> Signed-off-by: Shuang <lvshuang.xjs@alibaba-inc.com> |
||
|---|---|---|
| .. | ||
| client.md | ||
| columnar-shuffle.md | ||
| ha.md | ||
| index.md | ||
| master.md | ||
| metrics.md | ||
| network-module.md | ||
| network.md | ||
| quota.md | ||
| worker.md | ||