c6c319d865
### What changes were proposed in this pull request? add a new parameter to cap the max memory can be used for sort writer buffer ### Why are the changes needed? with a huge number of partitions, the threshold based on buffer size * number of partitions without this cap can be too large, e.g. 64K * 100000 = 6G ### Does this PR introduce _any_ user-facing change? a new parameter ### How was this patch tested? ut Closes #2388 from CodingCat/adaptive_followup. Lead-authored-by: CodingCat <zhunansjtu@gmail.com> Co-authored-by: zky.zhoukeyong <zky.zhoukeyong@alibaba-inc.com> Co-authored-by: Keyong Zhou <zhouky@apache.org> Co-authored-by: Keyong Zhou <zky.zhoukeyong@alibaba-inc.com> Signed-off-by: zky.zhoukeyong <zky.zhoukeyong@alibaba-inc.com> |
||
|---|---|---|
| .. | ||
| src | ||
| pom.xml | ||