03498ce46b
### What changes were proposed in this pull request? Add a configuration "celeborn.worker.storage.expireDirs.timeout" with a default value of 6h in rsswork. This configuration is used to set the expiration time for app local directories. https://issues.apache.org/jira/browse/CELEBORN-1046 ### Why are the changes needed? When Celeborn periodically deletes the directories of apps, it determines whether the app needs to be deleted based on the shuffleKeySet in memory. However, this method may not accurately indicate the completion of the app and could potentially lead to the unintentional deletion of shuffle data. ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? Closes #1998 from wilsonjie/CELEBORN-1046. Authored-by: sunjunjie <sunjunjie@zto.com> Signed-off-by: zky.zhoukeyong <zky.zhoukeyong@alibaba-inc.com> |
||
|---|---|---|
| .. | ||
| client.md | ||
| columnar-shuffle.md | ||
| ha.md | ||
| index.md | ||
| master.md | ||
| metrics.md | ||
| network.md | ||
| quota.md | ||
| worker.md | ||