### What changes were proposed in this pull request? Enable custom network location aware replication, based on a custom impl of `DNSToSwitchMapping`. ### Why are the changes needed? Resolution of network location of multiple workers at master can be expensive at times. This way, each worker resolves its own network location and sends to master via the RegisterWorker transport message. If worker cannot resolve, fallback to attempting to resolve at master (during update meta or reload of snapshot). Proposal: [Celeborn Custom Network Location Aware Replication](https://docs.google.com/document/d/11M_MKKnIXCTExJHMX-OMTq7SBpkl8fJMlpy8hLgmev0/edit#heading=h.s3vnydz589z5) ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Updated the unit tests. Closes #2367 from akpatnam25/CELEBORN-1313. Authored-by: Aravind Patnam <apatnam@linkedin.com> Signed-off-by: zky.zhoukeyong <zky.zhoukeyong@alibaba-inc.com> |
||
|---|---|---|
| .. | ||
| src | ||
| pom.xml | ||