celeborn/worker
Aravind Patnam ca64bb5143 [CELEBORN-1313] Custom Network Location Aware Replication
### What changes were proposed in this pull request?

Enable custom network location aware replication, based on a custom impl of `DNSToSwitchMapping`.

### Why are the changes needed?

Resolution of network location of multiple workers at master can be expensive at times. This way, each worker resolves its own network location and sends to master via the RegisterWorker transport message. If worker cannot resolve, fallback to attempting to resolve at master (during update meta or reload of snapshot). Proposal: [Celeborn Custom Network Location Aware Replication](https://docs.google.com/document/d/11M_MKKnIXCTExJHMX-OMTq7SBpkl8fJMlpy8hLgmev0/edit#heading=h.s3vnydz589z5)

### Does this PR introduce _any_ user-facing change?

No

### How was this patch tested?

Updated the unit tests.

Closes #2367 from akpatnam25/CELEBORN-1313.

Authored-by: Aravind Patnam <apatnam@linkedin.com>
Signed-off-by: zky.zhoukeyong <zky.zhoukeyong@alibaba-inc.com>
2024-03-13 11:10:30 +08:00
..
src [CELEBORN-1313] Custom Network Location Aware Replication 2024-03-13 11:10:30 +08:00
pom.xml [CELEBORN-977] Support RocksDB as recover DB backend 2023-09-19 09:20:33 +08:00