celeborn/client-flink
Angerszhuuuu 5471a6afe5
[CELEBORN-804] ShuffleClient should cleanup shuffle infos when trigger unregisterShuffle
### What changes were proposed in this pull request?

After discussion, we make sure that `shuffleManager.unregisterShuffle()` will be triggered by Spark both in driver and executor. In this pr:

  1. Add shuffle client both in driver and executor side in ShuffleManager
  2. ShuffleClient call cleanupShuffle() when trigger `unregisterShuffle`.

This replaced https://github.com/apache/incubator-celeborn/pull/1719

### Why are the changes needed?

### Does this PR introduce _any_ user-facing change?

### How was this patch tested?

Closes #1726 from AngersZhuuuu/CELEBORN-804.

Authored-by: Angerszhuuuu <angers.zhu@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2023-07-19 20:50:18 +08:00
..
common [CELEBORN-804] ShuffleClient should cleanup shuffle infos when trigger unregisterShuffle 2023-07-19 20:50:18 +08:00
flink-1.14 [CELEBORN-717][FLINK][FOLLOWUP] Fix ResultPartition lost numBytesOut/numBuffersOut metrics 2023-06-27 21:47:41 +08:00
flink-1.14-shaded [CELEBORN-716][BUILD] Correct the to name when renaming the Netty native library 2023-06-26 21:57:06 +08:00
flink-1.15 [CELEBORN-717][FLINK][FOLLOWUP] Fix ResultPartition lost numBytesOut/numBuffersOut metrics 2023-06-27 21:47:41 +08:00
flink-1.15-shaded [CELEBORN-716][BUILD] Correct the to name when renaming the Netty native library 2023-06-26 21:57:06 +08:00
flink-1.17 [CELEBORN-717][FLINK][FOLLOWUP] Fix ResultPartition lost numBytesOut/numBuffersOut metrics 2023-06-27 21:47:41 +08:00
flink-1.17-shaded [CELEBORN-716][BUILD] Correct the to name when renaming the Netty native library 2023-06-26 21:57:06 +08:00