apache/celeborn
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

[CELEBORN-2007] Reduce PartitionLocation memory usage

Browse Source 
### What changes were proposed in this pull request?

### Why are the changes needed?
Driver may have a large number of `PartitionLocation` objects, reducing some unnecessary fields of `PartitionLocation` can reduce the memory pressure of Driver.

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
GA

Closes #3274 from cxzl25/CELEBORN-2007.

Authored-by: sychen <sychen@ctrip.com>
Signed-off-by: Wang, Fei <fwang12@ebay.com>
This commit is contained in:
sychen 2025-05-22 17:20:58 -07:00 committed by Wang, Fei
parent d2befe0334
commit 634343e200
1 changed files with 6 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
common/src/main/java/org/apache/celeborn/common/protocol/PartitionLocation.java
View File

@ -78,8 +78,6 @@ public class PartitionLocation implements Serializable {
this.peer = loc.peer;
this.storageInfo = loc.storageInfo;
this.mapIdBitMap = loc.mapIdBitMap;
this._hostPushPort = host + ":" + pushPort;
this._hostFetchPort = host + ":" + fetchPort;
}
public PartitionLocation(
@ -152,8 +150,6 @@ public class PartitionLocation implements Serializable {
this.peer = peer;
this.storageInfo = hint;
this.mapIdBitMap = mapIdBitMap;
this._hostPushPort = host + ":" + pushPort;
this._hostFetchPort = host + ":" + fetchPort;
}
public int getId() {
@ -210,10 +206,16 @@ public class PartitionLocation implements Serializable {
}
public String hostAndFetchPort() {
if (_hostFetchPort == null) {
_hostFetchPort = host + ":" + fetchPort;
}
return _hostFetchPort;
}
public String hostAndPushPort() {
if (_hostPushPort == null) {
_hostPushPort = host + ":" + pushPort;
}
return _hostPushPort;
}