apache/celeborn
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

[CELEBORN-1759] Fix reserve slots might lost partition location between 0.4 client and 0.5 server

Browse Source 
### What changes were proposed in this pull request?
Fix the worker parses `ReserveSlots` logic for compatibility

### Why are the changes needed?
When upgrading to 0.5, the 0.4 client reserves slots for the 0.5 worker. If there is only a replicate location, the worker parses abnormally, causing the actual reserve to fail, but returns success to the client.
The worker log "Reserved 0 primary location and 0 replica location" appears.

### Does this PR introduce _any_ user-facing change?
When upgrading to 0.5 from 0.4, fix potential reserve slot failure scenario.(only replica location).

### How was this patch tested?
Manual test.

Closes #2968 from onebox-li/fix-reserve-compatibility.

Authored-by: onebox-li <lyh-36@163.com>
Signed-off-by: mingji <fengmingxiao.fmx@alibaba-inc.com>
This commit is contained in:
onebox-li 2024-12-03 16:57:53 +08:00 committed by mingji
parent 3dd810cd9b
commit 7102174eda
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
common/src/main/scala/org/apache/celeborn/common/protocol/message/ControlMessages.scala
View File

@ -1271,7 +1271,7 @@ object ControlMessages extends Logging {
val pbReserveSlots = PbReserveSlots.parseFrom(message.getPayload)
val userIdentifier = PbSerDeUtils.fromPbUserIdentifier(pbReserveSlots.getUserIdentifier)
val (primaryLocations, replicateLocations) =
if (pbReserveSlots.getPrimaryLocationsList.isEmpty) {
if (pbReserveSlots.getPrimaryLocationsList.isEmpty && pbReserveSlots.getReplicaLocationsList.isEmpty) {
PbSerDeUtils.fromPbPackedPartitionLocationsPair(
pbReserveSlots.getPartitionLocationsPair)
} else {