apache/celeborn
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

[CELEBORN-1208][WORKER] Unify parse uniqueId to WorkerInfo

Browse Source 
### What changes were proposed in this pull request?

Unify parse uniqueId to WorkerInfo

### Why are the changes needed?

Keep parse uniqueId behavior consistent and avoid multiple changes

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
Unit test

Closes #2202 from zwangsheng/CELEBORN-1208.

Authored-by: zwangsheng <binjieyang@apache.org>
Signed-off-by: zky.zhoukeyong <zky.zhoukeyong@alibaba-inc.com>
This commit is contained in:
zwangsheng 2024-01-02 21:27:58 +08:00 committed by zky.zhoukeyong
parent 7be05b430b
commit d1b30b2827
1 changed files with 1 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
common/src/main/scala/org/apache/celeborn/common/util/PbSerDeUtils.scala
View File

@ -313,10 +313,8 @@ object PbSerDeUtils {
def fromPbWorkerResource(pbWorkerResource: util.Map[String, PbWorkerResource]): WorkerResource = {
val slots = new WorkerResource()
pbWorkerResource.asScala.foreach { case (uniqueId, pbWorkerResource) =>
val Array(host, rpcPort, pushPort, fetchPort, replicatePort) = uniqueId.split(":")
val networkLocation = pbWorkerResource.getNetworkLocation
val workerInfo =
new WorkerInfo(host, rpcPort.toInt, pushPort.toInt, fetchPort.toInt, replicatePort.toInt)
val workerInfo = WorkerInfo.fromUniqueId(uniqueId)
workerInfo.networkLocation = networkLocation
val primaryPartitionLocation = new util.ArrayList[PartitionLocation](pbWorkerResource
.getPrimaryPartitionsList.asScala.map(PbSerDeUtils.fromPbPartitionLocation).asJava)