apache/celeborn
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

[CELEBORN-2134] When creating a DiskFile, retrieve the storage type b…

Browse Source 
…ased on the mount point

### What changes were proposed in this pull request?

 When creating a DiskFile, retrieve the storage type based on the mount point

### Why are the changes needed?

If the worker disk is an SSD, it should be set to the SSD type, and the storage type can be retrieved based on the mount point.

### Does this PR introduce _any_ user-facing change?

NO

### How was this patch tested?

CI

Closes #3456 from xy2953396112/CELEBORN-2134.

Authored-by: xxx <953396112@qq.com>
Signed-off-by: mingji <fengmingxiao.fmx@alibaba-inc.com>
This commit is contained in:
xxx 2025-08-28 17:21:51 +08:00 committed by mingji
parent 1be3094fb2
commit db70473de2
1 changed files with 2 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
worker/src/main/scala/org/apache/celeborn/service/deploy/worker/storage/StorageManager.scala
View File

@ -1175,12 +1175,13 @@ final private[worker] class StorageManager(conf: CelebornConf, workerSource: Abs
}
val filePath = file.getAbsolutePath
val fileMeta = getFileMeta(partitionType, mountPoint, conf.shuffleChunkSize)
val storageType = diskInfos.get(mountPoint).storageType
val diskFileInfo = new DiskFileInfo(
userIdentifier,
partitionSplitEnabled,
fileMeta,
filePath,
StorageInfo.Type.HDD)
storageType)
logInfo(s"created file at $filePath")
diskFileInfos.computeIfAbsent(shuffleKey, diskFileInfoMapFunc).put(
fileName,