[CELEBORN-714][HELM] Improved the local disk binding mechanism of Kubernetes HostPath

### What changes were proposed in this pull request?
Add `diskType` in `charts/celeborn/values.yml` to help configuration `celeborn.worker.storage.dirs`

Result like:
```properties
celeborn.worker.storage.dirs=/mnt/disk1:disktype=HDD,/mnt/disk2:disktype=HDD,/mnt/disk3:disktype=HDD,/mnt/disk4:disktype=SSD
```

### Why are the changes needed?
Help user specify local disk type.

### Does this PR introduce _any_ user-facing change?
Yes

### How was this patch tested?
Local dry-run
```shell
helm install celeborn charts/celeborn --dry-run
```

Closes #1623 from zwangsheng/CELEBORN-714.

Authored-by: zwangsheng <2213335496@qq.com>
Signed-off-by: mingji <fengmingxiao.fmx@alibaba-inc.com>
This commit is contained in:
zwangsheng 2023-06-26 10:52:37 +08:00 committed by mingji
parent 6b82ecdfa0
commit 1ae92b56e0
No known key found for this signature in database
GPG Key ID: 6392F71F37356FA0
2 changed files with 8 additions and 2 deletions

View File

@ -37,10 +37,11 @@ data:
celeborn.master.ha.ratis.raft.server.storage.dir={{ (index $dirs 0).mountPath }}
{{- $path := "" }}
{{- range $worker := .Values.volumes.worker }}
{{- $info := (cat $worker.mountPath ":disktype=" (get $worker "diskType" | default "HDD") | nospace) }}
{{- if eq $path "" }}
{{- $path = $worker.mountPath }}
{{- $path = $info }}
{{- else }}
{{- $path = ( list $path $worker.mountPath | join ",") }}
{{- $path = ( list $path $info | join ",") }}
{{- end }}
{{- end }}
celeborn.worker.storage.dirs={{ $path }}

View File

@ -47,6 +47,7 @@ securityContext:
# Note: size only works in emptyDir type
# hostPath only works in hostPath type using to set `volumes hostPath path`
# Celeborn Master will pick first volumes for store raft log
# diskType only works in Celeborn Worker with hostPath type to manifest local disk type
volumes:
master:
- mountPath: /mnt/rss_ratis
@ -57,18 +58,22 @@ volumes:
- mountPath: /mnt/disk1
hostPath: /mnt/disk1
type: hostPath
diskType: HDD
size: 1Gi
- mountPath: /mnt/disk2
hostPath: /mnt/disk2
type: hostPath
diskType: HDD
size: 1Gi
- mountPath: /mnt/disk3
hostPath: /mnt/disk3
type: hostPath
diskType: HDD
size: 1Gi
- mountPath: /mnt/disk4
hostPath: /mnt/disk4
type: hostPath
diskType: HDD
size: 1Gi
# celeborn configurations