celeborn/assets
Wang, Fei 81a0d5113c [CELEBORN-1660] Cache available workers and only count the available workers device free capacity
### What changes were proposed in this pull request?
1. cache the available workers
2. Only count the available workers device free capacity.
3. place the metrics_AvailableWorkerCount_Value in overall and metrics_WorkerCount_Value in `Master` part

### Why are the changes needed?
Cache  the available workers to reduce the computation that need to loop the workers frequently.
To have an accurate device capacity overview that does not include the excluded workers, decommissioning workers, etc.

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?
UT.

<img width="1705" alt="image" src="https://github.com/user-attachments/assets/bee17b4e-785d-4112-8410-dbb684270ec0">

Closes #2827 from turboFei/device_free.

Authored-by: Wang, Fei <fwang12@ebay.com>
Signed-off-by: Shuang <lvshuang.xjs@alibaba-inc.com>
2024-11-14 11:10:45 +08:00
..
diagram [CELEBORN-746][BUILD] Rename project files from rss-xx to celeborn-xx 2023-06-29 16:30:02 +08:00
grafana [CELEBORN-1660] Cache available workers and only count the available workers device free capacity 2024-11-14 11:10:45 +08:00
spark-patch [CELEBORN-858] Generate patch to each Spark 3.x minor version 2023-11-10 15:35:54 +08:00