celeborn/assets
Wang, Fei def5254ec2
[CELEBORN-1706] Use bytes(IEC) unit instead of bytes(SI) for size related metrics in prometheus dashboard
### What changes were proposed in this pull request?
Use unit `bytes(IEC)`(`decbytes`, 1,024 bytes in a kibibyte ) for below 18 metrics(disk and memory related) instead of `bytes(SI)`(`bytes`, 1,000 bytes in a kilobyte).
- metrics_DeviceCelebornTotalBytes_Value
- metrics_DeviceCelebornFreeBytes_Value
- metrics_PartitionSize_Value
- metrics_ActiveShuffleSize_Value
- metrics_NettyMemory_Value
- metrics_DiskBuffer_Value
- metrics_push_usedHeapMemory_Value
- metrics_push_usedDirectMemory_Value
- metrics_fetch_usedHeapMemory_Value
- metrics_fetch_usedDirectMemory_Value
- metrics_replicate_usedHeapMemory_Value
- metrics_replicate_usedDirectMemory_Value
- metrics_BufferStreamReadBuffer_Value
- metrics_SortMemory_Value
- metrics_DeviceOSFreeBytes_Value
- metrics_DeviceCelebornFreeBytes_Value
- metrics_diskBytesWritten_Value
- metrics_hdfsBytesWritten_Value

Also apply for 6 jvm metrics
- metrics_jvm_memory_heap_init_Value
- metrics_jvm_memory_non_heap_init_Value
- metrics_jvm_memory_total_init_Value
- metrics_jvm_memory_pools_init_Value
- metrics_jvm_direct_capacity_Value
- metrics_jvm_mapped_capacity_Value

### Why are the changes needed?

Some size related metrics use `bytes(IEC)` and some use `bytes(SI)`.
<img width="1715" alt="image" src="https://github.com/user-attachments/assets/8dd1727b-4e16-487c-b2f9-f70116bc27d3">

<img width="1722" alt="image" src="https://github.com/user-attachments/assets/17ed933a-3f01-4a91-a170-aa7a042f4947">

The main difference between bytes in the International System of Units (SI) and the International Electrotechnical Commission (IEC) is the number of bytes in a kilobyte:
SI: 1,000 bytes in a kilobyte
IEC: 1,024 bytes in a kibibyte

FYI: https://www.drupal.org/project/drupal/issues/1114538#:~:text=According%20to%20the%20SI%20standard,e.g.%20a%20stick%20of%20RAM.

4545cdc401/assets/grafana/celeborn-dashboard.json (L5636-L5699)

### Does this PR introduce _any_ user-facing change?

Yes, metrics unit changed.

### How was this patch tested?
Not needed, we already use `decbytes` in the dashboard json.

Closes #2896 from turboFei/unit_decbytes.

Authored-by: Wang, Fei <fwang12@ebay.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2024-11-14 10:40:57 +08:00
..
diagram [CELEBORN-746][BUILD] Rename project files from rss-xx to celeborn-xx 2023-06-29 16:30:02 +08:00
grafana [CELEBORN-1706] Use bytes(IEC) unit instead of bytes(SI) for size related metrics in prometheus dashboard 2024-11-14 10:40:57 +08:00
spark-patch [CELEBORN-858] Generate patch to each Spark 3.x minor version 2023-11-10 15:35:54 +08:00