celeborn/common
TheodoreLx 1ead784fa1 [CELEBORN-2085] Use a fixed buffer for flush copying to reduce GC
### What changes were proposed in this pull request?

Apply for a byte array in advance and use it as a transfer when copying is needed during flush

### Why are the changes needed?

For HdfsFlushTask, OssFlushTask, and S3FlushTask, you need to copy the CompositeByteBuf in the parameter to a byte array when flushing, and then use the respective clients to write the byte array to the storage.
When the flush throughput rate is very high, this copying will cause very serious GC problems and affect the performance of the worker

### Does this PR introduce _any_ user-facing change?

no

### How was this patch tested?

cluster test

Closes #3394 from TheodoreLx/copy-on-flush.

Authored-by: TheodoreLx <1548069580@qq.com>
Signed-off-by: Shuang <lvshuang.xjs@alibaba-inc.com>
2025-08-08 13:57:21 +08:00
..
benchmarks
src [CELEBORN-2085] Use a fixed buffer for flush copying to reduce GC 2025-08-08 13:57:21 +08:00
pom.xml [CELEBORN-1530] support MPU for S3 2024-11-22 15:03:53 +08:00