apache/celeborn
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

[CELEBORN-789] Increase default value of flushBuffer's max components

Browse Source 
### What changes were proposed in this pull request?
Set default value of ```celeborn.worker.push.compositeBuffer.maxComponents``` to 256, to be aligned with 0.2.1-incubating version.

### Why are the changes needed?

Default 16 is too small, and causes ~~severe GC~~ and CPU high load.

<img width="1719" alt="image" src="https://github.com/apache/incubator-celeborn/assets/26535726/9ab9675e-c19e-44f1-af46-90c29dc4df75">

### Does this PR introduce _any_ user-facing change?
No, it's internal config.

### How was this patch tested?
Passes GA.

Closes #1707 from waitinfuture/789.

Authored-by: zky.zhoukeyong <zky.zhoukeyong@alibaba-inc.com>
Signed-off-by: zky.zhoukeyong <zky.zhoukeyong@alibaba-inc.com>
This commit is contained in:
zky.zhoukeyong 2023-07-12 20:18:48 +08:00
parent 1642090f9f
commit dcf6be29d8
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
common/src/main/scala/org/apache/celeborn/common/CelebornConf.scala
View File

@ -2600,7 +2600,7 @@ object CelebornConf extends Logging {
"When set to 1, Netty's direct memory is close to disk buffer, but performance " +
"might decrease due to frequent memory copy during compaction.")
.intConf
.createWithDefault(16)
.createWithDefault(128)
val WORKER_FETCH_HEARTBEAT_ENABLED: ConfigEntry[Boolean] =
buildConf("celeborn.worker.fetch.heartbeat.enabled")