apache/celeborn
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

[MINOR] Remove unused TODO comments in CelebornTierProducerAgent#processBuffer

Browse Source 
### What changes were proposed in this pull request?
Remove unused TODO comments in CelebornTierProducerAgent#processBuffer

### Why are the changes needed?
In order for buffers to be packed together, we are going to modify the Flink side implementation to delegate buffer compression to tiers. But after discussion, we have been able to handle the case of receiving the compressed buffer on the Celeborn side, so this TODO is no longer needed.

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
No need.

Closes #2883 from reswqa/remove_unused_todo.

Authored-by: Weijie Guo <reswqa@163.com>
Signed-off-by: Shuang <lvshuang.xjs@alibaba-inc.com>
This commit is contained in:
Weijie Guo 2024-11-07 10:58:58 +08:00 committed by Shuang
parent 7dcd25925f
commit d44b23c852
1 changed files with 3 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
client-flink/flink-1.20/src/main/java/org/apache/celeborn/plugin/flink/tiered/CelebornTierProducerAgent.java
View File

@ -453,9 +453,9 @@ public class CelebornTierProducerAgent implements TierProducerAgent {
Buffer buffer = originBuffer;
if (originBuffer.isCompressed()) {
// In flink 1.20.0, it will receive a compressed buffer. However, since we need to write
// data to this buffer and the compressed buffer is read-only,
// we must create a new Buffer object to the wrap origin buffer.
// Flink hybrid shuffle will send a compressed buffer to tier. However, since we need to
// write data to this buffer and the compressed buffer is read-only, we must create a
// new Buffer object to the wrap origin buffer.
NetworkBuffer networkBuffer =
new NetworkBuffer(
originBuffer.getMemorySegment(),
@ -466,9 +466,6 @@ public class CelebornTierProducerAgent implements TierProducerAgent {
buffer = networkBuffer;
}
// TODO: To enhance performance, the flink should pass an no-compressed buffer to producer
// agent and we compress the buffer here
// set the buffer meta
BufferUtils.setCompressedDataWithoutHeader(buffer, originBuffer);