efa22a4936
### What changes were proposed in this pull request? ### Why are the changes needed? ```bash flink-1.18.0 ./bin/start-cluster.sh ./bin/flink run examples/streaming/WordCount.jar --execution-mode BATCH ``` ```java Caused by: java.lang.NoSuchMethodError: org.apache.flink.runtime.io.network.partition.consumer.SingleInputGate.<init>(Ljava/lang/String;ILorg/apache/flink/runtime/jobgraph/IntermediateDataSetID;Lorg/apache/flink/runtime/io/network/partition/ResultPartitionType;Lorg/apache/flink/runtime/executiongraph/IndexRange;ILorg/apache/flink/runtime/io/network/partition/PartitionProducerStateProvider;Lorg/apache/flink/util/function/SupplierWithException;Lorg/apache/flink/runtime/io/network/buffer/BufferDecompressor;Lorg/apache/flink/core/memory/MemorySegmentProvider;ILorg/apache/flink/runtime/throughput/ThroughputCalculator;Lorg/apache/flink/runtime/throughput/BufferDebloater;)V at org.apache.celeborn.plugin.flink.RemoteShuffleInputGate$FakedRemoteInputChannel.<init>(RemoteShuffleInputGate.java:225) at org.apache.celeborn.plugin.flink.RemoteShuffleInputGate.getChannel(RemoteShuffleInputGate.java:179) at org.apache.flink.runtime.io.network.partition.consumer.InputGate.setChannelStateWriter(InputGate.java:90) at org.apache.flink.runtime.taskmanager.InputGateWithMetrics.setChannelStateWriter(InputGateWithMetrics.java:120) at org.apache.flink.streaming.runtime.tasks.StreamTask.injectChannelStateWriterIntoChannels(StreamTask.java:524) at org.apache.flink.streaming.runtime.tasks.StreamTask.<init>(StreamTask.java:496) ``` Flink 1.18.0 release https://flink.apache.org/2023/10/24/announcing-the-release-of-apache-flink-1.18/ Interface `org.apache.flink.runtime.io.network.buffer.Buffer` adds `setRecycler` method. [[FLINK-32549](https://issues.apache.org/jira/browse/FLINK-32549)][network] Tiered storage memory manager supports ownership transfer for buffers `org.apache.flink.runtime.io.network.partition.consumer.SingleInputGate` constructor adds parameters. [[FLINK-31638](https://issues.apache.org/jira/browse/FLINK-31638)][network] Introduce the TieredStorageConsumerClient to SingleInputGate [[FLINK-31642](https://issues.apache.org/jira/browse/FLINK-31642)][network] Introduce the MemoryTierConsumerAgent to TieredStorageConsumerClient ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? ```bash flink-1.18.0 ./bin/flink run examples/streaming/WordCount.jar --execution-mode BATCH Executing example with default input data. Use --input to specify file input. Printing result to stdout. Use --output to specify output path. Job has been submitted with JobID d7fc5f0ca018a54e9453c4d35f7c598a Program execution finished Job with JobID d7fc5f0ca018a54e9453c4d35f7c598a has finished. Job Runtime: 1635 ms ``` <img width="1297" alt="image" src="https://github.com/apache/incubator-celeborn/assets/3898450/6a5266bf-2386-4386-b98b-a60d2570fa99"> Closes #2063 from cxzl25/CELEBORN-1105. Authored-by: sychen <sychen@ctrip.com> Signed-off-by: Shuang <lvshuang.tb@gmail.com> |
||
|---|---|---|
| .. | ||
| client.md | ||
| faulttolerant.md | ||
| flink.md | ||
| glutensupport.md | ||
| integrate.md | ||
| lifecyclemanager.md | ||
| master.md | ||
| overview.md | ||
| readdata.md | ||
| sbt.md | ||
| shuffleclient.md | ||
| slotsallocation.md | ||
| spark.md | ||
| storage.md | ||
| trafficcontrol.md | ||
| worker.md | ||
| workerexclusion.md | ||