apache/celeborn
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

[CELEBORN-1687] Highlight flink session cluster issue in doc

Browse Source 
### What changes were proposed in this pull request?

If we use celeborn shuffle service, we can't submit both batch and streaming to the same flink session cluster. This should be highlight in doc.

### Does this PR introduce _any_ user-facing change?

No

### How was this patch tested?

No need.

Closes #2879 from reswqa/session-doc.

Authored-by: Weijie Guo <reswqa@163.com>
Signed-off-by: mingji <fengmingxiao.fmx@alibaba-inc.com>
This commit is contained in:
Weijie Guo 2024-11-06 10:52:34 +08:00 committed by mingji
parent 64f201dd83
commit f2e9043028
2 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
README.md
View File

@ -312,7 +312,7 @@ spark.executor.userClassPathFirst false
### Deploy Flink client
**Important: Only Flink batch jobs are supported for now.**
**Important: Only Flink batch jobs are supported for now. Due to the Shuffle Service in Flink is cluster-granularity, if you want to use Celeborn in a session cluster, it will not be able to submit both streaming and batch job to the same cluster. We plan to get rid of this restriction for Hybrid Shuffle mode in a future release.**
Copy `$CELEBORN_HOME/flink/*.jar` to `$FLINK_HOME/lib/`.

2
docs/deploy.md
View File

@ -210,7 +210,7 @@ spark.executor.userClassPathFirst false
## Deploy Flink client
**Important: Only Flink batch jobs are supported for now.**
**Important: Only Flink batch jobs are supported for now. Due to the Shuffle Service in Flink is cluster-granularity, if you want to use Celeborn in a session cluster, it will not be able to submit both streaming and batch job to the same cluster. We plan to get rid of this restriction for Hybrid Shuffle mode in a future release.**
Celeborn release binary contains clients for Flink 1.14.x, Flink 1.15.x, Flink 1.17.x, Flink 1.18.x, Flink 1.19.x and Flink 1.20.x, copy the corresponding client jar into Flink's
`lib/` directory: