apache/celeborn
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
15d1463be8
celeborn/openapi/openapi-client
History
mingji df01fadc9f
[CELEBORN-1601] Support revise lost shuffles 
### What changes were proposed in this pull request?
To support revising lost shuffle IDs in a long-running job such as flink batch jobs.

### Why are the changes needed?
1. To support revise lost shuffles.
2. To add an HTTP endpoint to revise lost shuffles manually.

### Does this PR introduce _any_ user-facing change?
NO.

### How was this patch tested?
Cluster tests.

Closes #2746 from FMX/b1600.

Lead-authored-by: mingji <fengmingxiao.fmx@alibaba-inc.com>
Co-authored-by: Ethan Feng <fengmingxiao.fmx@alibaba-inc.com>
Signed-off-by: SteNicholas <programgeek@163.com>
2024-10-21 16:44:37 +08:00
..
src/main [CELEBORN-1601] Support revise lost shuffles 2024-10-21 16:44:37 +08:00
pom.xml [CELEBORN-1607] Enable useEnumCaseInsensitive for openapi-generator 2024-09-23 20:43:04 +08:00
README.md [CELEBORN-1477] Using openapi-generator apache-httpclient library instead of jersey2 2024-07-31 15:02:41 +08:00

README.md

Celeborn OpenAPI Client

To update the OpenAPI specification

  • just update the specification under openapi/openapi-client/src/main/openapi3/ and keep the schema definitions consistent between master and worker.
  • Install JDK 11 or above by whatever mechanism is appropriate for your system, and set that version to be the default Java version (e.g., by setting env variable JAVA_HOME)
  • run the following: 
    build/mvn -pl openapi/openapi-client clean package -Pgenerate
    or 
    build/sbt "clean;celeborn-openapi-client/generate"
    This will regenerate the OpenAPI data models + APIs in the celeborn-openapi-client SDK.