apache/celeborn
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
508317a40d
celeborn/openapi/openapi-client
History
Wang, Fei 1515ed38b2 [CELEBORN-1477] Using openapi-generator apache-httpclient library instead of jersey2 
### What changes were proposed in this pull request?
We used `jersey2` library for celeborn-openapi-client before, and I found that there is dependencies lack issue for shaded celeborn-openapi-client.
I tried to raise a [PR #2640] to fix it, but seems It is difficult to maintain the dependencies transition from jersey dependencies.

And I received the suggestion from pan to migrate the library from jersey2 to `apache-httpclient`.

FYI: for https://openapi-generator.tech/docs/generators/java/

<img width="500" alt="image" src="https://github.com/user-attachments/assets/d102a7c9-46cd-4fd7-a2a0-7396a815776d">

To leverage the latest openapi-generator plugin, I upgrade the openapi-generator version to latest 7.7.0 and it requires JDK11+.
Due celeborn does not drop the Java8 support so far, so I include the generated code into repo and add user guide for re-generation.

### Why are the changes needed?

To fix dependencies leak issue and maintain the dependencies easily.

### Does this PR introduce _any_ user-facing change?

No, this SDK has not been released, so no user-facing change.

### How was this patch tested?

Testing with sample maven project.

pom.xml:
```
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>org.example</groupId>
    <artifactId>test_openapi</artifactId>
    <version>1.0-SNAPSHOT</version>

    <properties>
        <maven.compiler.source>8</maven.compiler.source>
        <maven.compiler.target>8</maven.compiler.target>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.apache.celeborn</groupId>
            <artifactId>celeborn-openapi-client_2.12</artifactId>
            <version>0.6.0-SNAPSHOT</version>
        </dependency>
    </dependencies>
</project>
```

Testing code:
```
package org.example;

import org.apache.celeborn.rest.v1.master.MasterApi;
import org.apache.celeborn.rest.v1.master.WorkerApi;
import org.apache.celeborn.rest.v1.master.invoker.ApiClient;

public class Main {
    public static void main(String[] args) throws Exception {

        String cmUrl = "http://***:9098";
        MasterApi masterApi  = new MasterApi(new ApiClient().setBasePath(cmUrl));
        System.out.println(masterApi.getMasterGroupInfo().getLeader().getAddress().split(":")[0]);
        WorkerApi workerApi = new WorkerApi(new ApiClient().setBasePath(cmUrl));
        System.out.println(workerApi.getWorkers());
        System.out.println(workerApi.getWorkerEvents());
    }
}
```

```
java -Dfile.encoding=UTF-8 -classpath /Users/fwang12/todo/test_openapi/target/classes:/Users/fwang12/todo/celeborn/openapi/openapi-client/target/celeborn-openapi-client_2.12-0.6.0-SNAPSHOT.jar org.example.Main
```

<img width="1727" alt="image" src="https://github.com/user-attachments/assets/2da8b126-be96-4c37-9a33-ba196024f2ba">

Closes #2641 from turboFei/appache_httpclient.

Lead-authored-by: Wang, Fei <fwang12@ebay.com>
Co-authored-by: Fei Wang <cn.feiwang@gmail.com>
Co-authored-by: Cheng Pan <pan3793@gmail.com>
Signed-off-by: Shuang <lvshuang.xjs@alibaba-inc.com>
2024-07-31 15:02:41 +08:00
..
src/main [CELEBORN-1477] Using openapi-generator apache-httpclient library instead of jersey2 2024-07-31 15:02:41 +08:00
pom.xml [CELEBORN-1477] Using openapi-generator apache-httpclient library instead of jersey2 2024-07-31 15:02:41 +08:00
README.md [CELEBORN-1477] Using openapi-generator apache-httpclient library instead of jersey2 2024-07-31 15:02:41 +08:00

README.md

Celeborn OpenAPI Client

To update the OpenAPI specification

  • just update the specification under openapi/openapi-client/src/main/openapi3/ and keep the schema definitions consistent between master and worker.
  • Install JDK 11 or above by whatever mechanism is appropriate for your system, and set that version to be the default Java version (e.g., by setting env variable JAVA_HOME)
  • run the following: 
    build/mvn -pl openapi/openapi-client clean package -Pgenerate
    or 
    build/sbt "clean;celeborn-openapi-client/generate"
    This will regenerate the OpenAPI data models + APIs in the celeborn-openapi-client SDK.