### What changes were proposed in this pull request?
This PR adds support for zstd compression in CppClient.
### Why are the changes needed?
To support writing to Celeborn with CppClient.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
By compilation and UTs.
Closes#3454 from Jraaay/feat/cpp_client_zstd_compression.
Authored-by: Jray <1075860716@qq.com>
Signed-off-by: SteNicholas <programgeek@163.com>
### What changes were proposed in this pull request?
Support PushData network message in cppClient.
### Why are the changes needed?
PushData is the network message of writing to cppClient.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Compilation and UTs.
Closes#3434 from HolyLow/issue/celeborn-2115-support-push-date-in-cpp-client.
Authored-by: HolyLow <jiaming.xie7@gmail.com>
Signed-off-by: mingji <fengmingxiao.fmx@alibaba-inc.com>
### What changes were proposed in this pull request?
This PR adds support for lz4 compression in CppClient.
### Why are the changes needed?
To support writing to Celeborn with CppClient.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
By compilation and UTs.
Closes#3412 from Jraaay/feat/cpp_client_lz4_compression.
Authored-by: Jray <1075860716@qq.com>
Signed-off-by: Shuang <lvshuang.xjs@alibaba-inc.com>
### What changes were proposed in this pull request?
This PR supports Revive/ChangeLocationResponse messages in cppClient.
### Why are the changes needed?
These messages are used when writing triggers revive operation.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Compilation and UTs.
Closes#3413 from HolyLow/issue/celeborn-2098-support-revive-changelocationresponse.
Authored-by: HolyLow <jiaming.xie7@gmail.com>
Signed-off-by: SteNicholas <programgeek@163.com>
### What changes were proposed in this pull request?
Fix performance issue on ReadOnlyByteBuffer::readToReadOnlyBuffer.
### Why are the changes needed?
ReadOnlyByteBuffer::readToReadOnlyBuffer now is slow on a long iobuf chain because it used wrong api to clone an iobuf block.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
By compilation and UTs.
Closes#3415 from Jraaay/fix/readToReadOnlyBuffer.
Authored-by: Jray <1075860716@qq.com>
Signed-off-by: SteNicholas <programgeek@163.com>
### What changes were proposed in this pull request?
Support RegisterShuffle/Response messages in CppClient.
### Why are the changes needed?
To support the procedure of registering shuffle and accepting response in CppClient.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Compilation and UTs.
Closes#3410 from HolyLow/issue/celeborn-2095-support-registershuffle-response.
Authored-by: HolyLow <jiaming.xie7@gmail.com>
Signed-off-by: mingji <fengmingxiao.fmx@alibaba-inc.com>
### What changes were proposed in this pull request?
This PR adds support for zstd decompression in CppClient.
### Why are the changes needed?
To support reading from Celeborn with CppClient.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
By compilation and UTs.
Closes#3411 from Jraaay/feat/cpp_client_zstd_decompression.
Authored-by: Jray <1075860716@qq.com>
Signed-off-by: mingji <fengmingxiao.fmx@alibaba-inc.com>
### What changes were proposed in this pull request?
This PR adds support for lz4 decompression in CppClient.
### Why are the changes needed?
To support reading from Celeborn with CppClient.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
By compilation and UTs.
Closes#3402 from Jraaay/feat/cpp_client_lz4_decompression.
Authored-by: Jray <1075860716@qq.com>
Signed-off-by: SteNicholas <programgeek@163.com>
### What changes were proposed in this pull request?
This PR adds support for MapperEnd/Response in CppClient.
### Why are the changes needed?
To support writing to Celeborn with CppClient.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
By compilation and UTs.
Closes#3372 from HolyLow/issue/celeborn-2070-support-registershuffle-mapperend.
Authored-by: HolyLow <jiaming.xie7@gmail.com>
Signed-off-by: SteNicholas <programgeek@163.com>
### What changes were proposed in this pull request?
Inspired by [FLINK-38038](https://issues.apache.org/jira/projects/FLINK/issues/FLINK-38038?filter=allissues]), I used [Tongyi Lingma](https://lingma.aliyun.com/) and qwen3-thinking LLM to identify and fix some typo issues in the Celeborn codebase. For example:
- backLog → backlog
- won`t → won't
- can to be read → can be read
- mapDataPartition → mapPartitionData
- UserDefinePasswordAuthenticationProviderImpl → UserDefinedPasswordAuthenticationProviderImpl
### Why are the changes needed?
Remove typos to improve source code readability for users and ease development for developers.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Code and documentation cleanup does not require additional testing.
Closes#3356 from codenohup/fix-typo.
Authored-by: codenohup <huangxu.walker@gmail.com>
Signed-off-by: SteNicholas <programgeek@163.com>
### What changes were proposed in this pull request?
This PR adds code style checking in github action for cppClient.
### Why are the changes needed?
To keep the cpp code style consistent.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
By github action procedure.
Closes#3252 from HolyLow/issue/celeborn-1978-add-codestyle-check-to-cppclient.
Authored-by: HolyLow <jiaming.xie7@gmail.com>
Signed-off-by: mingji <fengmingxiao.fmx@alibaba-inc.com>
### What changes were proposed in this pull request?
This PR adds joint test suite between java and cpp code, which test writing data with javaClient and reading with cppClient. Besides, the CICD procedure is added to verify the code automatically.
### Why are the changes needed?
The joint test suite could be used to verify the cppClient's correctness.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Compilation and CICD procedure.
Closes#3202 from HolyLow/issue/celeborn-1958-add-integration-test-to-cppclient.
Authored-by: HolyLow <jiaming.xie7@gmail.com>
Signed-off-by: mingji <fengmingxiao.fmx@alibaba-inc.com>
### What changes were proposed in this pull request?
This PR adds reader end's ShuffleClient to cppClient.
### Why are the changes needed?
ShuffleClient is the user interface for cppClient usage.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Compilation.
Closes#3156 from HolyLow/issue/celeborn-1915-add-reader-shuffle-client-to-cppclient.
Authored-by: HolyLow <jiaming.xie7@gmail.com>
Signed-off-by: mingji <fengmingxiao.fmx@alibaba-inc.com>
### What changes were proposed in this pull request?
This PR adds CelebornInputStream to cppClient.
### Why are the changes needed?
The CelebornInputStream is the readerClient's feeding stream.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Compilation.
Closes#3151 from HolyLow/issue/celeborn-1906-add-celeborn-input-stream-to-cppclient.
Authored-by: HolyLow <jiaming.xie7@gmail.com>
Signed-off-by: mingji <fengmingxiao.fmx@alibaba-inc.com>
### What changes were proposed in this pull request?
This PR adds WorkerPartitionReader to cppClient.
### Why are the changes needed?
WorkerPartitionReader is the building block of CelebornInputStream.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Compilation and UTs.
Closes#3137 from HolyLow/issue/celeborn-1881-add-workerpartitionreader-to-cppclient.
Authored-by: HolyLow <jiaming.xie7@gmail.com>
Signed-off-by: mingji <fengmingxiao.fmx@alibaba-inc.com>
### What changes were proposed in this pull request?
This PR adds NettyRpcEndpointRef to cppClient.
### Why are the changes needed?
NettyRpcEndpointRef is responsible for communicating with LifecycleManager.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Compilation and UTs.
Closes#3110 from HolyLow/issue/celeborn-1871-add-nettyrpcendpointref-to-cpp-client.
Authored-by: HolyLow <jiaming.xie7@gmail.com>
Signed-off-by: Shuang <lvshuang.xjs@alibaba-inc.com>
### What changes were proposed in this pull request?
This PR adds TransportClient to cppClient.
### Why are the changes needed?
The TransportClient is responsible for sending/receiving the message to/from network.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Compilation and UTs.
Closes#3097 from HolyLow/issue/celeborn-1863-add-transport-client-to-cpp-client.
Authored-by: HolyLow <jiaming.xie7@gmail.com>
Signed-off-by: mingji <fengmingxiao.fmx@alibaba-inc.com>
### What changes were proposed in this pull request?
This PR adds MessageDispatcher class to cppClient.
### Why are the changes needed?
MessageDispatcher is responsible for recording the connection between MessageFuture and MessagePromise, which is the base of async message transferring mechanism.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Compilation and UTs.
Closes#3077 from HolyLow/issue/celeborn-1845-add-message-dispatcher-to-cpp-client.
Authored-by: HolyLow <jiaming.xie7@gmail.com>
Signed-off-by: mingji <fengmingxiao.fmx@alibaba-inc.com>
### What changes were proposed in this pull request?
This PR adds Message implementation to cppClient.
### Why are the changes needed?
The Message is the data structure that transfers between network stack layers.
The decode/encode methods are supported and are compatible with existing java implementation.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Compilation and UTs.
Closes#3066 from HolyLow/issue/celeborn-1836-add-message-to-cpp-client.
Authored-by: HolyLow <jiaming.xie7@gmail.com>
Signed-off-by: Shuang <lvshuang.xjs@alibaba-inc.com>
### What changes were proposed in this pull request?
This PR adds MessageDecoder to cppClient.
### Why are the changes needed?
MessageDecoder is the underlaying decoding stack for wangle network msg processing framework.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Compilation and UTs.
Closes#3060 from HolyLow/issue/celeborn-1827-add-message-decoder-to-cpp-client.
Authored-by: HolyLow <jiaming.xie7@gmail.com>
Signed-off-by: mingji <fengmingxiao.fmx@alibaba-inc.com>
### What changes were proposed in this pull request?
This PR adds ControlMessages to cppClient.
### Why are the changes needed?
The ControlMessages are used to communicate with the CelebornServer and LifecycleManager.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Compilation and UTs.
Closes#3052 from HolyLow/issue/celeborn-1821-add-control-messages-to-cpp-client.
Authored-by: HolyLow <jiaming.xie7@gmail.com>
Signed-off-by: mingji <fengmingxiao.fmx@alibaba-inc.com>
### What changes were proposed in this pull request?
This PR refactors cppClient to use nested namespace.
### Why are the changes needed?
The nested namespace would improve symbol isolation and make the celebornCpp project more extensible.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Compilation and UTs.
Closes#3050 from HolyLow/issue/celeborn-1819-refactor-cppClient-with-nested-namespace.
Authored-by: HolyLow <jiaming.xie7@gmail.com>
Signed-off-by: mingji <fengmingxiao.fmx@alibaba-inc.com>
### What changes were proposed in this pull request?
Add transportMessage to cppClient.
### Why are the changes needed?
TransportMessage is the building block of controlMessages.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Compilation and UTs.
Closes#3042 from HolyLow/issue/celeborn-1814-add-transport-message-to-cppClient.
Authored-by: HolyLow <jiaming.xie7@gmail.com>
Signed-off-by: mingji <fengmingxiao.fmx@alibaba-inc.com>
### What changes were proposed in this pull request?
This PR adds PartitionLocation to cppClient, which is the component of protocol module.
### Why are the changes needed?
To support communication message of PartitionLocation.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Compilation and UTs.
Closes#3035 from HolyLow/issue/celeborn-1809-add-partition-location-to-cppClient.
Authored-by: HolyLow <jiaming.xie7@gmail.com>
Signed-off-by: mingji <fengmingxiao.fmx@alibaba-inc.com>
### What changes were proposed in this pull request?
Add CelebornConf to cppClient.
### Why are the changes needed?
The CelebornConf will be used as configuration module in cppClient.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Compilation and UTs.
Closes#3027 from HolyLow/issue/celeborn-1799-add-celeborn-conf-to-cppClient.
Authored-by: HolyLow <jiaming.xie7@gmail.com>
Signed-off-by: mingji <fengmingxiao.fmx@alibaba-inc.com>
### What changes were proposed in this pull request?
Add baseConf to cppClient, which is the building block of conf module.
### Why are the changes needed?
To support CelebornCpp configuration module in cppClient.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Compilation and UTs.
Closes#3013 from HolyLow/issue/celeborn-1785-add-base-conf-to-cppClient.
Authored-by: HolyLow <jiaming.xie7@gmail.com>
Signed-off-by: mingji <fengmingxiao.fmx@alibaba-inc.com>
### What changes were proposed in this pull request?
Add memory module to cppClient to provide ByteBuffer functionality.
### Why are the changes needed?
The memory module is added to provide ByteBuffer functionality, which would be used across the data parsing layers.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Compilation and UTs.
Closes#2996 from HolyLow/issue/celeborn-1772-add-memory-module-to-cppClient.
Authored-by: HolyLow <jiaming.xie7@gmail.com>
Signed-off-by: mingji <fengmingxiao.fmx@alibaba-inc.com>
### What changes were proposed in this pull request?
Add cppProto (TransportMessages.proto, softlinked as TransportMessagesCpp.proto) to cppClient
### Why are the changes needed?
To support proto in cppClient and align with java's definition.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Compilation.
Closes#2976 from HolyLow/issue/celeborn-1761-add-cpp-proto-to-cppClient.
Authored-by: HolyLow <jiaming.xie7@gmail.com>
Signed-off-by: mingji <fengmingxiao.fmx@alibaba-inc.com>
### What changes were proposed in this pull request?
This PR adds exceptions and checking utils code to CppClient.
Besides, the ctest framework is added to CppClient for UTs.
### Why are the changes needed?
To provide exception utils and UT frmework to CppClient.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Compilation and UTs.
Closes#2966 from HolyLow/issue/celeborn-1754-add-exceptions-utils-to-cppClient.
Authored-by: HolyLow <jiaming.xie7@gmail.com>
Signed-off-by: mingji <fengmingxiao.fmx@alibaba-inc.com>
### What changes were proposed in this pull request?
This PR adds CelebornException utils code to CppClient.
### Why are the changes needed?
To provide CelebornException utils.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Compilation.
Closes#2958 from HolyLow/issue/celeborn-1751-add-celeborn-exception-utils-to-cppClient.
Authored-by: HolyLow <jiaming.xie7@gmail.com>
Signed-off-by: mingji <fengmingxiao.fmx@alibaba-inc.com>
### What changes were proposed in this pull request?
This PR adds StackTrace utils code to CppClient.
### Why are the changes needed?
To provide StackTrace utils.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Compilation.
Closes#2951 from HolyLow/issue/celeborn-1740-add-stacktrace-utils-to-cppClient.
Authored-by: HolyLow <jiaming.xie7@gmail.com>
Signed-off-by: mingji <fengmingxiao.fmx@alibaba-inc.com>
### What changes were proposed in this pull request?
This PR adds CMakeList structure and ProcessBase utils code to CppClient.
### Why are the changes needed?
To organize the compiling structure and to provide ProcessBase utils.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Compilation.
Closes#2940 from HolyLow/issue/celeborn-1741-add-processbase-utils-to-cppClient.
Authored-by: HolyLow <jiaming.xie7@gmail.com>
Signed-off-by: mingji <fengmingxiao.fmx@alibaba-inc.com>
### What changes were proposed in this pull request?
This PR adds environment setup tools, docker image & container especially, for CppClient development.
### Why are the changes needed?
To develop CppClient functionality and solve library dependencies.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
E2E.
Closes#2927 from HolyLow/issue/celeborn-1724-add-cppClient-devTools.
Authored-by: HolyLow <jiaming.xie7@gmail.com>
Signed-off-by: mingji <fengmingxiao.fmx@alibaba-inc.com>
