Commit Graph

1099 Commits

Author SHA1 Message Date
Ethan Feng
971c93d4d9
[CELEBORN-419][FLINK] Fix memory leak when receive RPCs with body. (#1343) 2023-03-14 11:27:36 +08:00
Shuang
b499c0df7f
[CELEBORN-417][FLINK] fix memory leak when handler already removed (#1342) 2023-03-13 23:58:21 +08:00
If Wind
44ba2d00c9
[CELEBORN-415] Fix syntax error in prometheus-podmonitor.yaml (#1340) 2023-03-13 19:51:11 +08:00
Angerszhuuuu
3600ccc4e3
[CELEBORN-409] Add PartitionLocationInfo to worker's http request (#1335) 2023-03-13 17:02:28 +08:00
Angerszhuuuu
6f1ab70403
[CELEBORN-406] Add blacklist to http request to indicate blacklisted worker (#1334) 2023-03-13 16:44:46 +08:00
Angerszhuuuu
144a8cdb3f
[CELEBORN-408] Add lost worker infos to http request (#1333) 2023-03-13 15:27:41 +08:00
Ethan Feng
2385215578
[CELEBORN-394] Refine memory manager's log. (#1332) 2023-03-13 15:13:33 +08:00
Angerszhuuuu
b56624d3c1
[CELEBORN-405] Add metrics about lost workers (#1330)
* [CELEBORN-405] Add metrics about lost workers
2023-03-13 14:49:49 +08:00
Ethan Feng
c78023824a
[CELEBORN-397][FLINK] Flink plugin support UnpooledByteBufAllocator. (#1324) 2023-03-13 11:36:13 +08:00
Ethan Feng
bb8401e401
[CELEBORN-403][FLINK] Add metrics about buffer dispatcher request queue length. (#1329) 2023-03-13 11:15:00 +08:00
Kaijie Chen
c6eced69fc
[CELEBORN-402] Enable autolink to Jira (#1328) 2023-03-13 09:56:05 +08:00
If Wind
5476f9ca95
[CELEBORN-401] Modify prometheus-podmonitor.yaml to collect metrics correctly (#1327) 2023-03-11 20:35:35 +08:00
Angerszhuuuu
a336f12cc8
[CELEBORN-400] Add RPC metrics for OpenStream (#1326) 2023-03-10 21:22:05 +08:00
jiaoqingbo
92ad56c47d
[CELEBORN-393] responseBuilder.setCmdType should be called only once in MetaHandler's handleReadRequest method (#1322) 2023-03-10 21:17:22 +08:00
Angerszhuuuu
4b334df7a6
[CELEBORN-399] Make fileSorterExecutors thread num can be customized (#1325) 2023-03-10 21:10:43 +08:00
Ethan Feng
2d4a4f25bd
[CELEBORN-389][FLINK] Fix remove transportClient from readClientHandler caused NPE (#1323) 2023-03-10 14:43:25 +08:00
Shuang
ec745e36d1
[CELEBORN-391][Flink] Refine register/release synchronization (#1321) 2023-03-09 20:00:50 +08:00
Keyong Zhou
21bdfdb21b
[CELEBORN-390][FLINK] Refine synchronization in FlinkShuffleClientImpl#updateFileGroup (#1320) 2023-03-09 16:49:18 +08:00
Ethan Feng
8e167c6488
[CELEBORN-387][FLINK] Remove unnecessary limitZeroInFlight from sendMessageInternal. (#1319) 2023-03-09 12:31:09 +08:00
Ethan Feng
aebb870d08
[CELEBORN-386][FLINK] Async open DataPartitionReader to release Netty thread earlier. (#1318) 2023-03-09 12:31:01 +08:00
Ethan Feng
393fabc0c3
[CELEBORN-385] Add rolling file in log4j configuration template (#1317) 2023-03-09 12:30:40 +08:00
Fu Chen
7d43e4dd3e
[CELEBORN-384] Fix master-statefulset.yaml syntax error (#1316) 2023-03-08 19:17:08 +08:00
Ethan Feng
9fc77980ba
[CELEBORN-380][FLINK] Enable flink integration test in GitHub CI. (#1312) 2023-03-08 10:52:14 +08:00
jiaoqingbo
84795bc63b
[CELEBORN-382] Call checkDiskFullAndSplit in the handlePushData method to avoid repeated definitions (#1313) 2023-03-07 18:55:46 +08:00
Keyong Zhou
fd1ac2f711
[CELEBORN-379][FLINK] Fix checkState in TransportFrameDecoderWithBufferSupplier#decodeBodyCopyOut (#1311) 2023-03-06 21:49:18 +08:00
Cheng Pan
fd7d8361f6
[CELEBORN-377][BUILD] build/mvn should always respect maven.version defined in pom.xml (#1310) 2023-03-06 15:14:18 +08:00
Angerszhuuuu
e805c74aad
[CELEBORN-360] Export necessary env in load-celeborn-env.sh (#1308) 2023-03-03 21:09:11 +08:00
zhongqiangchen
94361cd3ed
[CELEBORN-367][FOLLOWUP] [FLINK] fix conflicts in RemoteShuffleResultPartitionS… (#1306) 2023-03-03 17:58:34 +08:00
Keyong Zhou
9608a11819
[CELEBORN-348][FOLLOWUP] Refine comparator to support nanoseconds whi… (#1305) 2023-03-03 17:55:59 +08:00
Cheng Pan
88976d9fd9
[CELEBORN-375][INFRA] Enable CI on branch-* (#1307) 2023-03-03 16:49:32 +08:00
zhongqiangchen
9641f94255
[CELEBORN-369] [FLINK] Add ut for RemoteShuffleResultPartition (#1297) 2023-03-03 15:46:43 +08:00
Ethan Feng
675a7da393
[CELEBORN-368][FLINK] Pass exceptions in buffer stream. (#1304) 2023-03-03 15:43:30 +08:00
Keyong Zhou
f2fd8a5c15
[CELEBORN-373] Add sorted files into grafana dashboard (#1303) 2023-03-02 23:41:16 +08:00
Keyong Zhou
9aabb43699
[CELEBORN-372] Remove the standard Apache License header from the top of third-party source files (#1301) 2023-03-02 19:07:01 +08:00
zhongqiangchen
9dc1bc2b1c
[CELEBORN-367] [FLINK] Move pushdata functions used by mappartition from ShuffleClientImpl to FlinkShuffleClientImpl (#1295) 2023-03-02 18:50:38 +08:00
Keyong Zhou
dcedf7b0a9
[CELEBORN-348] Support fetchTime in load-aware slots assignment strategy (#1287) 2023-03-02 18:31:50 +08:00
Angerszhuuuu
734f14991a
[CELEBORN-360] Add celeborn ratis shell command line (#1294)
* [CELEBORN-360] Add celeborn ratis shell command line
2023-03-02 16:30:45 +08:00
Angerszhuuuu
ca79a9ce31
[CELEBORN-359][FOLLOWUP] Update LICENSE and NOTICE for ratis-shell (#1298)
* [CELEBORN-359][FOLLOWUP] Update LICENSE and NOTICE for ratis-shell
2023-03-02 15:48:23 +08:00
Angerszhuuuu
786fcd6744
[CELEBORN-336] Revive Failed should use keep the corresponding StatusCode (#1283)
* [CELEBORN-336] Revive Failed should use keep the corresponding StatusCode
2023-03-01 18:57:51 +08:00
Angerszhuuuu
4c90e0b02a
[CELEBORN-359] Add ratis-shell to celeborn (#1292) 2023-03-01 17:04:57 +08:00
Ethan Feng
328a6ff2f5
Revert "[CELEBORN-355][BUILD] Create shaded module for Celeborn common (#1290)" (#1293)
This reverts commit 725028a10a.
2023-03-01 16:59:02 +08:00
Shuang
bc7da3154f
[CELEBORN-354][Flink] fix succeedPartitionIds may contain new added partitionIds (#1289) 2023-03-01 15:45:24 +08:00
Kerwin Zhang
725028a10a
[CELEBORN-355][BUILD] Create shaded module for Celeborn common (#1290) 2023-03-01 15:29:45 +08:00
Ethan Feng
d4af8fd094
[CELEBORN-353][FLINK] Fix incorrect read buffer metric. (#1288) 2023-03-01 11:08:13 +08:00
zhongqiangchen
cb76c4de4c
[CELEBORN-350][FLINK] Add PluginConf to be compatible with old configurations 2023-02-28 20:36:11 +08:00
jiaoqingbo
7dc1ab13db
[CELEBORN-351] Add \n to the log to make log print clearer (#1285) 2023-02-28 17:55:17 +08:00
Shuang
5654c62f35
[CELEBORN-347][Flink] fix memory leak and refactor BufferStreamManager (#1282) 2023-02-28 15:18:59 +08:00
Angerszhuuuu
eda21ead24
[CELEBORN-344] Change PUSH_DATA_FAIL_MASTER/SALVE to PUSH_DATA_WRITE_FAIL_MASTER/SALVE (#1281) 2023-02-28 11:29:40 +08:00
Keyong Zhou
7adf1fca41
[CELEBORN-295] Optimize data push (#1232)
* [CELEBORN-295] Add double buffer for sort pusher
2023-02-28 10:35:55 +08:00
Angerszhuuuu
24f5478adc
[CELEBORN-338] Clean duplicated exception message of handling push data (#1274) 2023-02-28 10:35:18 +08:00