Commit Graph

125 Commits

Author SHA1 Message Date
Cheng Pan
08f647ea3e
Remove unexpected log4j configuration file from package jar and minor improvement (#331) 2022-08-13 11:10:48 +08:00
AngersZhuuuu
221b068773
[ISSUE-327][REFACTOR] Remove unused logAvailableFlushBuffersTask (#328) 2022-08-13 11:09:59 +08:00
Ethan Feng
f3bcb7f6a8
[ISSUE-146]update slots distribution mechanism (#273) 2022-08-12 23:38:19 +08:00
Keyong Zhou
d166e042be
[ISSUE-329] Should not sleep if reserve slots successfully in reserveSlotsWithRetry (#330) 2022-08-12 12:27:27 +08:00
Keyong Zhou
46cbe4fb04
[ISSUE-288] fix netty memory leak(#310) 2022-08-12 00:49:42 +08:00
Keyong Zhou
d19b475500
[ISSUE-325] Log error msg for ChunkFetchFailure (#326) 2022-08-12 00:18:52 +08:00
Binjie Yang
2137e84ab7
add rss version (#316) 2022-08-10 21:03:46 +08:00
Binjie Yang
8ececd60a6
fix (#314) 2022-08-10 16:37:43 +08:00
Fan Yilun
7a858bd70b
[DOC] Add doc and helm chart to describe how to deploy on k8s (#309) 2022-08-09 21:15:15 +08:00
nafiy
50c0399940
[ISSUE-308][DOC] Fix the wrong configuration doc of rss.ha.port and provide example in readme. (#311) 2022-08-08 17:32:47 +08:00
nafiy
eeda030599
Add metrics for marking active master (#307) 2022-08-07 18:00:49 +08:00
AngersZhuuuu
a30fb5b3b0
[ISSUE-297] NPE in TaskCompletionListener due to Spark OOM in ShuffleInMemorySorter causing tasks to hang (#298) 2022-08-04 22:47:04 +08:00
Cheng Pan
d01ee81ee6
Bump Ratis 2.3.0 and related toolchains (#299) 2022-08-04 21:59:42 +08:00
Cheng Pan
efdf283918
ShuffleBlockInfo should be static inner class (#301) 2022-08-04 21:50:58 +08:00
Cheng Pan
ce147a45e1
Bump maven 3.6.3 (#300) 2022-08-04 21:05:21 +08:00
AngersZhuuuu
cf2b895afb
[ISSUE-293][REFACTOR] Init worker rpc endpoint and reserve slot in parallel to speed up register shuffle process (#294)
[ISSUE-293][REFACTOR] Init worker rpc endpoint and reserve slot in parallel to speed up register shuffle process (#294)
2022-08-03 20:00:30 +08:00
Ethan Feng
594feab279
fix unexpected index update. (#295) 2022-08-02 18:16:26 +08:00
AngersZhuuuu
e57ad27887
[ISSUE-291][REFACTOR] When worker endpoint initializing failed, print clear warning log (#292) 2022-08-02 12:03:59 +08:00
zky.zhoukeyong
67ac3c029c [DOC] Fix typo 2022-08-01 22:15:09 +08:00
AngersZhuuuu
c3cb2b9fc0
[ISSUE-289][BUG] Write data should use default object class to avoid NPE (#290)
* [ISSUE-289][BUG] Write data should use default object class to avoid NPE
2022-08-01 16:02:29 +08:00
dxheming
8e3f48ec12
Refactor deprecated netty ConcurrentSet (#285) 2022-07-27 20:35:46 +08:00
Ethan Feng
82f0475d9b
[BUG] Fix reserve slots failed due to take buffer stuck (#283)
* [BUG] Fix reserve slots failed due to take buffer stuck
2022-07-26 18:20:39 +08:00
AngersZhuuuu
7a760466aa
[ISSUE-281][BUG] Use correct maxDestLength to check if buffer can satisfy compress result (#282) 2022-07-26 15:56:05 +08:00
AngersZhuuuu
9324b1e89a
[ISSUE-257][FEATURE] Reserve slots support customized retry times (#258) 2022-07-26 15:23:25 +08:00
zky.zhoukeyong
457f5874a2 Delete System.out.println 2022-07-25 20:03:21 +08:00
AngersZhuuuu
fe17914942
Refactor pom import issue (#277) 2022-07-25 17:49:55 +08:00
Keyong Zhou
e11af5d948
Support passed-in buffer supplier for FrameDecoder (#278) 2022-07-25 16:46:29 +08:00
Ethan Feng
cb42b2fa5c
[BUG] multi-thread flusher causes data inconsistent with chunk offsets (#275) 2022-07-23 11:17:38 +08:00
Keyong Zhou
ebadb13070
[ISSUE-269] Remove unused inceptor in TransportFrameDecoder (#270) 2022-07-17 17:15:35 +08:00
Keyong Zhou
6442f38a33
[ISSUE-267] Extend API to support more partition types: MapPartition,… (#268) 2022-07-17 16:28:37 +08:00
Keyong Zhou
56a0b9072b
[ISSUE-261] Refine message class hierarchy (#266) 2022-07-16 17:00:09 +08:00
Keyong Zhou
382170ad84
[ISSUE-264] Refine network code (#265) 2022-07-16 16:02:35 +08:00
Keyong Zhou
7da8f64691
[ISSUE-262] Remove unused bootstrap (#263) 2022-07-16 11:01:44 +08:00
haiming
7ae06aa7cc
[ISSUE-259] Build RSS on M1 Macbook Pro / Air (#260) 2022-07-14 15:35:29 +08:00
AngersZhuuuu
e87cc7b648
[ISSUE-231][FOLLOWUP] Remove unused FILE_MODULE (#255) 2022-07-12 16:18:10 +08:00
AngersZhuuuu
36cc234dd4
[ISSUE-246][REFACTOR] Refactor LifecycleManager to make it's code more clear and more readable (#252) 2022-07-12 15:37:49 +08:00
dxheming
386dfc9f2c
[Issue 253] Fix netty memory leak in TransportResponseHandler (#254) 2022-07-12 14:38:24 +08:00
Keyong Zhou
691beb7889
[ISSUE-247] Extract PushHandler, FetchHandler, RpcHandler from Worker… (#251) 2022-07-12 11:40:42 +08:00
Keyong Zhou
d8c5758124
[ISSUE-249] Fix OutOfBounds when shuffle has no data(q24b) (#250) 2022-07-10 18:03:54 +08:00
AngersZhuuuu
d5548e6be4
Update ControlMessages.scala (#248) 2022-07-10 12:19:19 +08:00
AngersZhuuuu
f80c86a675
[ISSUE-222] Destroy and DestroyResponse should remove null check (#238) 2022-07-09 15:44:17 +08:00
Keyong Zhou
1b12a2e883
[ISSUE-239] Move prcessFetchChunks into ChunkFetchHandler (#243) 2022-07-09 14:21:18 +08:00
AngersZhuuuu
49caced462
[ISSUE-222][BUG] GetReduceFileGroups should remove code about return null value (#236) 2022-07-09 12:14:08 +08:00
Keyong Zhou
07d7f99964
[ISSUE-233] Remove unused FlushBuffer and MinimalByteBuf (#242) 2022-07-09 12:09:10 +08:00
AngersZhuuuu
c28eeb078c
[ISSUE-222] CommitFiles and CommitFilesResponse should remove null check (#237) 2022-07-08 22:32:54 +08:00
Keyong Zhou
6ce0e42300
[ISSUE-240] Revert pom modification about profiles (#241) 2022-07-08 22:02:08 +08:00
mingji
e43720f6a1 Add refactor requests template. 2022-07-08 17:12:16 +08:00
AngersZhuuuu
6e5c282229
[ISSUE-222] GetBlacklist/GetBlacklistResponse should replace null value with empty list (#235) 2022-07-08 14:49:09 +08:00
Keyong Zhou
f0b0030898
[ISSUE-216] Support sized chunk fetch request (#218) 2022-07-08 14:10:08 +08:00
AngersZhuuuu
d2a0ad480e
[ISSUE-222][BUG] RequestSlotResponse/RegisterShuffleResponse should handle null issue (#226) 2022-07-08 12:33:40 +08:00