Commit Graph

62 Commits

Author SHA1 Message Date
Cheng Pan
08f647ea3e
Remove unexpected log4j configuration file from package jar and minor improvement (#331) 2022-08-13 11:10:48 +08:00
Ethan Feng
f3bcb7f6a8
[ISSUE-146]update slots distribution mechanism (#273) 2022-08-12 23:38:19 +08:00
Keyong Zhou
46cbe4fb04
[ISSUE-288] fix netty memory leak(#310) 2022-08-12 00:49:42 +08:00
Keyong Zhou
d19b475500
[ISSUE-325] Log error msg for ChunkFetchFailure (#326) 2022-08-12 00:18:52 +08:00
Cheng Pan
d01ee81ee6
Bump Ratis 2.3.0 and related toolchains (#299) 2022-08-04 21:59:42 +08:00
AngersZhuuuu
9324b1e89a
[ISSUE-257][FEATURE] Reserve slots support customized retry times (#258) 2022-07-26 15:23:25 +08:00
zky.zhoukeyong
457f5874a2 Delete System.out.println 2022-07-25 20:03:21 +08:00
AngersZhuuuu
fe17914942
Refactor pom import issue (#277) 2022-07-25 17:49:55 +08:00
Keyong Zhou
e11af5d948
Support passed-in buffer supplier for FrameDecoder (#278) 2022-07-25 16:46:29 +08:00
Keyong Zhou
ebadb13070
[ISSUE-269] Remove unused inceptor in TransportFrameDecoder (#270) 2022-07-17 17:15:35 +08:00
Keyong Zhou
6442f38a33
[ISSUE-267] Extend API to support more partition types: MapPartition,… (#268) 2022-07-17 16:28:37 +08:00
Keyong Zhou
56a0b9072b
[ISSUE-261] Refine message class hierarchy (#266) 2022-07-16 17:00:09 +08:00
Keyong Zhou
382170ad84
[ISSUE-264] Refine network code (#265) 2022-07-16 16:02:35 +08:00
Keyong Zhou
7da8f64691
[ISSUE-262] Remove unused bootstrap (#263) 2022-07-16 11:01:44 +08:00
AngersZhuuuu
e87cc7b648
[ISSUE-231][FOLLOWUP] Remove unused FILE_MODULE (#255) 2022-07-12 16:18:10 +08:00
AngersZhuuuu
36cc234dd4
[ISSUE-246][REFACTOR] Refactor LifecycleManager to make it's code more clear and more readable (#252) 2022-07-12 15:37:49 +08:00
dxheming
386dfc9f2c
[Issue 253] Fix netty memory leak in TransportResponseHandler (#254) 2022-07-12 14:38:24 +08:00
Keyong Zhou
691beb7889
[ISSUE-247] Extract PushHandler, FetchHandler, RpcHandler from Worker… (#251) 2022-07-12 11:40:42 +08:00
AngersZhuuuu
d5548e6be4
Update ControlMessages.scala (#248) 2022-07-10 12:19:19 +08:00
AngersZhuuuu
f80c86a675
[ISSUE-222] Destroy and DestroyResponse should remove null check (#238) 2022-07-09 15:44:17 +08:00
Keyong Zhou
1b12a2e883
[ISSUE-239] Move prcessFetchChunks into ChunkFetchHandler (#243) 2022-07-09 14:21:18 +08:00
AngersZhuuuu
49caced462
[ISSUE-222][BUG] GetReduceFileGroups should remove code about return null value (#236) 2022-07-09 12:14:08 +08:00
AngersZhuuuu
c28eeb078c
[ISSUE-222] CommitFiles and CommitFilesResponse should remove null check (#237) 2022-07-08 22:32:54 +08:00
AngersZhuuuu
6e5c282229
[ISSUE-222] GetBlacklist/GetBlacklistResponse should replace null value with empty list (#235) 2022-07-08 14:49:09 +08:00
Keyong Zhou
f0b0030898
[ISSUE-216] Support sized chunk fetch request (#218) 2022-07-08 14:10:08 +08:00
AngersZhuuuu
d2a0ad480e
[ISSUE-222][BUG] RequestSlotResponse/RegisterShuffleResponse should handle null issue (#226) 2022-07-08 12:33:40 +08:00
Ethan Feng
04148fef2b
[ISSUE-228]Fix unexpected closed exceptions occurred while committing files. (#232) 2022-07-07 22:15:16 +08:00
AngersZhuuuu
9e26fb9f87
[ISSUE-228][BUG] When CommitFile, worker endpoint may be closed by ReserveSlots failed, we should try reconnect endpoint when endpoint not active (#231) 2022-07-07 22:14:48 +08:00
Keyong Zhou
892acc51d9
[ISSUE-213] Cleanup unused code in RPC module (#214) 2022-07-02 13:16:26 +08:00
nafiy
6f8fb8747f
Modify argument class and add config (#212) 2022-07-01 23:17:24 +08:00
Keyong Zhou
ff823a9c03
Cleanup unused code in network (#206)
* Refine doc
* Cleanup unused code in network
2022-07-01 14:39:17 +08:00
AngersZhuuuu
909e8b2f53
[ISSUE-190][BUG] After WorkerLost, response to worker heartbeat RPC to, then worker can clean the data. (#192) 2022-06-29 22:25:29 +08:00
Ethan Feng
9e7e38dcbd
[Feature] Multithread disk flusher (#182) 2022-06-29 15:03:02 +08:00
AngersZhuuuu
5f8521e880
[ISSUE-180][FEATURE] Disable rss.worker.closeIdleConnections by default (#181) 2022-06-28 19:12:09 +08:00
AngersZhuuuu
40c89d28d9
Update NettyRpcEnv.scala (#179) 2022-06-28 18:28:40 +08:00
AngersZhuuuu
5c82b763eb
[ISSUE-169][FEATURE] Make app heartbeat interval can be customized (#170)
* [ISSUE-169][FEATURE] Make app heartbeat interval can be customized

* Update LifecycleManager.scala
2022-06-27 20:58:00 +08:00
dxheming
566ed18088
fix worker pushserverport replicateserverport conflict (#165) 2022-06-27 15:09:39 +08:00
mingji
d4d8eb3838 update pom version. 2022-06-24 14:28:42 +08:00
Ethan Feng
124c474c50
[BUG]Fix worker lost in HA mode when leader lost. (#155) 2022-06-17 22:07:42 +08:00
Ethan Feng
6811cc22fc
[issue-146] Add storage hint to indicate storage location. (#147) 2022-06-14 15:57:11 +08:00
Keyong Zhou
49f1ee6088
Reuse SendBuffer among tasks in Executor (#131) 2022-06-03 11:26:30 +08:00
Ethan Feng
1113f437c6
[FEATURE] Remove dependency on spark-tags from common module (#126) (#128) 2022-05-31 15:24:08 +08:00
nafiy
491f89bbb5
[FEATURE]Add metrics source for JVM and CPU (#125)
* Add metrics source for JVM and CPU

* Fix scala style issue
2022-05-30 13:26:54 +08:00
Ethan Feng
86adc0d244
[Feature]Add metrics documentation and grafana dashboard. (#117) 2022-05-20 12:12:41 +08:00
Ethan Feng
3019e2712b
[bug]fix parameter position error. (#96) 2022-04-15 16:41:21 +08:00
Ethan Feng
409da82964
[Bug]fix stuck under high memory pressure. (#90) 2022-04-14 18:53:39 +08:00
Ethan Feng
baa2836216
Add metrics: (#85)
1.shuffle fetch send data time.
 2.open stream time.
 3.memory critical count.
2022-04-02 15:05:27 +08:00
Ethan Feng
9ad8254b0a
AQE support. (#67) 2022-04-01 20:19:01 +08:00
AngersZhuuuu
86bbeea9b4
[BUG] Register shuffle with configurable retry times and retry wait time (#83) 2022-04-01 16:59:37 +08:00
AngersZhuuuu
eacb9a1217
Refactor the configuration (#72) 2022-03-11 12:03:43 +08:00