Angerszhuuuu
e61130d397
[CELEBORN-423][FOLLOWUP] Format http request ( #1353 )
...
* [CELEBORN-423][FOLLOWUP] Format http request
2023-03-15 16:30:23 +08:00
Angerszhuuuu
1f56a5e5d1
[CELEBORN-423] Format http request result ( #1349 )
2023-03-15 10:32:01 +08:00
Angerszhuuuu
3907d70212
[CELEBORN-421] Add shutdown and registered to http request ( #1346 )
...
* [CELEBORN-421] Add shutdown and registered to http request
2023-03-14 18:23:21 +08:00
Angerszhuuuu
7d7279a9bc
[CELEBORN-420] Add unavailablePeers to http request ( #1345 )
...
* [CELEBORN-420] Add unavailablePeers to http request
2023-03-14 17:23:45 +08:00
Angerszhuuuu
3600ccc4e3
[CELEBORN-409] Add PartitionLocationInfo to worker's http request ( #1335 )
2023-03-13 17:02:28 +08:00
Angerszhuuuu
6f1ab70403
[CELEBORN-406] Add blacklist to http request to indicate blacklisted worker ( #1334 )
2023-03-13 16:44:46 +08:00
Angerszhuuuu
144a8cdb3f
[CELEBORN-408] Add lost worker infos to http request ( #1333 )
2023-03-13 15:27:41 +08:00
Ethan Feng
bb8401e401
[CELEBORN-403][FLINK] Add metrics about buffer dispatcher request queue length. ( #1329 )
2023-03-13 11:15:00 +08:00
Angerszhuuuu
a336f12cc8
[CELEBORN-400] Add RPC metrics for OpenStream ( #1326 )
2023-03-10 21:22:05 +08:00
Angerszhuuuu
4b334df7a6
[CELEBORN-399] Make fileSorterExecutors thread num can be customized ( #1325 )
2023-03-10 21:10:43 +08:00
jiaoqingbo
84795bc63b
[CELEBORN-382] Call checkDiskFullAndSplit in the handlePushData method to avoid repeated definitions ( #1313 )
2023-03-07 18:55:46 +08:00
Ethan Feng
675a7da393
[CELEBORN-368][FLINK] Pass exceptions in buffer stream. ( #1304 )
2023-03-03 15:43:30 +08:00
Keyong Zhou
dcedf7b0a9
[CELEBORN-348] Support fetchTime in load-aware slots assignment strategy ( #1287 )
2023-03-02 18:31:50 +08:00
Angerszhuuuu
eda21ead24
[CELEBORN-344] Change PUSH_DATA_FAIL_MASTER/SALVE to PUSH_DATA_WRITE_FAIL_MASTER/SALVE ( #1281 )
2023-02-28 11:29:40 +08:00
Keyong Zhou
7adf1fca41
[CELEBORN-295] Optimize data push ( #1232 )
...
* [CELEBORN-295] Add double buffer for sort pusher
2023-02-28 10:35:55 +08:00
Angerszhuuuu
24f5478adc
[CELEBORN-338] Clean duplicated exception message of handling push data ( #1274 )
2023-02-28 10:35:18 +08:00
Rex(Hui) An
798ff90bb7
[CELEBORN-342] Fix the wrong avg produce bytes in Congestion control ( #1279 )
2023-02-27 16:29:37 +08:00
Keyong Zhou
3c8c58e09d
[CELEBORN-301] Refactor PartitionLocationInfo to use ConcurrentHashMap ( #1278 )
2023-02-26 16:46:30 +08:00
Angerszhuuuu
a7587c3fe7
[CELEBORN-337] Remove unnecessary StatusCode.message ( #1272 )
...
* [CELEBORN-337] Remove unnecessary StatusCode.message
2023-02-24 15:11:07 +08:00
Shuang
9754616d79
[CELEBORN-330] fix deadlock when use the same netty channel to receive data while other thread wait the response ( #1265 )
2023-02-23 17:57:43 +08:00
Angerszhuuuu
fc8540a2e6
[CELEBORN-325] After worker restart, throw NPE when receive not found partition ( #1259 )
...
* [CELEBORN-325] After worker restart, throw NPE when receive not found partition
2023-02-22 15:19:34 +08:00
Ethan Feng
0df08fbdf3
[CELEBORN-320][FLINK] fix handle wrong message type in FetchHandler. ( #1254 )
2023-02-21 11:51:01 +08:00
Ethan Feng
26a3bb5e72
[CELEBORN-308] Fix flusher will exit unexpectedly if flush task write failed. ( #1249 )
2023-02-20 21:45:37 +08:00
Ethan Feng
0c8bb83114
[CELEBORN-234] Implement buffer stream. ( #1221 )
2023-02-17 17:38:36 +08:00
zhongqiangchen
5236df68af
[CELEBORN-292] optimize mappartitionfilewriter flushing index and reading data header ( #1225 )
2023-02-17 13:42:28 +08:00
zhongqiangchen
79096d60d0
[CELEBORN-293] WorkerSource registers timer for mappartition message metrics ( #1226 )
2023-02-17 11:29:54 +08:00
Ethan Feng
1dcfdb0c8f
[CELEBORN-281] Add metrics about buffer stream read buffer. ( #1216 )
2023-02-17 11:20:07 +08:00
Angerszhuuuu
57f775a7e9
[CELEBORN-273] Move push data timeout checker into TransportResponseHandler to keep callback status consistence ( #1208 )
2023-02-16 18:27:37 +08:00
Ethan Feng
534853bf8a
[CELEBORN-278] Add openStreamWithCredit RPC. ( #1214 )
2023-02-16 14:07:13 +08:00
zhongqiangchen
2c508dae0f
[CELEBORN-307] fix ArrayComparisonFailure while running lz4 ut ( #1241 )
2023-02-16 13:41:17 +08:00
Rex(Hui) An
2068e6ae37
[CELEBORN-279] Add user level push data speed metric ( #1213 )
2023-02-13 12:04:44 +08:00
Rex(Hui) An
adb6592d31
[CELEBORN-277] PushDataHandle callback could miss soft split status ( #1212 )
2023-02-09 14:57:18 +08:00
Rex(Hui) An
f88f5fcf55
[CELEBORN-207][FOLLOW_UP] Master could miss the congestion status if enable push.data.replicate
2023-02-07 22:57:39 +08:00
Rex(Hui) An
cfe81969c9
[CELEBORN-275] WrappedCallback should only handle response from replica ( #1209 )
2023-02-07 18:18:13 +08:00
Rex(Hui) An
bb113ec9be
[CELEBORN-207] Support network congestion control ( #1066 )
2023-02-07 12:06:18 +08:00
Angerszhuuuu
c4020100db
[CELEBORN-271][BUG] PushState in PushDataHandler should should use peer's location
2023-02-06 11:31:57 +08:00
Angerszhuuuu
ecc3a0e52f
[CELEBORN-272][BUG] Don't do replication should directly use callback not wrappedCallback ( #1205 )
2023-02-06 11:28:12 +08:00
zhongqiangchen
8e903840af
[CELEBORN-243][REWORK]fix bug that os's disk usage is low but celeborn thinks that it's high_disk_usage ( #1202 )
2023-02-04 14:27:44 +08:00
Angerszhuuuu
2e68912812
[CELEBORN-269][BUG] Disable replication throw NPE when removeBatch in pushDataHandler ( #1203 )
2023-02-03 20:06:59 +08:00
Shuang
2634476758
[CELEBORN-267] reuse stream when client channel reconnected ( #1200 )
2023-02-03 15:12:45 +08:00
Angerszhuuuu
4b6f7e4593
[CELEBORN-239][IMPROVEMENT] Worker replicate should enable push data timeout too ( #1185 )
2023-02-03 11:53:15 +08:00
zhongqiangczq
ff17a61ec5
[CELEBORN-243] fix bug that os's disk usage is low but celeborn thinks that it's high_disk_usage ( #1184 )
2023-02-02 10:41:11 +08:00
Shuang
7162be2fae
[CELEBORN-201] Separate partitionLocationInfo in LifecycleManager and worker ( #1149 )
2023-01-31 18:53:36 +08:00
Angerszhuuuu
1311fb53d1
[CELEBORN-243][CELEBORN-245][IMPROVEMENT] Create push client failed and connection failed cause push failed should have their own ERROR type ( #1181 )
...
* [CELEBORN-243][IMPROVEMENT] Create push client failed should have a ERROR type
2023-01-30 17:47:22 +08:00
Angerszhuuuu
8611a64400
[CELEBORN-237][IMPROVEMENT] push failed error message should show partition info ( #1178 )
...
* [CELEBORN-237][IMPROVEMENT] push failed error message should show partition info
2023-01-28 18:41:54 +08:00
Ethan Feng
a239f9f284
[CELEBORN-228]Refactor PartitionFileSorter to avoid specific JDK dependency. ( #1168 )
2023-01-16 20:06:47 +08:00
zy.jordan
bb96700415
[CELEBORN-223] The default rpc thread num of pushServer/replicateServer/fetchServer should be the number of total of Flusher's thread ( #1163 )
2023-01-16 12:03:46 +08:00
zhongqiangczq
3661222d98
[CELEBORN-195] add implementation to MapPartitionFileWriter ( #1141 )
2023-01-13 16:41:11 +08:00
zy.jordan
19197b9190
[CELEBORN-214] Push/Replicate/Fetch io threads default value is 16 ( #1158 )
2023-01-10 17:46:56 +08:00
nafiy
9635725480
[CELEBORN-204][IMPROVEMENT]Collect disk usage metrics in byte unit by default ( #1153 )
2023-01-09 17:36:18 +08:00
Ethan Feng
5595f2f4b3
[CELEBORN-124]Add buffer stream. ( #1069 )
2023-01-06 15:54:52 +08:00
Shuang
3b2be25a50
[CELEBORN-173] refactor minicluster and fix ut ( #1147 )
2023-01-05 20:39:19 +08:00
Angerszhuuuu
5edb21d210
[CELEBORN-168][FOLLOWUP] Device metrics should use long value and add size unit in metric name ( #1143 )
...
* [CELEBORN-168][FOLLOWUP] Device metrics should use long value and add size unit in metric name
2023-01-05 11:45:19 +08:00
nafiy
3e80cf2b87
[CELEBORN-168][FEATURE] Add disk usage related metrics for Worker ( #1127 )
2023-01-05 10:35:51 +08:00
Angerszhuuuu
425e31797c
[CELEBORN-182][BUG] StorageManager should not delete shuffle file when enable graceful shutdown ( #1126 )
2022-12-30 18:13:36 +08:00
Angerszhuuuu
7d7192af14
[CELEBORN-179][BUG] Repeat remove expired shuffle throw NPE ( #1124 )
2022-12-29 15:47:05 +08:00
Angerszhuuuu
6411fe71b1
[CELEBORN-178][BUG] Default registered flag should be false, not null ( #1123 )
2022-12-29 15:24:09 +08:00
nafiy
77cb7a0477
[CELEBORN-169][REFACTOR] Extract ObservedDevice out from LocalDeviceMonitor ( #1113 )
...
* [CELEBORN-169][REFACTOR] Extract ObservedDevice out from LocalDeviceMonitor
2022-12-28 14:29:00 +08:00
Ethan Feng
5aa959a335
[CELEBORN-157] Change prefix of configurations to celeborn. ( #1104 )
2022-12-21 15:17:28 +08:00
nafiy
f13dfb7421
[CELEBORN-113][FEATURE] Add metrics to monitor non-critical error number on local device ( #1100 )
2022-12-20 22:30:55 +08:00
Keyong Zhou
2f0682265e
[CELEBORN-119] Add timeout for pushdata ( #1097 )
2022-12-20 20:40:42 +08:00
zky.zhoukeyong
3a4fd04f33
Revert "[CELEBORN-120] MapPartitionFileWriter completes some functions to support mappartition ( #1062 )"
...
This reverts commit aa5a4d8c41
2022-12-17 11:10:54 +08:00
Ethan Feng
39394526a8
[CELEBORN-142]Keep committed partition locations semantic consistent when commit files on HDFS. ( #1091 )
2022-12-16 19:02:02 +08:00
zhongqiangczq
aa5a4d8c41
[CELEBORN-120] MapPartitionFileWriter completes some functions to support mappartition ( #1062 )
2022-12-16 18:40:22 +08:00
nafiy
c931663e5f
[CELEBORN-110][REFACTOR] Notify critical error after collecting a certain number of non-critical error ( #1055 )
2022-12-16 15:47:36 +08:00
Ethan Feng
65cb36c002
[CELEBORN-83][FOLLOWUP] Fix various bugs when using HDFS as storage. ( #1065 )
2022-12-15 15:20:29 +08:00
Cheng Pan
ec371c0026
[CELEBORN-132] ShuffleClient should not implement Cloneable ( #1077 )
2022-12-14 10:04:39 +08:00
Keyong Zhou
5a3d397781
[CELEBORN-130] Correct CommitFilesTime metric ( #1073 )
2022-12-13 20:02:21 +08:00
zhongqiangczq
97991a3404
[CELEBORN-126] Fileinfo adds member bufferSize ( #1068 )
2022-12-13 16:36:26 +08:00
zhongqiangczq
edf85de8f6
[CELEBORN-123] PushDataHandler handleRpcRequestCore fix bug about val isMaster ( #1063 )
2022-12-12 15:51:58 +08:00
zhongqiangczq
c7258cfc03
[CELEBORN-103] add handleMapPartitionPushData to support mappartition ( #1048 )
2022-12-08 11:22:43 +08:00
zhongqiangczq
ea1c630173
[CELEBORN-80] FileWriter supports MapPartition ( #1025 )
2022-12-08 10:46:26 +08:00
Ethan Feng
acfaf59ab3
[CELEBORN-91] Refactor memory tracker to support read buffer. ( #1038 )
...
* [CELEBORN-91] Refactor memory tracker to support read buffer.
2022-12-05 15:38:43 +08:00
zhongqiangczq
b262591da8
[CELEBORN-71] pushdatahandler supports mappartition write: handshake/regionstart/regionfinish ( #1013 )
2022-12-05 13:05:35 +08:00
Binjie Yang
d6ee3c18bc
[CELEBORN-98][IMPROVEMENT] Remove unreachable code block in master/work arguments ( #1042 )
2022-12-02 22:53:28 +08:00
Angerszhuuuu
fc5ca42c14
[CELEBORN-96][REFACTOR] PushMergedData return partition not found use same code path ( #1039 )
2022-12-02 14:09:00 +08:00
Ethan Feng
dd02070e4b
[CELEBORN-83] Fix various bug when using HDFS as storage.
...
1. fix incompatibility between Hadoop 2 and Hadoop 3.
2. fix hdfs writer will never be called when there are no healthy disks.
3. fix an NPE when HDFS file writer close.
2022-11-30 19:33:18 +08:00
Ethan Feng
02e446284d
[CELEBORN-74] Device monitor should respect storage dir configured usable space ( #1023 )
2022-11-29 17:10:18 +08:00
Angerszhuuuu
d26e73209b
[CELEBORN-76] Support batch commit hard split partition before stage end
2022-11-29 13:09:01 +08:00
Angerszhuuuu
c8e5315b9c
[CELEBORN-23][FOLLOWUP] Both master and slave data should return HARD_SPLIT during shutdown ( #1018 )
2022-11-28 22:05:07 +08:00
Keyong Zhou
61e04b77fd
[CELEBORN-70][FOLLOWUP] Add epoch for each commitFiles request. ( #1015 )
...
* [CELEBORN-70][FOLLOWUP] Add epoch for each commitFiles request. Address comments.
2022-11-28 14:08:20 +08:00
Ethan Feng
cfa9b7f700
[CELEBORN-18] Refactor stream manager to distinguish map partition and reduce partition. ( #997 )
2022-11-28 12:02:38 +08:00
Keyong Zhou
d381df71f8
[CELEBORN-70] Add epoch for each commitFiles request ( #1012 )
2022-11-27 21:05:14 +08:00
Keyong Zhou
f8bb2cd47d
[CELEBORN-12]Retry on CommitFile request ( #1011 )
2022-11-26 20:56:24 +08:00
Keyong Zhou
9214b82181
[CELEBORN-68] Client might fetch incorrect data chunk ( #1010 )
2022-11-26 18:06:06 +08:00
Ethan Feng
93dbf3f8b1
[CELEBORN-67] Revert "Fix fetch incorrect data chunk" related commits ( #1006 )
...
* Revert "[CELEBORN-50][FOLLOWUP] Channel inactive may cause new client use old stream id to fetch data (#999 )"
This reverts commit 1e8f6dc5e8#1000 )"
This reverts commit f1c4d675d6#998 )"
This reverts commit 0be4b3399c#995 )"
This reverts commit 2b05228871#926 )"
This reverts commit 6f043f8a#954 )"
This reverts commit 64e8ebf1
2022-11-25 20:57:47 +08:00
Angerszhuuuu
2b05228871
[CELEBORN-47][IMPROVEMENT] Refine logs about tracking fetch chunk ( #995 )
2022-11-23 11:56:10 +08:00
Ethan Feng
ee243f286d
[CELEBORN-4] Add metrics about top disk used apps. ( #985 )
2022-11-22 20:06:36 +08:00
Angerszhuuuu
e12000cb67
[CELEBORN-42][BUG] PushMergedData use wrong call back when partition not found ( #991 )
2022-11-22 18:29:15 +08:00
Angerszhuuuu
5ec278f99a
[ISSUE-987][FEATURE] During worker shutdown, return HARD_SPLIT for all existed partition ( #988 )
2022-11-22 14:29:55 +08:00
zhongqiangczq
7adcb5b933
[CELEBORN-6] [REFACTOR] PushDataHandler code refactor ( #966 )
2022-11-16 11:04:24 +08:00
leesf
0b8376e2c7
Cleanup some code ( #943 )
2022-11-11 13:58:39 +08:00
Ethan Feng
6f043f8ae9
[BUG] Fix fetch incorrect data chunk ( #926 )
2022-11-09 22:31:39 +08:00
leesf
aac68c3571
Rename RssException to CelebornException ( #938 )
2022-11-08 10:08:21 +08:00
leesf
496f44eda4
Shutdown worker if initialized failed. ( #931 )
2022-11-07 19:33:35 +08:00
Angerszhuuuu
99a7b85708
[ISSUE-932][REFACTOR] Device check should not directly reportError ( #933 )
...
* [ISSUE-932][REFACTOR] Device check should not directly reportError
2022-11-07 15:15:08 +08:00
nafiy
11081eac6c
[ISSUE-879][BUG] When notifyError, should destroy corresponding file writers ( #912 )
...
* [ISSUE-879][BUG] When notifyError, should destroy corresponding file writers
2022-11-07 14:01:51 +08:00
Angerszhuuuu
100e0057e8
[ISSUE-921][BUG] Flush Error should report non critical error ( #928 )
2022-11-07 11:56:11 +08:00
leesf
3699683a3b
Fix and migrate some configs ( #927 )
2022-11-07 09:41:38 +08:00
Angerszhuuuu
38e15d89e6
[ISSUE-902][IMPROVEMENT][FOLLOWUP] LifecycleManager should reserve blacklist with irrecoverable status ( #914 )
2022-11-04 15:54:45 +08:00
Angerszhuuuu
ea4ed10e5c
[ISSUE-901][BUG] During worker graceful shutdown, worker should report itself as unavailable and avoid master allocate slots on it. ( #905 )
2022-11-02 16:09:58 +08:00
Zhen Wang
643eb84541
[MINOR] Fix typo ( #898 )
2022-11-01 10:03:15 +08:00
nafiy
ce3dc889fa
[ISSUE-867][BUG] Create writer failed should report non-critical error instead of critical error ( #883 )
2022-10-31 21:23:16 +08:00
nafiy
9b1c70f219
[ISSUE-880][BUG] onTrim when flushFileWriters() should catch each file writer's exception, avoid block flush all file writers ( #894 )
2022-10-31 14:31:22 +08:00
Angerszhuuuu
87fcfa767f
[ISSUE-887][REFACTOR] Configuration type convert to Enum ( #888 )
...
* [ISSUE-332][FOLLOWUP] Add deps in worker's pom
* [Refactor] Modify package name of utils to keep consistence
* [Refactor] Modify package name of utils to keep consistence
* [REFACTOR] Remove unused isRegistered in controller
* [ISSUE-887][REFACTOR] Configuration type convert to Enum
* update
* update
* Update RssShuffleManager.java
2022-10-29 13:41:06 +08:00
Cheng Pan
d7be6006e7
Migrate network related conf to structured conf system ( #875 )
...
* Migrate network related conf to structured conf system
* migrate
* fix
* fix
* worker
* fix
* nit
* review
* nit
2022-10-28 10:45:52 +08:00
Angerszhuuuu
d283cca4e1
[ISSUE-869][REFACTOR] Migrate partition size/sorter related conf to Celeborn ConfigEntity ( #870 )
2022-10-27 16:49:55 +08:00
Angerszhuuuu
26dcc118c6
[ISSUE-871][REFACTOR] Migrate Worker conf to Celeborn Configuration System ( #873 )
...
* [ISSUE-871][REFACTOR] Migrate Worker conf to Celeborn Configuration System
2022-10-27 15:35:29 +08:00
Angerszhuuuu
5333819cb0
[ISSUE-866][BUG] Create File twice should show clear log ( #876 )
2022-10-27 14:52:45 +08:00
nafiy
e44e8c9610
[ISSUE-828][REFACTOR] Migrate memory tracker related configs to ConfigEntry ( #831 )
...
* [ISSUE-828][REFACTOR] Migrate memory tracker related configs to ConfigEntry
* Fix based on review
* update doc
* resolve review feedback
* fix
* Fix based on review
* fix based on review
2022-10-25 21:16:53 +08:00
AngersZhuuuu
0bd0a3e9f4
[ISSUE-847][REFACTOR] Migrate codec conf to Celeborn Configuration System ( #848 )
...
* [ISSUE-847][REFACTOR] Migrate codec conf to Celeborn Configuration System
* Update CelebornConf.scala
* follow comments
* update
* update
* update
* Update client.md
2022-10-25 09:16:46 +08:00
Ethan Feng
4df0d4a456
[TEST] Fix unstable LZ4 unit test ( #816 )
2022-10-24 15:36:06 +08:00
Cheng Pan
8d7d397e71
Fix Configuration page and polish naming ( #838 )
...
* Fix Configuration page and polish naming
* nit
* nit
* comment
2022-10-24 12:46:25 +08:00
Ethan Feng
74843f20a9
[BUG] Fix worker lost caused by UnsupportedOperationException ( #837 )
2022-10-24 11:20:42 +08:00
Keyong Zhou
63752e7a37
[BUG] RegisterShuffle should not increase epoch ( #833 )
2022-10-23 23:40:32 +08:00
Ethan Feng
392a252baa
[FOLLOWUP][ISSUE-813]Update doc and fix typo. ( #825 )
2022-10-22 23:02:22 +08:00
AngersZhuuuu
f2610e3b6f
[ISSUE-829][REFACTOR] Unify name of PUSH_DATA_FAIL_MAIN ( #830 )
2022-10-21 19:06:33 +08:00
nafiy
1a8a36e8fe
[ISSUE-812][Refactor] Migrate metrics system related configs to ConfigEntry ( #821 )
2022-10-21 13:57:58 +08:00
AngersZhuuuu
a773c8e6db
[ISSUE-820][Refactor] Rename RssConf to CelebornConf ( #826 )
2022-10-20 20:13:13 +08:00
AngersZhuuuu
8344479df1
[ISSUE-818][REFACTOR] Move existing RssConf.xxx conf method to RssConf class ( #822 )
...
* [ISSUE-818][REFACTOR] Move existing RssConf.xxx conf method to RssConf class
Co-authored-by: Ethan Feng <ethan.aquarius.fmx@gmail.com>
2022-10-20 18:10:59 +08:00
Ethan Feng
5c761a8df3
[ISSUE-813][Refactor] Refactor flusher configurations. ( #813 )
...
* Refactor flusher configurations.
* Refactor flusher configurations.
* Update.
* remove brackets.
* update docs.
* rename.
* update.
* update docs.
* update.
* update.
* update.
* update.
* update.
* update.
* update.
* format.
* update.
* update.
2022-10-20 15:23:17 +08:00
nafiy
1e5bed2da7
[ISSUE-806][REFACTOR] Remove ResourceConsumption out of ControlMessage ( #810 )
...
* [ISSUE-806][REFACTOR] Remove ResourceConsumption out of ControlMessage
* add line before method
* reformat
2022-10-19 17:14:51 +08:00
AngersZhuuuu
23c65a27a9
[ISSUE-798][REFACTOR] Migrate worker-recover related conf to ConfigEntry ( #799 )
2022-10-19 16:42:00 +08:00
nafiy
a75bce905e
[ISSUE-805][REFACTOR] Remove UserIdentifier out of ControlMessage ( #808 )
2022-10-19 15:32:53 +08:00
Cheng Pan
efad4abb5d
Migrate a bunch of configurations ( #786 )
2022-10-18 10:44:01 +08:00
nafiy
0e5beb9562
[ISSUE-774][REFACTOR] Add cache to avoid redundant UserIdentifier object when recover fileinfo ( #781 )
2022-10-17 21:27:54 +08:00
nafiy
0dcf946c9b
[ISSUE-751][REFACTOR] Move userResourceConsumption to WorkerInfo's parameter and format WorkerInfo's toString() ( #767 )
2022-10-17 17:58:39 +08:00
Cheng Pan
ea67f4e060
Introduce categories to ConfigEntry and migrate configurations ( #775 )
2022-10-17 16:56:54 +08:00
Ethan Feng
0959894155
[BUG]Fix rss worker register failure problem. ( #777 )
2022-10-17 09:50:04 +08:00
Cheng Pan
96e969f46e
[BUILD] Extract project.version to Maven Property ( #772 )
2022-10-16 19:01:40 +08:00
nafiy
373b4a744a
[ISSUE-750][Refactor] Add UserIdentifier as a field of file info ( #759 )
2022-10-13 23:15:44 +08:00
Cheng Pan
5829bda21a
Rework and migrate HA configuration system ( #763 )
2022-10-13 22:35:01 +08:00
Cheng Pan
f01a696313
Migrate and refactor configuration for master endpoints ( #752 )
2022-10-11 21:33:21 +08:00
nafiy
3ed38f1e72
[ISSUE-642][FEATURE] worker storage manger store user to shuffke key relation and recover from level db ( #706 )
2022-10-10 18:18:34 +08:00
AngersZhuuuu
13aeb4b644
[ISSUE-736][BUG] Heartbeat worker should update disk info into WorkInfo too to keep consistence with master ( #737 )
2022-10-09 15:41:01 +08:00
AngersZhuuuu
f2a234f870
[ISSUE-739][REFACTOR] Use object wrap pb message method ( #740 )
2022-10-09 11:53:48 +08:00
Ethan Feng
6deda248ac
[REFACTOR]move lifecycle manager to correct package. ( #730 )
2022-10-08 18:14:08 +08:00
Cheng Pan
ab16b4f101
[INFRA] Rename modules w/ celeborn prefix ( #723 )
2022-10-08 08:05:57 +08:00
