Ethan Feng
a239f9f284
[CELEBORN-228]Refactor PartitionFileSorter to avoid specific JDK dependency. ( #1168 )
2023-01-16 20:06:47 +08:00
Ethan Feng
e219e8b44e
[CELEBORN-171] Support JDK11. ( #1169 )
2023-01-16 19:35:54 +08:00
zy.jordan
bb96700415
[CELEBORN-223] The default rpc thread num of pushServer/replicateServer/fetchServer should be the number of total of Flusher's thread ( #1163 )
2023-01-16 12:03:46 +08:00
Keyong Zhou
fa7ba43136
[CELEBORN-225] Add global default configuration for number of flusher… ( #1165 )
2023-01-14 13:20:44 +08:00
zhongqiangczq
b5e59ceefc
[CELEBORN-222] Flink plugin RemoteShuffleOutputGate adds ut about nettybufferTransform ( #1162 )
2023-01-13 16:48:11 +08:00
zhongqiangczq
cbeed65ecd
[CELEBORN-202][Flink] Flink plugin BuffPacker adds unpack implements for shu… ( #1150 )
2023-01-13 16:47:03 +08:00
zhongqiangczq
1836fe187b
[CELEBORN-197] in mappartition, check transportClient whether changed while sending messages ( #1145 )
2023-01-13 16:45:26 +08:00
zhongqiangczq
3661222d98
[CELEBORN-195] add implementation to MapPartitionFileWriter ( #1141 )
2023-01-13 16:41:11 +08:00
zhongqiangczq
411ab09ffb
[CELEBORN-158][Flink] Add ShuffleServiceFactory to Support MapPartition in … ( #1105 )
2023-01-13 16:38:46 +08:00
Binjie Yang
2bb8e7a2a2
[CELEBORN-218][HELM] Move helm chart to dedicated directory ( #1160 )
2023-01-11 16:11:27 +08:00
Shuang
810a8d01e0
[CELEBORN-212] refresh client if current client is inactive. ( #1159 )
2023-01-11 11:54:50 +08:00
Binjie Yang
fb91544843
[CELEBORN-210][HELM] Add recommended labels in celeborn chart ( #1156 )
...
* [HELM] Add recommended labels in celeborn chart
* fix master and worker affinity
2023-01-10 19:18:14 +08:00
Shuang
1332362bff
[CELEBORN-213] Add configuration for whether to close idle connections in client side ( #1157 )
2023-01-10 19:13:33 +08:00
zy.jordan
19197b9190
[CELEBORN-214] Push/Replicate/Fetch io threads default value is 16 ( #1158 )
2023-01-10 17:46:56 +08:00
Angerszhuuuu
e155ec122a
[CELEBORN-190] doPushMergedData should also support revive multiple times, not only twice ( #1136 )
2023-01-10 11:39:40 +08:00
Shuang
37ccc40336
[CELEBORN-208][IMPROVEMENT] Make flink default scala version to 2.12 due to ide problem. ( #1154 )
2023-01-10 11:37:52 +08:00
nafiy
9635725480
[CELEBORN-204][IMPROVEMENT]Collect disk usage metrics in byte unit by default ( #1153 )
2023-01-09 17:36:18 +08:00
Ethan Feng
c860fdf748
[CELEBORN-205] bump version to 0.3.0-SNAPSHOT. ( #1152 )
2023-01-09 11:52:34 +08:00
Shuang
2ec06472fe
[CELEBORN-203] fix NPE when removeExpiredShuffle in LifecycleManager. ( #1151 )
2023-01-06 18:32:17 +08:00
Angerszhuuuu
0d5809ff0c
[CELEBORN-192][IMPROVEMENT] Change FAILED status to REQUEST_FAILED since it's all used when RPC request failed. ( #1139 )
2023-01-06 16:53:04 +08:00
Ethan Feng
5595f2f4b3
[CELEBORN-124]Add buffer stream. ( #1069 )
2023-01-06 15:54:52 +08:00
Shuang
3b2be25a50
[CELEBORN-173] refactor minicluster and fix ut ( #1147 )
2023-01-05 20:39:19 +08:00
Angerszhuuuu
415452d9c4
[CELEBORN-189][IMPROVEMENT] PushDataFailedSlave should add slave worker to blacklist ( #1135 )
2023-01-05 20:12:07 +08:00
Fu Chen
ab449ffdd7
[CELEBORN-198] Fix the wrong configuration path of plugin protobuf-maven-plugin and … ( #1146 )
2023-01-05 20:09:31 +08:00
Ethan Feng
1f95ccb55a
[CELEBORN-193][FOLLOWUP] Reduce unnecessary binary files. ( #1142 )
2023-01-05 15:04:44 +08:00
Angerszhuuuu
fe8dfb05f3
[CELEBORN-196][REFACTOR] Rename batchHandleRequestPartitions to handleRequestPartitions ( #1144 )
2023-01-05 14:37:10 +08:00
Angerszhuuuu
5edb21d210
[CELEBORN-168][FOLLOWUP] Device metrics should use long value and add size unit in metric name ( #1143 )
...
* [CELEBORN-168][FOLLOWUP] Device metrics should use long value and add size unit in metric name
2023-01-05 11:45:19 +08:00
nafiy
3e80cf2b87
[CELEBORN-168][FEATURE] Add disk usage related metrics for Worker ( #1127 )
2023-01-05 10:35:51 +08:00
Shuang
33c2387594
[CELEBORN-175][BUILD] Include Flink client into the binary release tarball ( #1120 )
...
* [CELEBORN-175] with flink plugin when release packages
* improve
* flink use scala 2.11.x as default
* remove unused version align
* add space before end slash
* improve error message
* update default flink version
2023-01-05 10:31:32 +08:00
Ethan Feng
01b7ea97c9
[CELEBORN-193] Reduce source package size. ( #1140 )
2023-01-03 19:28:03 +08:00
Cheng Pan
0c29c5dd57
[CELEBORN-180][BUILD][FOLLOWUP] Update CI workflow and docs ( #1134 )
2023-01-03 17:58:51 +08:00
Angerszhuuuu
2315f2f988
[CELEBORN-191][BUG] ShuffleClient registerShuffle return RESERVE_SLOTS_FAILED should also been print out ( #1138 )
2023-01-03 17:13:31 +08:00
Cheng Pan
d8b6c7e24d
[CELEBORN-180][FOLLOWUP] Declare PROJECT_DIR before using
2023-01-03 16:44:07 +08:00
Cheng Pan
c9dcf312f8
[CELEBORN-186][BUILD] Auto upload/move release artifacts to svn ( #1130 )
2023-01-03 15:04:40 +08:00
jxysoft
41b1fa46d3
[CELEBORN-185][SPARK] Can't release shuffle data if rss fallback to nss ( #1133 )
...
Co-authored-by: xianyao.jiang <xianyao.jiang@antfin.com>
2023-01-03 14:28:09 +08:00
Cheng Pan
8cadf14864
[CELEBORN-187][LICENSE] Update NOTICE for 2023 ( #1131 )
2023-01-03 14:13:28 +08:00
Cheng Pan
a16ba0e807
[CELEBORN-180][BUILD] Script for creating binary release artifact ( #1129 )
2023-01-03 12:58:42 +08:00
Shuang
5cba307189
[CELEBORN-146] refactor ShuffleMapperAttempts & GetReducerFileGroup ( #1116 )
2022-12-30 18:15:23 +08:00
Angerszhuuuu
425e31797c
[CELEBORN-182][BUG] StorageManager should not delete shuffle file when enable graceful shutdown ( #1126 )
2022-12-30 18:13:36 +08:00
Cheng Pan
b8758a7cb6
[CELEBORN-181][TEST] Rename RssFunSuite to CelebornFunSuite ( #1125 )
2022-12-29 18:10:14 +08:00
Cheng Pan
932a6c2703
[CELEBORN-177] Script for creating source release artifact ( #1122 )
2022-12-29 18:04:54 +08:00
Angerszhuuuu
7d7192af14
[CELEBORN-179][BUG] Repeat remove expired shuffle throw NPE ( #1124 )
2022-12-29 15:47:05 +08:00
Angerszhuuuu
6411fe71b1
[CELEBORN-178][BUG] Default registered flag should be false, not null ( #1123 )
2022-12-29 15:24:09 +08:00
RexAn
6432a129be
[CELEBORN-61][CELEBORN-62][FOLLOW_UP] Fix some issues for slow start ( #1119 )
2022-12-29 12:07:20 +08:00
Binjie Yang
63943cd5cc
[CELEBORN-147][IT]Extraction of common integration test cases ( #1092 )
2022-12-29 12:03:09 +08:00
Angerszhuuuu
b13ddac9d2
[CELEBORN-172][Refactor] Load/Make snapshot use Protobuf serde ( #1118 )
2022-12-29 11:51:14 +08:00
Angerszhuuuu
829f35c753
[CELEBORN-176][BUG] Fix wrong alternative conf of celeborn.worker.flusher.ssd.threads ( #1121 )
2022-12-29 11:11:20 +08:00
Ethan Feng
489ac422c8
[CELEBORN-166] Add LICENSE and NOTICE for binary release artifacts ( #1114 )
...
* [CELEBORN-166] Automate release build process.
* Update .gitignore
Co-authored-by: Cheng Pan <chengpan@apache.org>
* sort dependencies.
* remove dependency list.
* 1.add git ignore.
2.add notice binary.
* update.
* update.
* style.
* style.
* update.
* Revert "style."
This reverts commit b90eb94196a9080d04e732c6cfa5ddf8c570ce01.
* Revert "style."
This reverts commit ef7e70d33f4251f69631c9c4a3c571a714dd902c.
* partial revert 63ae9f6
* Revert "update."
This reverts commit 9fcb3580a229ac53b3b0b2a8e52a6a7f0ef7daad.
* move to release folder
Co-authored-by: Cheng Pan <chengpan@apache.org>
2022-12-29 11:07:30 +08:00
nafiy
77cb7a0477
[CELEBORN-169][REFACTOR] Extract ObservedDevice out from LocalDeviceMonitor ( #1113 )
...
* [CELEBORN-169][REFACTOR] Extract ObservedDevice out from LocalDeviceMonitor
2022-12-28 14:29:00 +08:00
Angerszhuuuu
5603e62e95
[CELEBORN-174][REFACTOR] Move AppDiskUsage related to meta package ( #1117 )
2022-12-27 15:24:42 +08:00
