Ethan Feng
1f95ccb55a
[CELEBORN-193][FOLLOWUP] Reduce unnecessary binary files. ( #1142 )
2023-01-05 15:04:44 +08:00
Angerszhuuuu
fe8dfb05f3
[CELEBORN-196][REFACTOR] Rename batchHandleRequestPartitions to handleRequestPartitions ( #1144 )
2023-01-05 14:37:10 +08:00
Angerszhuuuu
5edb21d210
[CELEBORN-168][FOLLOWUP] Device metrics should use long value and add size unit in metric name ( #1143 )
...
* [CELEBORN-168][FOLLOWUP] Device metrics should use long value and add size unit in metric name
2023-01-05 11:45:19 +08:00
nafiy
3e80cf2b87
[CELEBORN-168][FEATURE] Add disk usage related metrics for Worker ( #1127 )
2023-01-05 10:35:51 +08:00
Shuang
33c2387594
[CELEBORN-175][BUILD] Include Flink client into the binary release tarball ( #1120 )
...
* [CELEBORN-175] with flink plugin when release packages
* improve
* flink use scala 2.11.x as default
* remove unused version align
* add space before end slash
* improve error message
* update default flink version
2023-01-05 10:31:32 +08:00
Ethan Feng
01b7ea97c9
[CELEBORN-193] Reduce source package size. ( #1140 )
2023-01-03 19:28:03 +08:00
Cheng Pan
0c29c5dd57
[CELEBORN-180][BUILD][FOLLOWUP] Update CI workflow and docs ( #1134 )
2023-01-03 17:58:51 +08:00
Angerszhuuuu
2315f2f988
[CELEBORN-191][BUG] ShuffleClient registerShuffle return RESERVE_SLOTS_FAILED should also been print out ( #1138 )
2023-01-03 17:13:31 +08:00
Cheng Pan
d8b6c7e24d
[CELEBORN-180][FOLLOWUP] Declare PROJECT_DIR before using
2023-01-03 16:44:07 +08:00
Cheng Pan
c9dcf312f8
[CELEBORN-186][BUILD] Auto upload/move release artifacts to svn ( #1130 )
2023-01-03 15:04:40 +08:00
jxysoft
41b1fa46d3
[CELEBORN-185][SPARK] Can't release shuffle data if rss fallback to nss ( #1133 )
...
Co-authored-by: xianyao.jiang <xianyao.jiang@antfin.com>
2023-01-03 14:28:09 +08:00
Cheng Pan
8cadf14864
[CELEBORN-187][LICENSE] Update NOTICE for 2023 ( #1131 )
2023-01-03 14:13:28 +08:00
Cheng Pan
a16ba0e807
[CELEBORN-180][BUILD] Script for creating binary release artifact ( #1129 )
2023-01-03 12:58:42 +08:00
Shuang
5cba307189
[CELEBORN-146] refactor ShuffleMapperAttempts & GetReducerFileGroup ( #1116 )
2022-12-30 18:15:23 +08:00
Angerszhuuuu
425e31797c
[CELEBORN-182][BUG] StorageManager should not delete shuffle file when enable graceful shutdown ( #1126 )
2022-12-30 18:13:36 +08:00
Cheng Pan
b8758a7cb6
[CELEBORN-181][TEST] Rename RssFunSuite to CelebornFunSuite ( #1125 )
2022-12-29 18:10:14 +08:00
Cheng Pan
932a6c2703
[CELEBORN-177] Script for creating source release artifact ( #1122 )
2022-12-29 18:04:54 +08:00
Angerszhuuuu
7d7192af14
[CELEBORN-179][BUG] Repeat remove expired shuffle throw NPE ( #1124 )
2022-12-29 15:47:05 +08:00
Angerszhuuuu
6411fe71b1
[CELEBORN-178][BUG] Default registered flag should be false, not null ( #1123 )
2022-12-29 15:24:09 +08:00
RexAn
6432a129be
[CELEBORN-61][CELEBORN-62][FOLLOW_UP] Fix some issues for slow start ( #1119 )
2022-12-29 12:07:20 +08:00
Binjie Yang
63943cd5cc
[CELEBORN-147][IT]Extraction of common integration test cases ( #1092 )
2022-12-29 12:03:09 +08:00
Angerszhuuuu
b13ddac9d2
[CELEBORN-172][Refactor] Load/Make snapshot use Protobuf serde ( #1118 )
2022-12-29 11:51:14 +08:00
Angerszhuuuu
829f35c753
[CELEBORN-176][BUG] Fix wrong alternative conf of celeborn.worker.flusher.ssd.threads ( #1121 )
2022-12-29 11:11:20 +08:00
Ethan Feng
489ac422c8
[CELEBORN-166] Add LICENSE and NOTICE for binary release artifacts ( #1114 )
...
* [CELEBORN-166] Automate release build process.
* Update .gitignore
Co-authored-by: Cheng Pan <chengpan@apache.org>
* sort dependencies.
* remove dependency list.
* 1.add git ignore.
2.add notice binary.
* update.
* update.
* style.
* style.
* update.
* Revert "style."
This reverts commit b90eb94196a9080d04e732c6cfa5ddf8c570ce01.
* Revert "style."
This reverts commit ef7e70d33f4251f69631c9c4a3c571a714dd902c.
* partial revert 63ae9f6
* Revert "update."
This reverts commit 9fcb3580a229ac53b3b0b2a8e52a6a7f0ef7daad.
* move to release folder
Co-authored-by: Cheng Pan <chengpan@apache.org>
2022-12-29 11:07:30 +08:00
nafiy
77cb7a0477
[CELEBORN-169][REFACTOR] Extract ObservedDevice out from LocalDeviceMonitor ( #1113 )
...
* [CELEBORN-169][REFACTOR] Extract ObservedDevice out from LocalDeviceMonitor
2022-12-28 14:29:00 +08:00
Angerszhuuuu
5603e62e95
[CELEBORN-174][REFACTOR] Move AppDiskUsage related to meta package ( #1117 )
2022-12-27 15:24:42 +08:00
Ethan Feng
3cdc25286d
[CELEBORN-165] Fix ut RetryCommitFilesTest failure. ( #1111 )
2022-12-22 11:39:40 +08:00
Cheng Pan
bf97a2227b
[CELEBORN-163][BUILD] Rename Flink modules and enable Flink CI ( #1110 )
2022-12-21 23:54:47 +08:00
Cheng Pan
7105f98829
[CELEBORN-160][BUILD] Spilt CI workflow ( #1107 )
2022-12-21 23:47:01 +08:00
Cheng Pan
e71e77447f
[CELEBORN-161][BUILD] Disable scaladocs ( #1108 )
2022-12-21 23:35:52 +08:00
Cheng Pan
8a7204fda3
[CELEBORN-159][INFRA] Upgrade requirements.txt to keep consistent w/ celeborn-website ( #1106 )
2022-12-21 22:22:41 +08:00
Ethan Feng
5aa959a335
[CELEBORN-157] Change prefix of configurations to celeborn. ( #1104 )
2022-12-21 15:17:28 +08:00
zhongqiangczq
907364dbf2
[CELEBORN-156] add remoteShuffleResultPartition in flink-plugin ( #1103 )
2022-12-21 12:22:17 +08:00
nafiy
f13dfb7421
[CELEBORN-113][FEATURE] Add metrics to monitor non-critical error number on local device ( #1100 )
2022-12-20 22:30:55 +08:00
Keyong Zhou
2f0682265e
[CELEBORN-119] Add timeout for pushdata ( #1097 )
2022-12-20 20:40:42 +08:00
Cheng Pan
1822896156
[CELEBORN-151][K8S] Celeborn should run as celeborn instead of root in container ( #895 )
2022-12-20 15:57:14 +08:00
Keyong Zhou
a2dd72f20c
[CELEBORN-155] Wrong TimeUnit for registerShuffleRetryWait in Shuffle… ( #1099 )
2022-12-19 17:32:18 +08:00
Shuang
13769f0f0a
[CELEBORN-121] Refactor batchHandleCommitPartition ( #1089 )
2022-12-19 12:39:39 +08:00
zhongqiangczq
d31c94a903
[CELEBORN-154] replace shuffleclient's shutDown to shutdown ( #1098 )
2022-12-19 09:33:44 +08:00
zky.zhoukeyong
3a4fd04f33
Revert "[CELEBORN-120] MapPartitionFileWriter completes some functions to support mappartition ( #1062 )"
...
This reverts commit aa5a4d8c41
2022-12-17 11:10:54 +08:00
Ethan Feng
39394526a8
[CELEBORN-142]Keep committed partition locations semantic consistent when commit files on HDFS. ( #1091 )
2022-12-16 19:02:02 +08:00
zhongqiangczq
aa5a4d8c41
[CELEBORN-120] MapPartitionFileWriter completes some functions to support mappartition ( #1062 )
2022-12-16 18:40:22 +08:00
nafiy
c931663e5f
[CELEBORN-110][REFACTOR] Notify critical error after collecting a certain number of non-critical error ( #1055 )
2022-12-16 15:47:36 +08:00
Cheng Pan
dc66369973
[CELEBORN-150][BUILD] Reduce binary tarball size by sharing jars ( #1095 )
...
* [CELEBORN-150][BUILD] Reduce binary tarball size by sharing jars
* nit
* nit
* docker
* nit
* cp -R
2022-12-16 14:30:17 +08:00
Binjie Yang
853d0df191
[CELEBORN-149] Upload failure CI unit test logs for developer debug ( #1094 )
2022-12-16 04:31:13 +08:00
nafiy
ddab27a1d7
[CELEBORN-145][REFACTOR] Add reason in CheckQuotaResponse ( #1093 )
...
* [CELEBORN-145][REFACTOR] Add reason in CheckQuotaResponse
2022-12-15 18:16:34 +08:00
Ethan Feng
65cb36c002
[CELEBORN-83][FOLLOWUP] Fix various bugs when using HDFS as storage. ( #1065 )
2022-12-15 15:20:29 +08:00
Ethan Feng
df5ed8ec09
[CELEBORN-144]Lower code coverage requirements. ( #1090 )
2022-12-15 13:48:47 +08:00
Shuang
e3576e4e7a
[CELEBORN-117] refactor CommitManager, implements M/R Partition Commi… ( #1060 )
2022-12-15 11:09:59 +08:00
Angerszhuuuu
a341ccb22c
[CELEBORN-140][FOLLOWUP] quota.yaml file not support default value -1 now ( #1088 )
2022-12-15 10:59:02 +08:00
