Ethan Feng
d58feef3a7
[CELEBORN-280] Enable Jacoco multi-module mode to collect coverage report. ( #1215 )
2023-03-30 14:24:42 +08:00
Shuang
45013b8bae
[CELEBORN-489][FLINK]fix retry client for open stream ( #1397 )
2023-03-30 11:44:19 +08:00
Aravind Patnam
2c3005ad5b
[CELEBORN-491] Improve exception logging in RssInputStream ( #1398 )
2023-03-30 10:21:07 +08:00
Shuang
3dd649f5b4
[CELEBORN-488] [FLINK] Calculate required shuffle memory before allocating slots if resources are specified ( #1396 )
2023-03-29 17:12:32 +08:00
Cheng Pan
23d054b33a
[CELEBORN-372][LICENSE] Remove exclusion from Apache RAT ( #1392 )
2023-03-28 21:41:22 +08:00
zhongqiangchen
cd92c423cd
[CELEBORN-475] Support extra tags for prometheus metrics ( #1385 )
...
[CELEBORN-475] Support extra tags for prometheus metrics
2023-03-28 21:22:28 +08:00
Ethan Feng
cbe65a7e2e
[CELEBORN-477][FOLLOWUP] Report failed partition to flink framework. ( #1393 )
2023-03-28 18:56:39 +08:00
Ethan Feng
6cee85748d
[CELEBORN-477][FLINK] Report failed partition to flink framework. ( #1391 )
2023-03-28 15:54:37 +08:00
CVEDetect
7001f10461
[CELEBORN-482] Fix CVE dependency issue
2023-03-27 22:47:03 +08:00
Angerszhuuuu
9d9a2d4ea8
[CELEBORN-479][FOLLOWUP] Return empty tasks instead of null to avoid NPE ( #1388 )
2023-03-27 17:03:06 +08:00
Keyong Zhou
cb19ed1c66
[CELEBORN-479][PERF] Refactor DataPushQueue.takePushTask to avoid busy wait ( #1386 )
2023-03-27 16:18:55 +08:00
Fei Wang
b40c573069
[CELEBORN-474][FOLLOWUP] Using inner static ConcurrentHashMap class and only apply for JDK8 ( #1384 )
2023-03-27 16:16:23 +08:00
Kerwin Zhang
27a1f369cf
[CELEBORN-472] Support using Celeborn in the scenario of switching multiple SparkContexts in the same process ( #1379 )
2023-03-27 16:10:34 +08:00
Fei Wang
0d1695abd8
[CELEBORN-473] Enable file system cache for viewfs in ShuffleClient as well
2023-03-26 09:44:04 +08:00
Fei Wang
7c444cb0c5
[CELEBORN-474] Speed up ConcurrentHashMap#computeIfAbsent ( #1383 )
2023-03-26 09:41:59 +08:00
Fei Wang
c609c0ebaa
[MINOR] Fix typo and remove unused code ( #1381 )
...
* fix typo
* remove unused
2023-03-25 23:20:33 +08:00
Angerszhuuuu
acf6fd3bd2
[CELEBORN-345] TransportResponseHandler create too much thread ( #1373 )
2023-03-24 17:16:26 +08:00
Shuang
89b3f3887d
[CELEBORN-356] [FLINK] Support release single partition resource ( #1314 )
2023-03-24 17:15:28 +08:00
cxzl25
2adbce942a
[CELEBORN-471] Fix String.format wrong type in ShuffleClientImpl ( #1378 )
2023-03-24 16:05:48 +08:00
Keyong Zhou
2bfa7e8965
[CELEBORN-466][FLINK] ReadBufferDispatcher.recycle should log error when refCnt != 1 ( #1377 )
2023-03-23 20:33:28 +08:00
Lianne Li
a071bdf6d7
[CELEBORN-449] Repair the hdfs path regex ( #1367 )
...
* Path protocols are all started with xxx://, and is unnecessary to restrict the content after that. Actually, it makes an error when write shuffle files which like "xxx://abc/shuffle/hadoop/rss-worker/shuffle_data/spark-0de72e2ce2e24f6db69c2228dd12a514/0/0-0-0"
---------
Co-authored-by: ming.li2 <ming.li2@dmall.com>
Co-authored-by: Cheng Pan <pan3793@gmail.com>
Co-authored-by: Ethan Feng <ethanfeng@apache.org>
2023-03-23 12:40:41 +08:00
Binjie Yang
a2e6687466
[CELEBORN-460][BUG][HELM]Helm Upgrade Release Fail due to change image version ( #1374 )
...
[CELEBORN-460][HELM]Helm Upgrade Release Fail due to change image version
2023-03-23 12:38:53 +08:00
Keyong Zhou
885e0cef32
[CELEBORN-459] Remove chunkTracker from FileManagedBuffers to avoid conflict with stream reuse ( #1372 )
2023-03-22 11:43:38 +08:00
Keyong Zhou
3d6fba553b
[CELEBORN-454] Code refine for worker ( #1371 )
2023-03-22 10:39:14 +08:00
Keyong Zhou
13c610838b
[CELEBORN-455] Use 4 bytes instead of 16 to read mapId in FileWriter.… ( #1369 )
2023-03-21 19:21:50 +08:00
jiaoqingbo
ba89ee98ce
[CELEBORN-453] In the removeExpiredShuffle method, the taskShuffleId should be removed instead of the shuffleId ( #1368 )
2023-03-21 16:49:50 +08:00
Angerszhuuuu
f16c7b414e
[CELEBORN-445] Add CelebornRackResolver to support rack reoslve ( #1366 )
2023-03-21 16:40:46 +08:00
Binjie Yang
87267a6493
[CELEBORN-447][Helm]Should nslookup dns with namespace before start master & worker ( #1365 )
2023-03-21 14:52:42 +08:00
Angerszhuuuu
56d796638f
[CELEBORN-438] Move ServletPath to MetricsSytsem ( #1364 )
2023-03-20 18:22:40 +08:00
Ethan Feng
2ac87ae53e
[CELEBORN-440][FLINK] Implement numConcurrentReading in RemoteShuffleInputGate. ( #1363 )
2023-03-20 17:43:42 +08:00
Keyong Zhou
107868d4f1
[CELEBORN-441][FLINK] Move ShuffleTaskInfo to Flink Plugin ( #1361 )
2023-03-20 13:31:53 +08:00
Keyong Zhou
9401db2bc8
[CELEBORN-443] Code refine for client and common ( #1362 )
2023-03-20 10:37:43 +08:00
乐活优格
0b78c6d325
[CELEBORN-442]Support hdfs compatible file system ( #1360 )
2023-03-18 11:47:46 +08:00
Keyong Zhou
b18042d08e
[CELEBORN-435][FLINK] Code refine for RemoteShuffleInputGate ( #1358 )
2023-03-17 16:55:05 +08:00
hongdd
ae390d9615
[CELEBORN-439] Fix java version check in start-work ( #1359 )
2023-03-17 16:45:57 +08:00
Ethan Feng
0ebad677d7
[CELEBORN-434] Add constrain about memory manager's parameters. ( #1356 )
2023-03-17 15:14:03 +08:00
Keyong Zhou
66cc720861
[CELEBORN-432][FLINK] Refine RemoteShuffleOutputGate and RemoteShuffleInputGateFactory ( #1355 )
2023-03-16 14:07:11 +08:00
Keyong Zhou
637081f604
[CELEBORN-428][FLINK] Remove unnecessary lock in PartitionSortedBuffer ( #1352 )
2023-03-16 11:56:23 +08:00
Ethan Feng
599bdbeb72
[CELEBORN-420] Add hosts template and docs about start-all scripts. ( #1354 )
2023-03-16 11:33:32 +08:00
Angerszhuuuu
e61130d397
[CELEBORN-423][FOLLOWUP] Format http request ( #1353 )
...
* [CELEBORN-423][FOLLOWUP] Format http request
2023-03-15 16:30:23 +08:00
Angerszhuuuu
889e8ca644
[CELEBORN-423][FOLLOWUP] Format http request ( #1351 )
2023-03-15 14:40:05 +08:00
Angerszhuuuu
1f56a5e5d1
[CELEBORN-423] Format http request result ( #1349 )
2023-03-15 10:32:01 +08:00
Shuang
4f6e90a7d9
[CELEBORN-418][FLINK][FOLLOW UP]Need drop unused bytes from netty when task was already failed ( #1350 )
2023-03-14 21:04:11 +08:00
Ethan Feng
6f317c77ee
[CELEBORN-422][FLINK] Remove unused fields in ReadData. ( #1347 )
2023-03-14 19:49:00 +08:00
Shuang
63ab1a66a9
[CELEBORN-418][FLINK] Need drop unused bytes from netty when task was already failed ( #1348 )
2023-03-14 19:48:41 +08:00
Angerszhuuuu
3907d70212
[CELEBORN-421] Add shutdown and registered to http request ( #1346 )
...
* [CELEBORN-421] Add shutdown and registered to http request
2023-03-14 18:23:21 +08:00
Angerszhuuuu
7d7279a9bc
[CELEBORN-420] Add unavailablePeers to http request ( #1345 )
...
* [CELEBORN-420] Add unavailablePeers to http request
2023-03-14 17:23:45 +08:00
Shuang
1fa00c0317
[CELEBORN-391][FLINK][FOLLOW UP] fix clean stream twice & refine log & add ut ( #1344 )
2023-03-14 15:38:49 +08:00
Angerszhuuuu
364acbc66a
[CELEBORN-407] Add conf setting to http request ( #1337 )
...
* [CELEBORN-407] Add conf setting to http request
2023-03-14 14:47:56 +08:00
Shuang
cd5241d399
[CELEBORN-381][FLINK] notify the task with the error message when channel in active. ( #1341 )
2023-03-14 11:28:03 +08:00
