Angerszhuuuu
89c3013122
[ISSUE-851][REFACTOR] Migrate quota configruation to Celeborn Configuration System ( #852 )
...
* [ISSUE-851][REFACTOR] Migrate quota configruation to Celeborn Configuration System
2022-10-26 14:09:44 +08:00
Cheng Pan
a282b3af8a
[INFRA] Add NOTICE ( #860 )
2022-10-26 11:38:15 +08:00
Cheng Pan
24eae43f56
[INFRA] Add DISCLAIMER ( #859 )
2022-10-26 11:37:38 +08:00
Cheng Pan
873eeeb1ed
[BUILD] Add apache- prefix in release tarball name ( #854 )
2022-10-25 22:39:48 +08:00
nafiy
e44e8c9610
[ISSUE-828][REFACTOR] Migrate memory tracker related configs to ConfigEntry ( #831 )
...
* [ISSUE-828][REFACTOR] Migrate memory tracker related configs to ConfigEntry
* Fix based on review
* update doc
* resolve review feedback
* fix
* Fix based on review
* fix based on review
2022-10-25 21:16:53 +08:00
Ethan Feng
8800fc4a8e
[Refactor] Refine rpc cache configs ( #853 )
...
* refine rpc cache configs.
* update.
* update.
* update.
2022-10-25 20:28:18 +08:00
Ethan Feng
45ef716737
[Feature] Cache GetReducerFileGroupResponse to avoid lifecycle manager oom. ( #792 )
2022-10-25 16:16:44 +08:00
Cheng Pan
e71c0228aa
Migrate columnar shuffle configurations to ConfigEntry ( #844 )
2022-10-25 14:26:11 +08:00
AngersZhuuuu
2ebf873b3c
[ISSUE-845][REFACTOR] Migrate partition split related conf to Celeborn Configuration System ( #846 )
...
[ISSUE-845][REFACTOR] Migrate partition split related conf to Celeborn Configuration System
2022-10-25 10:54:45 +08:00
AngersZhuuuu
0bd0a3e9f4
[ISSUE-847][REFACTOR] Migrate codec conf to Celeborn Configuration System ( #848 )
...
* [ISSUE-847][REFACTOR] Migrate codec conf to Celeborn Configuration System
* Update CelebornConf.scala
* follow comments
* update
* update
* update
* Update client.md
2022-10-25 09:16:46 +08:00
Kerwin Zhang
1c6b356775
[FEATURE] Add new ColumnarBatchBuild for columnar shuffle codegen ( #840 )
2022-10-24 21:07:11 +08:00
Cheng Pan
e3d649fff3
Change slot to slots for consistency ( #843 )
2022-10-24 20:49:28 +08:00
AngersZhuuuu
0fdb19065a
[ISSUE-841][REFACTOR] Migrate shuffle client side conf to Celeborn Configuration System ( #842 )
2022-10-24 20:48:48 +08:00
Ethan Feng
4df0d4a456
[TEST] Fix unstable LZ4 unit test ( #816 )
2022-10-24 15:36:06 +08:00
Cheng Pan
8d7d397e71
Fix Configuration page and polish naming ( #838 )
...
* Fix Configuration page and polish naming
* nit
* nit
* comment
2022-10-24 12:46:25 +08:00
Ethan Feng
74843f20a9
[BUG] Fix worker lost caused by UnsupportedOperationException ( #837 )
2022-10-24 11:20:42 +08:00
Ethan Feng
196f3800c2
Fix incorrect action command type configs. ( #836 )
2022-10-24 10:05:11 +08:00
Keyong Zhou
63752e7a37
[BUG] RegisterShuffle should not increase epoch ( #833 )
2022-10-23 23:40:32 +08:00
Ethan Feng
392a252baa
[FOLLOWUP][ISSUE-813]Update doc and fix typo. ( #825 )
2022-10-22 23:02:22 +08:00
nafiy
d0058fb2c5
[ISSUE-780][REFACTOR] Refactor PartitionLocation's methods ( #791 )
2022-10-22 22:46:45 +08:00
AngersZhuuuu
f2610e3b6f
[ISSUE-829][REFACTOR] Unify name of PUSH_DATA_FAIL_MAIN ( #830 )
2022-10-21 19:06:33 +08:00
nafiy
1a8a36e8fe
[ISSUE-812][Refactor] Migrate metrics system related configs to ConfigEntry ( #821 )
2022-10-21 13:57:58 +08:00
Ethan Feng
77ff94ebf0
[FEATURE]Add action to process stale pull requests. ( #817 )
2022-10-21 10:39:51 +08:00
Ethan Feng
6f914997ed
[FEATURE]Add slash regression trigger. ( #823 )
2022-10-20 23:35:25 +08:00
AngersZhuuuu
a773c8e6db
[ISSUE-820][Refactor] Rename RssConf to CelebornConf ( #826 )
2022-10-20 20:13:13 +08:00
AngersZhuuuu
8344479df1
[ISSUE-818][REFACTOR] Move existing RssConf.xxx conf method to RssConf class ( #822 )
...
* [ISSUE-818][REFACTOR] Move existing RssConf.xxx conf method to RssConf class
Co-authored-by: Ethan Feng <ethan.aquarius.fmx@gmail.com>
2022-10-20 18:10:59 +08:00
Shuang
60e3297e98
[ISSUE-555][REFACTOR] Delete epoch from PushData ( #819 )
2022-10-20 16:51:51 +08:00
Ethan Feng
5c761a8df3
[ISSUE-813][Refactor] Refactor flusher configurations. ( #813 )
...
* Refactor flusher configurations.
* Refactor flusher configurations.
* Update.
* remove brackets.
* update docs.
* rename.
* update.
* update docs.
* update.
* update.
* update.
* update.
* update.
* update.
* update.
* format.
* update.
* update.
2022-10-20 15:23:17 +08:00
nafiy
1e5bed2da7
[ISSUE-806][REFACTOR] Remove ResourceConsumption out of ControlMessage ( #810 )
...
* [ISSUE-806][REFACTOR] Remove ResourceConsumption out of ControlMessage
* add line before method
* reformat
2022-10-19 17:14:51 +08:00
AngersZhuuuu
23c65a27a9
[ISSUE-798][REFACTOR] Migrate worker-recover related conf to ConfigEntry ( #799 )
2022-10-19 16:42:00 +08:00
Cheng Pan
c8ed929882
Bump maven-surefire-plugin 3.0.0-M7 ( #803 )
2022-10-19 16:39:17 +08:00
Cheng Pan
d4b810af7d
Bump maven-shade-plugin 3.4.0 ( #804 )
2022-10-19 16:38:58 +08:00
Zhen Wang
31be2c6896
[MINOR] Use rssConf instead of sparkConf to avoid unnecessary parsing to generate rssConf. ( #800 )
2022-10-19 16:25:08 +08:00
Cheng Pan
bd2a747904
Bump slf4j 1.7.36 ( #807 )
2022-10-19 15:35:14 +08:00
nafiy
a75bce905e
[ISSUE-805][REFACTOR] Remove UserIdentifier out of ControlMessage ( #808 )
2022-10-19 15:32:53 +08:00
Cheng Pan
dc51923ff5
Bump os-maven-plugin 1.7.0 ( #802 )
2022-10-19 14:27:14 +08:00
AngersZhuuuu
7fedaaeca1
[ISSUE-795][BUG] Batch handle change partition throw NPE ( #796 )
2022-10-19 10:54:08 +08:00
Cheng Pan
cb07cf62c0
Auto generate configuration docs ( #794 )
2022-10-19 10:50:22 +08:00
Ethan Feng
b708d63678
Optimize regression logic. ( #793 )
2022-10-18 20:31:46 +08:00
Ethan Feng
bff2a7065b
Keep one copy of roaringbitmap to reduce memory usage. ( #790 )
2022-10-18 13:26:49 +08:00
nafiy
4e27c9464a
[ISSUE-773][BUG] WorkerInfos hashCode() should be consistent with equals() ( #789 )
2022-10-18 12:15:31 +08:00
Keyong Zhou
08b4a33fe2
[DOC] Refine doc for config celeborn.worker.storage.dirs ( #788 )
2022-10-18 10:59:16 +08:00
Cheng Pan
efad4abb5d
Migrate a bunch of configurations ( #786 )
2022-10-18 10:44:01 +08:00
nafiy
0e5beb9562
[ISSUE-774][REFACTOR] Add cache to avoid redundant UserIdentifier object when recover fileinfo ( #781 )
2022-10-17 21:27:54 +08:00
Ethan Feng
6706e5629c
Shade commons-lang3 pacakge to avoid conflicts. ( #785 )
2022-10-17 20:20:54 +08:00
nafiy
0dcf946c9b
[ISSUE-751][REFACTOR] Move userResourceConsumption to WorkerInfo's parameter and format WorkerInfo's toString() ( #767 )
2022-10-17 17:58:39 +08:00
Kerwin Zhang
80d9b917ea
[FEATURE] Add new ColumnBuilder types to support columnar shuffle codegen ( #779 )
2022-10-17 17:21:48 +08:00
Cheng Pan
ea67f4e060
Introduce categories to ConfigEntry and migrate configurations ( #775 )
2022-10-17 16:56:54 +08:00
Ethan Feng
b4b19892eb
[Feature]Add ci scripts for regression and benchmark. ( #766 )
2022-10-17 16:46:27 +08:00
Ethan Feng
0959894155
[BUG]Fix rss worker register failure problem. ( #777 )
2022-10-17 09:50:04 +08:00
