apache/celeborn
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
4cb4701ede
celeborn/docs/migration.md
Cheng Pan e22379c3ab [CELEBORN-638] Migrate configurations celeborn.ha.master.* to celeborn.master.ha.* 
### What changes were proposed in this pull request?

It was discussed during the last meeting, but abandoned due to the complication.

### Why are the changes needed?

Make the configuration unified.

### Does this PR introduce _any_ user-facing change?

Yes, but the legacy configurations still take effect.

### How was this patch tested?

New UTs.

Closes #1549 from pan3793/CELEBORN-638.

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: zky.zhoukeyong <zky.zhoukeyong@alibaba-inc.com>
2023-06-16 18:18:26 +08:00

1.6 KiB
Raw Blame History

hide license
navigation
Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Migration Guide

Upgrading from 0.2.1 to 0.3.0

  • From 0.3.0 on the default value for celeborn.client.push.replicate.enabled is changed from true to false, users who want replication on should explicitly enable replication. For example, to enable replication for Spark users should add the spark config when submitting job: spark.celeborn.client.push.replicate.enabled=true

  • From 0.3.0 on the default value for celeborn.worker.storage.workingDir is changed from hadoop/rss-worker/shuffle_data to rss-worker/shuffle_data, users who want to use origin working dir path should set this configuration.

  • Since 0.3.0, configuration namespace celeborn.ha.master is deprecated, and will be removed in the future versions. All configurations celeborn.ha.master.* should migrate to celeborn.master.ha.*.