5b3f43dffc
### What changes were proposed in this pull request? Add --add-opens to bootstrap shell scripts ### Why are the changes needed? Additional `--add-opens` is required for Java 17, notes, the `--add-opens` list is copied from Spark and was used for UT, I am not sure each of them is required but at least the UT passed with them. Details supplied by cfmcgrady [JEP 403](https://openjdk.java.net/jeps/403) targeted for [JDK 17](https://openjdk.java.net/projects/jdk/17/) will remove `--illegal-access` flag. That will be equivalent to `--illegal-access=deny`. this means using reflection to invoke protected methods of exported `java.*` APIs will no longer work. For example: ```shell > /Library/Java/JavaVirtualMachines/temurin-17.jdk/Contents/Home/bin/jshell | 欢迎使用 JShell -- 版本 17.0.7 | 要大致了解该版本, 请键入: /help intro jshell> java.nio.ByteBuffer direct = java.nio.ByteBuffer.allocateDirect(1); direct ==> java.nio.DirectByteBuffer[pos=0 lim=1 cap=1] jshell> direct.getClass().getDeclaredConstructor(long.class, int.class).setAccessible(true); | 异常错误 java.lang.reflect.InaccessibleObjectException：Unable to make private java.nio.DirectByteBuffer(long,int) accessible: module java.base does not "opens java.nio" to unnamed module 34c45dca | at AccessibleObject.checkCanSetAccessible (AccessibleObject.java:354) | at AccessibleObject.checkCanSetAccessible (AccessibleObject.java:297) | at Constructor.checkCanSetAccessible (Constructor.java:188) | at Constructor.setAccessible (Constructor.java:181) | at (#2:1) jshell> ``` ```shell > /Library/Java/JavaVirtualMachines/temurin-17.jdk/Contents/Home/bin/jshell -R --add-opens=java.base/java.nio=ALL-UNNAMED | 欢迎使用 JShell -- 版本 17.0.7 | 要大致了解该版本, 请键入: /help intro jshell> java.nio.ByteBuffer direct = java.nio.ByteBuffer.allocateDirect(1); direct ==> java.nio.DirectByteBuffer[pos=0 lim=1 cap=1] jshell> direct.getClass().getDeclaredConstructor(long.class, int.class).setAccessible(true); jshell> ``` ### Does this PR introduce _any_ user-facing change? Yes, for Java 17 support. ### How was this patch tested? CI and review Closes #1677 from pan3793/CELEBORN-763. Authored-by: Cheng Pan <chengpan@apache.org> Signed-off-by: Cheng Pan <chengpan@apache.org>
60 lines
3.0 KiB
Bash
Executable File
60 lines
3.0 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
#
|
|
# Licensed to the Apache Software Foundation (ASF) under one or more
|
|
# contributor license agreements. See the NOTICE file distributed with
|
|
# this work for additional information regarding copyright ownership.
|
|
# The ASF licenses this file to You under the Apache License, Version 2.0
|
|
# (the "License"); you may not use this file except in compliance with
|
|
# the License. You may obtain a copy of the License at
|
|
#
|
|
# http://www.apache.org/licenses/LICENSE-2.0
|
|
#
|
|
# Unless required by applicable law or agreed to in writing, software
|
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
# See the License for the specific language governing permissions and
|
|
# limitations under the License.
|
|
#
|
|
|
|
# Starts the celeborn worker on the machine this script is executed on.
|
|
|
|
if [ -z "${CELEBORN_HOME}" ]; then
|
|
export CELEBORN_HOME="$(cd "`dirname "$0"`"/..; pwd)"
|
|
fi
|
|
|
|
. "${CELEBORN_HOME}/sbin/load-celeborn-env.sh"
|
|
|
|
if [ "$CELEBORN_WORKER_MEMORY" = "" ]; then
|
|
CELEBORN_WORKER_MEMORY="1g"
|
|
fi
|
|
|
|
if [ "$CELEBORN_WORKER_OFFHEAP_MEMORY" = "" ]; then
|
|
CELEBORN_WORKER_OFFHEAP_MEMORY="1g"
|
|
fi
|
|
|
|
CELEBORN_JAVA_OPTS="$CELEBORN_WORKER_JAVA_OPTS"
|
|
CELEBORN_JAVA_OPTS="$CELEBORN_JAVA_OPTS -Xmx$CELEBORN_WORKER_MEMORY"
|
|
CELEBORN_JAVA_OPTS="$CELEBORN_JAVA_OPTS -XX:MaxDirectMemorySize=$CELEBORN_WORKER_OFFHEAP_MEMORY"
|
|
CELEBORN_JAVA_OPTS="$CELEBORN_JAVA_OPTS -Dio.netty.tryReflectionSetAccessible=true"
|
|
CELEBORN_JAVA_OPTS="$CELEBORN_JAVA_OPTS --illegal-access=warn"
|
|
CELEBORN_JAVA_OPTS="$CELEBORN_JAVA_OPTS --add-opens=java.base/java.lang=ALL-UNNAMED"
|
|
CELEBORN_JAVA_OPTS="$CELEBORN_JAVA_OPTS --add-opens=java.base/java.lang.invoke=ALL-UNNAMED"
|
|
CELEBORN_JAVA_OPTS="$CELEBORN_JAVA_OPTS --add-opens=java.base/java.lang.reflect=ALL-UNNAMED"
|
|
CELEBORN_JAVA_OPTS="$CELEBORN_JAVA_OPTS --add-opens=java.base/java.io=ALL-UNNAMED"
|
|
CELEBORN_JAVA_OPTS="$CELEBORN_JAVA_OPTS --add-opens=java.base/java.net=ALL-UNNAMED"
|
|
CELEBORN_JAVA_OPTS="$CELEBORN_JAVA_OPTS --add-opens=java.base/java.nio=ALL-UNNAMED"
|
|
CELEBORN_JAVA_OPTS="$CELEBORN_JAVA_OPTS --add-opens=java.base/java.util=ALL-UNNAMED"
|
|
CELEBORN_JAVA_OPTS="$CELEBORN_JAVA_OPTS --add-opens=java.base/java.util.concurrent=ALL-UNNAMED"
|
|
CELEBORN_JAVA_OPTS="$CELEBORN_JAVA_OPTS --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED"
|
|
CELEBORN_JAVA_OPTS="$CELEBORN_JAVA_OPTS --add-opens=java.base/jdk.internal.misc=ALL-UNNAMED"
|
|
CELEBORN_JAVA_OPTS="$CELEBORN_JAVA_OPTS --add-opens=java.base/sun.nio.ch=ALL-UNNAMED"
|
|
CELEBORN_JAVA_OPTS="$CELEBORN_JAVA_OPTS --add-opens=java.base/sun.nio.cs=ALL-UNNAMED"
|
|
CELEBORN_JAVA_OPTS="$CELEBORN_JAVA_OPTS --add-opens=java.base/sun.security.action=ALL-UNNAMED"
|
|
CELEBORN_JAVA_OPTS="$CELEBORN_JAVA_OPTS --add-opens=java.base/sun.util.calendar=ALL-UNNAMED"
|
|
export CELEBORN_JAVA_OPTS="$CELEBORN_JAVA_OPTS"
|
|
|
|
if [ "$WORKER_INSTANCE" = "" ]; then
|
|
WORKER_INSTANCE=1
|
|
fi
|
|
|
|
exec "${CELEBORN_HOME}/sbin/celeborn-daemon.sh" start org.apache.celeborn.service.deploy.worker.Worker "$WORKER_INSTANCE" "$@" |