a23b16a93f
# 🔍 Description ## Issue References 🔗 As described. ## Describe Your Solution 🔧 G1 garbage collector is supported in Java 8+ and has become the default garbage collector in Java 17+. Use G1GC to simplify JVM options example in kyuubi-env for KYUUBI_JAVA_OPTS and KYUUBI_BEELINE_OPTS, and for better production deployment usage of higher Java version users out of box. - use G1 for garbage collector - Remove CMS and ParNew related options - Use UseGCOverheadLimit option - Use UseStringDeduplication option for G1GC - Reorder options in resources, GC mechanism, GC printing/logging ## Types of changes 🔖 - [ ] Bugfix (non-breaking change which fixes an issue) - [x] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) ## Test Plan 🧪 #### Behavior Without This Pull Request ⚰️ No behavior change. #### Behavior With This Pull Request 🎉 Uncomment the template and use the suggested default JVM options for starting Kyuubi server. <img width="1235" alt="image" src="https://github.com/apache/kyuubi/assets/1935105/73df3579-a6a8-48a5-8ea5-3fd7a54ae71b"> #### Related Unit Tests CI test. Tested starting Kyuubi server locally. --- # Checklists ## 📝 Author Self Checklist - [x] My code follows the [style guidelines](https://kyuubi.readthedocs.io/en/master/contributing/code/style.html) of this project - [x] I have performed a self-review - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html) ## 📝 Committer Pre-Merge Checklist - [x] Pull request title is okay. - [x] No license issues. - [x] Milestone correctly set? - [x] Test coverage is ok - [x] Assignees are selected. - [x] Minimum number of approvals - [x] No changes are requested **Be nice. Be informative.** Closes #5729 from bowenliang123/javaopts-useg1. Closes #5729 ad7bdc1f9 [Bowen Liang] add UseStringDeduplication option fb2ac5d63 [Bowen Liang] nit 73a687d10 [Bowen Liang] increase GCLogFileSize 5c21ff923 [Bowen Liang] use g1gc in KYUUBI_JAVA_OPTS example of kyuubi-env template Authored-by: Bowen Liang <liangbowen@gf.com.cn> Signed-off-by: Kent Yao <yao@apache.org>
69 lines
4.2 KiB
Bash
Executable File
69 lines
4.2 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
#
|
|
# Licensed to the Apache Software Foundation (ASF) under one or more
|
|
# contributor license agreements. See the NOTICE file distributed with
|
|
# this work for additional information regarding copyright ownership.
|
|
# The ASF licenses this file to You under the Apache License, Version 2.0
|
|
# (the "License"); you may not use this file except in compliance with
|
|
# the License. You may obtain a copy of the License at
|
|
#
|
|
# http://www.apache.org/licenses/LICENSE-2.0
|
|
#
|
|
# Unless required by applicable law or agreed to in writing, software
|
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
# See the License for the specific language governing permissions and
|
|
# limitations under the License.
|
|
#
|
|
#
|
|
# - JAVA_HOME Java runtime to use. By default use "java" from PATH.
|
|
#
|
|
#
|
|
# - KYUUBI_CONF_DIR Directory containing the Kyuubi configurations to use.
|
|
# (Default: $KYUUBI_HOME/conf)
|
|
# - KYUUBI_LOG_DIR Directory for Kyuubi server-side logs.
|
|
# (Default: $KYUUBI_HOME/logs)
|
|
# - KYUUBI_PID_DIR Directory stores the Kyuubi instance pid file.
|
|
# (Default: $KYUUBI_HOME/pid)
|
|
# - KYUUBI_MAX_LOG_FILES Maximum number of Kyuubi server logs can rotate to.
|
|
# (Default: 5)
|
|
# - KYUUBI_JAVA_OPTS JVM options for the Kyuubi server itself in the form "-Dx=y".
|
|
# (Default: none).
|
|
# - KYUUBI_CTL_JAVA_OPTS JVM options for the Kyuubi ctl itself in the form "-Dx=y".
|
|
# (Default: none).
|
|
# - KYUUBI_BEELINE_OPTS JVM options for the Kyuubi BeeLine in the form "-Dx=Y".
|
|
# (Default: none)
|
|
# - KYUUBI_NICENESS The scheduling priority for Kyuubi server.
|
|
# (Default: 0)
|
|
# - KYUUBI_WORK_DIR_ROOT Root directory for launching sql engine applications.
|
|
# (Default: $KYUUBI_HOME/work)
|
|
# - HADOOP_CONF_DIR Directory containing the Hadoop / YARN configuration to use.
|
|
# - YARN_CONF_DIR Directory containing the YARN configuration to use.
|
|
#
|
|
# - SPARK_HOME Spark distribution which you would like to use in Kyuubi.
|
|
# - SPARK_CONF_DIR Optional directory where the Spark configuration lives.
|
|
# (Default: $SPARK_HOME/conf)
|
|
# - FLINK_HOME Flink distribution which you would like to use in Kyuubi.
|
|
# - FLINK_CONF_DIR Optional directory where the Flink configuration lives.
|
|
# (Default: $FLINK_HOME/conf)
|
|
# - FLINK_HADOOP_CLASSPATH Required Hadoop jars when you use the Kyuubi Flink engine.
|
|
# - HIVE_HOME Hive distribution which you would like to use in Kyuubi.
|
|
# - HIVE_CONF_DIR Optional directory where the Hive configuration lives.
|
|
# (Default: $HIVE_HOME/conf)
|
|
# - HIVE_HADOOP_CLASSPATH Required Hadoop jars when you use the Kyuubi Hive engine.
|
|
#
|
|
|
|
|
|
## Examples ##
|
|
|
|
# export JAVA_HOME=/usr/jdk64/jdk1.8.0_152
|
|
# export SPARK_HOME=/opt/spark
|
|
# export FLINK_HOME=/opt/flink
|
|
# export HIVE_HOME=/opt/hive
|
|
# export FLINK_HADOOP_CLASSPATH=/path/to/hadoop-client-runtime-3.3.2.jar:/path/to/hadoop-client-api-3.3.2.jar
|
|
# export HIVE_HADOOP_CLASSPATH=${HADOOP_HOME}/share/hadoop/common/lib/commons-collections-3.2.2.jar:${HADOOP_HOME}/share/hadoop/client/hadoop-client-runtime-3.1.0.jar:${HADOOP_HOME}/share/hadoop/client/hadoop-client-api-3.1.0.jar:${HADOOP_HOME}/share/hadoop/common/lib/htrace-core4-4.1.0-incubating.jar
|
|
# export HADOOP_CONF_DIR=/usr/ndp/current/mapreduce_client/conf
|
|
# export YARN_CONF_DIR=/usr/ndp/current/yarn/conf
|
|
# export KYUUBI_JAVA_OPTS="-Xmx10g -XX:MaxMetaspaceSize=512m -XX:MaxDirectMemorySize=1024m -XX:+UseG1GC -XX:+UseStringDeduplication -XX:+UnlockDiagnosticVMOptions -XX:+UseCondCardMark -XX:+UseGCOverheadLimit -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=./logs -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintTenuringDistribution -verbose:gc -Xloggc:./logs/kyuubi-server-gc-%t.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=20M"
|
|
# export KYUUBI_BEELINE_OPTS="-Xmx2g -XX:+UseG1GC -XX:+UnlockDiagnosticVMOptions -XX:+UseCondCardMark"
|