84928184fc
### Why are the changes needed? The current mechanism for handling operation timeouts in Kyuubi creates a new `ScheduledExecutorService` with a dedicated thread for each operation. In scenarios with a large number of concurrent operations, this results in excessive thread creation, which consumes substantial system resources and may adversely affect server performance and stability. This PR introduces a shared `ScheduledThreadPool` within the Operation Manager to centrally schedule operation timeouts. This approach avoids the overhead of creating an excessive number of threads, thereby reducing the system load. Additionally, both the pool size and thread keep-alive time are configurable via the `OPERATION_TIMEOUT_POOL_SIZE` and `OPERATION_TIMEOUT_POOL_KEEPALIVE_TIME` parameters. ### How was this patch tested? A new unit test for `newDaemonScheduledThreadPool` was added to `ThreadUtilsSuite.scala`. Furthermore, a dedicated `TimeoutSchedulerSuite` was introduced to verify operation timeout behavior. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #7121 from wangzhigang1999/master. Closes #7121 df7688dbf [wangzhigang] Refactor timeout management configuration and improve documentation 2b03b1e68 [wangzhigang] Remove deprecated `ThreadPoolTimeoutExecutor` class following refactor of operation timeout management. 52a8a516a [wangzhigang] Refactor operation timeout management to use per-OperationManager scheduler 7e46d47f8 [wangzhigang] Refactor timeout management by introducing ThreadPoolTimeoutExecutor f7f10881a [wangzhigang] Add operation timeout management with ThreadPoolTimeoutExecutor d8cd6c7d4 [wangzhigang] Update .gitignore to exclude .bloop and .metals directories Lead-authored-by: wangzhigang <wangzhigang1999@live.cn> Co-authored-by: wangzhigang <wzg443064@alibaba-inc.com> Signed-off-by: Cheng Pan <chengpan@apache.org>
92 lines
1.9 KiB
Plaintext
92 lines
1.9 KiB
Plaintext
#
|
|
# Licensed to the Apache Software Foundation (ASF) under one or more
|
|
# contributor license agreements. See the NOTICE file distributed with
|
|
# this work for additional information regarding copyright ownership.
|
|
# The ASF licenses this file to You under the Apache License, Version 2.0
|
|
# (the "License"); you may not use this file except in compliance with
|
|
# the License. You may obtain a copy of the License at
|
|
#
|
|
# http://www.apache.org/licenses/LICENSE-2.0
|
|
#
|
|
# Unless required by applicable law or agreed to in writing, software
|
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
# See the License for the specific language governing permissions and
|
|
# limitations under the License.
|
|
#
|
|
|
|
*#*#
|
|
*.#*
|
|
*.db
|
|
*.iml
|
|
*.ipr
|
|
*.iws
|
|
*.pyc
|
|
*.pyo
|
|
*.swp
|
|
*~
|
|
.DS_Store
|
|
.bloop/
|
|
.metals/
|
|
.cache
|
|
.classpath
|
|
.ensime
|
|
.ensime_cache/
|
|
.ensime_lucene
|
|
.generated-mima*
|
|
.vscode/
|
|
.idea/
|
|
.idea_modules/
|
|
.project
|
|
.pydevproject
|
|
.python-version
|
|
.scala_dependencies
|
|
.settings
|
|
build/apache-maven*
|
|
build/release/tmp
|
|
build/release/*.txt
|
|
build/scala*
|
|
build/test
|
|
target/
|
|
dist/
|
|
apache-kyuubi-*-bin*
|
|
*.gz
|
|
logs/
|
|
pid/
|
|
local/
|
|
out/
|
|
hs_err_pid*
|
|
spark-warehouse/
|
|
metastore_db
|
|
derby.log
|
|
rest-audit.log
|
|
**/.flattened-pom.xml
|
|
**/dependency-reduced-pom.xml
|
|
metrics/
|
|
/kyuubi-ha/embedded_zookeeper/
|
|
embedded_zookeeper/
|
|
/externals/kyuubi-spark-sql-engine/operation_logs/
|
|
/externals/kyuubi-spark-sql-engine/engine_operation_logs/
|
|
/externals/kyuubi-spark-sql-engine/spark-warehouse/
|
|
/work/
|
|
/upload/
|
|
/docs/_build/
|
|
/kyuubi-common/metrics/
|
|
/kyuubi-server/metrics/
|
|
**/operation_logs/
|
|
**/server_operation_logs/
|
|
**/engine_operation_logs/
|
|
conf/log4j2.xml
|
|
conf/kyuubi-defaults.conf
|
|
conf/kyuubi-env.sh
|
|
.java-version
|
|
|
|
# For Antlr
|
|
kyuubi-server/gen
|
|
kyuubi-server/src/main/antlr4/org/apache/kyuubi/sql/*.tokens
|
|
kyuubi-server/src/main/antlr4/org/apache/kyuubi/sql/gen/
|
|
|
|
# For draw.io
|
|
.$*.bkp
|
|
.$*.dtmp
|