apache/kyuubi
1
0
Fork 0
Code Issues Pull Requests Actions 5 Packages Projects Releases Wiki Activity

[KYUUBI #7158] Spark engine respects session-level idle timeout threshold

Browse Source 
### Why are the changes needed?
Fixes the same class of issue as
https://github.com/apache/kyuubi/pull/7138

Previously, `sessionIdleTimeoutThreshold` was initialized only once during session creation using `sessionManager.getConf`, preventing dynamic updates when clients pass new configurations during connection.

we now:
- Allow clients to set session-specific kyuubi.session.idle.timeout` during connection
- Dynamically adjust idle timeout per session
- Prevent connection pile-up by timely recycling idle sessions

Closes #7158 from 1358035421/lc/sessio_idle_timeout_threshold.

Closes #7158

abe513eed [liangzhaoyuan] fix review comments
3face844a [liangzhaoyuan] Use per-session idle timeout threshold instead of global sessionManager's value

Authored-by: liangzhaoyuan <lwlzyl19940916@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
This commit is contained in:
liangzhaoyuan 2025-08-04 11:28:10 +08:00 committed by Cheng Pan
parent 0b4b5cabc4
commit 9a50bfa814
No known key found for this signature in database
GPG Key ID: 8001952629BCC75D
1 changed files with 8 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/session/SparkSessionImpl.scala vendored
View File

@ -24,6 +24,7 @@ import org.apache.spark.sql.{AnalysisException, SparkSession}
import org.apache.spark.ui.SparkUIUtils.formatDuration
import org.apache.kyuubi.KyuubiSQLException
import org.apache.kyuubi.config.KyuubiConf.SESSION_IDLE_TIMEOUT
import org.apache.kyuubi.config.KyuubiReservedKeys.KYUUBI_SESSION_HANDLE_KEY
import org.apache.kyuubi.engine.spark.events.SessionEvent
import org.apache.kyuubi.engine.spark.operation.SparkSQLOperationManager
@ -57,6 +58,13 @@ class SparkSessionImpl(
}
}
override val sessionIdleTimeoutThreshold: Long = {
conf.get(SESSION_IDLE_TIMEOUT.key)
.map(_.toLong)
.getOrElse(
sessionManager.getConf.get(SESSION_IDLE_TIMEOUT))
}
private val sessionEvent = SessionEvent(this)
override def open(): Unit = {