apache/kyuubi
1
0
Fork 0
Code Issues Pull Requests Actions 5 Packages Projects Releases Wiki Activity

[KYUUBI #5158] Allow embedded Zookeeper binding IP address

Browse Source 
### _Why are the changes needed?_

This PR provides a new configuration `kyuubi.zookeeper.embedded.client.use.hostname` to control whether use hostname or IP address on binding, to improve the out-of-box experience of K8s case, similar to `kyuubi.frontend.connection.url.use.hostname`.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request

Closes #5158 from pan3793/zk-ip.

Closes #5158

260dae987 [Cheng Pan] nit
3ac0c430e [Cheng Pan] doc
de50f65e8 [Cheng Pan] false
890b13bd9 [Cheng Pan] Allow embedded Zookeeper binding IP address

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
This commit is contained in:
Cheng Pan 2023-08-16 01:53:51 +08:00
parent 14d0dab697
commit 4e4a4e1153
No known key found for this signature in database
GPG Key ID: 8001952629BCC75D
3 changed files with 27 additions and 13 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

25
docs/configuration/settings.md
View File

@ -453,18 +453,19 @@ You can configure the Kyuubi properties in `$KYUUBI_HOME/conf/kyuubi-defaults.co
### Zookeeper
| Key | Default | Meaning | Type | Since |
|--------------------------------------------------|--------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------|-------|
| kyuubi.zookeeper.embedded.client.port | 2181 | clientPort for the embedded ZooKeeper server to listen for client connections, a client here could be Kyuubi server, engine, and JDBC client | int | 1.2.0 |
| kyuubi.zookeeper.embedded.client.port.address | &lt;undefined&gt; | clientPortAddress for the embedded ZooKeeper server to | string | 1.2.0 |
| kyuubi.zookeeper.embedded.data.dir | embedded_zookeeper | dataDir for the embedded zookeeper server where stores the in-memory database snapshots and, unless specified otherwise, the transaction log of updates to the database. | string | 1.2.0 |
| kyuubi.zookeeper.embedded.data.log.dir | embedded_zookeeper | dataLogDir for the embedded ZooKeeper server where writes the transaction log . | string | 1.2.0 |
| kyuubi.zookeeper.embedded.directory | embedded_zookeeper | The temporary directory for the embedded ZooKeeper server | string | 1.0.0 |
| kyuubi.zookeeper.embedded.max.client.connections | 120 | maxClientCnxns for the embedded ZooKeeper server to limit the number of concurrent connections of a single client identified by IP address | int | 1.2.0 |
| kyuubi.zookeeper.embedded.max.session.timeout | 60000 | maxSessionTimeout in milliseconds for the embedded ZooKeeper server will allow the client to negotiate. Defaults to 20 times the tickTime | int | 1.2.0 |
| kyuubi.zookeeper.embedded.min.session.timeout | 6000 | minSessionTimeout in milliseconds for the embedded ZooKeeper server will allow the client to negotiate. Defaults to 2 times the tickTime | int | 1.2.0 |
| kyuubi.zookeeper.embedded.port | 2181 | The port of the embedded ZooKeeper server | int | 1.0.0 |
| kyuubi.zookeeper.embedded.tick.time | 3000 | tickTime in milliseconds for the embedded ZooKeeper server | int | 1.2.0 |
| Key | Default | Meaning | Type | Since |
|--------------------------------------------------|--------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------|-------|
| kyuubi.zookeeper.embedded.client.port | 2181 | clientPort for the embedded ZooKeeper server to listen for client connections, a client here could be Kyuubi server, engine, and JDBC client | int | 1.2.0 |
| kyuubi.zookeeper.embedded.client.port.address | &lt;undefined&gt; | clientPortAddress for the embedded ZooKeeper server to | string | 1.2.0 |
| kyuubi.zookeeper.embedded.client.use.hostname | false | When true, embedded Zookeeper prefer to bind hostname, otherwise, ip address. | boolean | 1.7.2 |
| kyuubi.zookeeper.embedded.data.dir | embedded_zookeeper | dataDir for the embedded zookeeper server where stores the in-memory database snapshots and, unless specified otherwise, the transaction log of updates to the database. | string | 1.2.0 |
| kyuubi.zookeeper.embedded.data.log.dir | embedded_zookeeper | dataLogDir for the embedded ZooKeeper server where writes the transaction log . | string | 1.2.0 |
| kyuubi.zookeeper.embedded.directory | embedded_zookeeper | The temporary directory for the embedded ZooKeeper server | string | 1.0.0 |
| kyuubi.zookeeper.embedded.max.client.connections | 120 | maxClientCnxns for the embedded ZooKeeper server to limit the number of concurrent connections of a single client identified by IP address | int | 1.2.0 |
| kyuubi.zookeeper.embedded.max.session.timeout | 60000 | maxSessionTimeout in milliseconds for the embedded ZooKeeper server will allow the client to negotiate. Defaults to 20 times the tickTime | int | 1.2.0 |
| kyuubi.zookeeper.embedded.min.session.timeout | 6000 | minSessionTimeout in milliseconds for the embedded ZooKeeper server will allow the client to negotiate. Defaults to 2 times the tickTime | int | 1.2.0 |
| kyuubi.zookeeper.embedded.port | 2181 | The port of the embedded ZooKeeper server | int | 1.0.0 |
| kyuubi.zookeeper.embedded.tick.time | 3000 | tickTime in milliseconds for the embedded ZooKeeper server | int | 1.2.0 |
## Spark Configurations

8
kyuubi-zookeeper/src/main/scala/org/apache/kyuubi/zookeeper/EmbeddedZookeeper.scala
View File

@ -42,7 +42,13 @@ class EmbeddedZookeeper extends AbstractService("EmbeddedZookeeper") {
val maxClientCnxns = conf.get(ZK_MAX_CLIENT_CONNECTIONS)
val minSessionTimeout = conf.get(ZK_MIN_SESSION_TIMEOUT)
val maxSessionTimeout = conf.get(ZK_MAX_SESSION_TIMEOUT)
host = conf.get(ZK_CLIENT_PORT_ADDRESS).getOrElse(findLocalInetAddress.getCanonicalHostName)
host = conf.get(ZK_CLIENT_PORT_ADDRESS).getOrElse {
if (conf.get(ZK_CLIENT_USE_HOSTNAME)) {
findLocalInetAddress.getCanonicalHostName
} else {
findLocalInetAddress.getHostAddress
}
}
try {
zks = new ZooKeeperServer(dataDirectory, dataDirectory, tickTime)

7
kyuubi-zookeeper/src/main/scala/org/apache/kyuubi/zookeeper/ZookeeperConf.scala
View File

@ -49,6 +49,13 @@ object ZookeeperConf {
.stringConf
.createOptional
val ZK_CLIENT_USE_HOSTNAME: ConfigEntry[Boolean] =
buildConf("kyuubi.zookeeper.embedded.client.use.hostname")
.doc("When true, embedded Zookeeper prefer to bind hostname, otherwise, ip address.")
.version("1.7.2")
.booleanConf
.createWithDefault(false)
val ZK_DATA_DIR: ConfigEntry[String] = buildConf("kyuubi.zookeeper.embedded.data.dir")
.doc("dataDir for the embedded zookeeper server where stores the in-memory database" +
" snapshots and, unless specified otherwise, the transaction log of updates to the database.")