apache/kyuubi
1
0
Fork 0
Code Issues Pull Requests Actions 5 Packages Projects Releases Wiki Activity

[KYUUBI #2156][FOLLOWUP] Fix configuration format in document

Browse Source 
### _Why are the changes needed?_

fix #2157 cofiguration format error

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #2165 from jiaoqingbo/2156-followup.

Closes #2156

9862ddb2 [jiaoqingbo] [KYUUBI #2156][FOLLOWUP] Fix configuration format in document
47157c5c [jiaoqingbo] Merge branch 'master' into 2156-followup
9c89c081 [jiaoqingbo] [KYUUBI #2156][FOLLOWUP]fix cofiguration format
46084328 [jiaoqingbo] Merge branch 'master' into 2156
96a22e59 [jiaoqingbo] add link to doc
dc11a214 [jiaoqingbo] [KYUUBI #2156] Change log to reflect exactly why getting token failed

Authored-by: jiaoqingbo <1178404354@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
This commit is contained in:
jiaoqingbo 2022-03-18 16:44:24 +08:00 committed by Cheng Pan
parent f65db03401
commit 62f685ff69
No known key found for this signature in database
GPG Key ID: 8001952629BCC75D
3 changed files with 24 additions and 24 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

20
docs/security/authentication.md
View File

@ -36,16 +36,16 @@ In `$KYUUBI_HOME/conf/kyuubi-defaults.conf`, specify `kyuubi.authentication` to
Key | Default | Meaning | Since
--- | --- | --- | ---
kyuubi\.authentication|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>NONE</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Client authentication types.<ul> <li>NOSASL: raw transport.</li> <li>NONE: no authentication check.</li> <li>KERBEROS: Kerberos/GSSAPI authentication.</li> <li>LDAP: Lightweight Directory Access Protocol authentication.</li></ul></div>|<div style='width: 20pt'>1.0.0</div>
<code>kyuubi.authentication</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>NONE</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Client authentication types.<ul> <li>NOSASL: raw transport.</li> <li>NONE: no authentication check.</li> <li>KERBEROS: Kerberos/GSSAPI authentication.</li> <li>LDAP: Lightweight Directory Access Protocol authentication.</li></ul></div>|<div style='width: 20pt'>1.0.0</div>
Key | Default | Meaning | Since
--- | --- | --- | ---
kyuubi\.authentication|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>NONE</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Client authentication types.<ul> <li>NOSASL: raw transport.</li> <li>NONE: no authentication check.</li> <li>KERBEROS: Kerberos/GSSAPI authentication.</li> <li>LDAP: Lightweight Directory Access Protocol authentication.</li></ul></div>|<div style='width: 20pt'>1.0.0</div>
kyuubi\.authentication<br>\.ldap\.base\.dn|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>&lt;undefined&gt;</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>LDAP base DN.</div>|<div style='width: 20pt'>1.0.0</div>
kyuubi\.authentication<br>\.ldap\.domain|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>&lt;undefined&gt;</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>LDAP base DN.</div>|<div style='width: 20pt'>1.0.0</div>
kyuubi\.authentication<br>\.ldap\.url|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>&lt;undefined&gt;</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>SPACE character separated LDAP connection URL(s).</div>|<div style='width: 20pt'>1.0.0</div>
kyuubi\.authentication<br>\.sasl\.qop|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>auth</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Sasl QOP enable higher levels of protection for Kyuubi communication with clients.<ul> <li>auth - authentication only (default)</li> <li>auth-int - authentication plus integrity protection</li> <li>auth-conf - authentication plus integrity and confidentiality protection. This is applicable only if Kyuubi is configured to use Kerberos authentication.</li> </ul></div>|<div style='width: 20pt'>1.0.0</div>
<code>kyuubi.authentication</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>NONE</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Client authentication types.<ul> <li>NOSASL: raw transport.</li> <li>NONE: no authentication check.</li> <li>KERBEROS: Kerberos/GSSAPI authentication.</li> <li>LDAP: Lightweight Directory Access Protocol authentication.</li></ul></div>|<div style='width: 20pt'>1.0.0</div>
<code>kyuubi.authentication.ldap.base.dn</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>&lt;undefined&gt;</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>LDAP base DN.</div>|<div style='width: 20pt'>1.0.0</div>
<code>kyuubi.authentication.ldap.domain</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>&lt;undefined&gt;</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>LDAP base DN.</div>|<div style='width: 20pt'>1.0.0</div>
<code>kyuubi.authentication.ldap.url</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>&lt;undefined&gt;</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>SPACE character separated LDAP connection URL(s).</div>|<div style='width: 20pt'>1.0.0</div>
<code>kyuubi.authentication.sasl.qop</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>auth</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Sasl QOP enable higher levels of protection for Kyuubi communication with clients.<ul> <li>auth - authentication only (default)</li> <li>auth-int - authentication plus integrity protection</li> <li>auth-conf - authentication plus integrity and confidentiality protection. This is applicable only if Kyuubi is configured to use Kerberos authentication.</li> </ul></div>|<div style='width: 20pt'>1.0.0</div>
#### Using KERBEROS
@ -64,10 +64,10 @@ Following configurations also need to be set to enable KERBEROS authentication:
Key | Default | Meaning | Since
--- | --- | --- | ---
kyuubi\.kinit<br>\.principal|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>&lt;undefined&gt;</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Name of the Kerberos principal.</div>|<div style='width: 20pt'>1.0.0</div>
kyuubi\.kinit\.keytab|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>&lt;undefined&gt;</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Location of Kyuubi server's keytab.</div>|<div style='width: 20pt'>1.0.0</div>
kyuubi\.kinit\.interval|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>PT1H</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>How often will Kyuubi server run `kinit -kt [keytab] [principal]` to renew the local Kerberos credentials cache</div>|<div style='width: 20pt'>1.0.0</div>
kyuubi\.kinit\.max<br>\.attempts|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>10</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>How many times will `kinit` process retry</div>|<div style='width: 20pt'>1.0.0</div>
<code>kyuubi.kinit.principal</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>&lt;undefined&gt;</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Name of the Kerberos principal.</div>|<div style='width: 20pt'>1.0.0</div>
<code>kyuubi.kinit.keytab</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>&lt;undefined&gt;</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Location of Kyuubi server's keytab.</div>|<div style='width: 20pt'>1.0.0</div>
<code>kyuubi.kinit.interval</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>PT1H</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>How often will Kyuubi server run `kinit -kt [keytab] [principal]` to renew the local Kerberos credentials cache</div>|<div style='width: 20pt'>1.0.0</div>
<code>kyuubi.kinit.max.attempts</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>10</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>How many times will `kinit` process retry</div>|<div style='width: 20pt'>1.0.0</div>
Please refer to [Kinit Auxiliary Service](kinit.html) to get configuration steps.

20
docs/security/hadoop_credentials_manager.md
View File

@ -70,11 +70,11 @@ Hadoop Credentials Manager is initialized when Kyuubi server starts.
Key | Default | Meaning | Type | Since
--- | --- | --- | --- | ---
kyuubi\.credentials<br>\.hadoopfs\.enabled|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>true</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>Whether to renew Hadoop filesystem delegation tokens</div>|<div style='width: 30pt'>boolean</div>|<div style='width: 20pt'>1.4.0</div>
kyuubi\.credentials<br>\.hadoopfs\.uris|<div style='width: 65pt;word-wrap: break-word;white-space: normal'></div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>Extra Hadoop filesystem URIs for which to request delegation tokens. The filesystem that hosts fs.defaultFS does not need to be listed here.</div>|<div style='width: 30pt'>seq</div>|<div style='width: 20pt'>1.4.0</div>
kyuubi\.credentials<br>\.hive\.enabled|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>true</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>Whether to renew Hive metastore delegation token</div>|<div style='width: 30pt'>boolean</div>|<div style='width: 20pt'>1.4.0</div>
kyuubi\.credentials<br>\.renewal\.interval|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>PT1H</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>How often Kyuubi renews one user's delegation tokens</div>|<div style='width: 30pt'>duration</div>|<div style='width: 20pt'>1.4.0</div>
kyuubi\.credentials<br>\.renewal\.retry\.wait|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>PT1M</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>How long to wait before retrying to fetch new credentials after a failure.</div>|<div style='width: 30pt'>duration</div>|<div style='width: 20pt'>1.4.0</div>
<code>kyuubi.credentials.hadoopfs.enabled</code>|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>true</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>Whether to renew Hadoop filesystem delegation tokens</div>|<div style='width: 30pt'>boolean</div>|<div style='width: 20pt'>1.4.0</div>
<code>kyuubi.credentials.hadoopfs.uris</code>|<div style='width: 65pt;word-wrap: break-word;white-space: normal'></div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>Extra Hadoop filesystem URIs for which to request delegation tokens. The filesystem that hosts fs.defaultFS does not need to be listed here.</div>|<div style='width: 30pt'>seq</div>|<div style='width: 20pt'>1.4.0</div>
<code>kyuubi.credentials.hive.enabled</code>|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>true</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>Whether to renew Hive metastore delegation token</div>|<div style='width: 30pt'>boolean</div>|<div style='width: 20pt'>1.4.0</div>
<code>kyuubi.credentials.renewal.interval</code>|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>PT1H</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>How often Kyuubi renews one user's delegation tokens</div>|<div style='width: 30pt'>duration</div>|<div style='width: 20pt'>1.4.0</div>
<code>kyuubi.credentials.renewal.retry.wait</code>|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>PT1M</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>How long to wait before retrying to fetch new credentials after a failure.</div>|<div style='width: 30pt'>duration</div>|<div style='width: 20pt'>1.4.0</div>
### Required Security Configs
@ -83,8 +83,8 @@ The necessary configurations for hdfs and hive to obtain delegation token are as
Key | Meaning | value
--- | --- | ---
<div style='width: 40pt;word-wrap: break-word;white-space: normal'>hadoop.security.authentication</div>|<div style='width: 40pt;word-wrap: break-word;white-space: normal'>Set the authentication for the cluster</div>|<div style='width: 40pt;word-wrap: break-word;white-space: normal'>kerberos</div>
<div style='width: 40pt;word-wrap: break-word;white-space: normal'>hive.metastore.uris</div>|<div style='width: 40pt;word-wrap: break-word;white-space: normal'>URI for client to contact metastore server</div>|<div style='width: 40pt;word-wrap: break-word;white-space: normal'>thrift://{metastoreHost}:{metastorePort}}</div>
<div style='width: 40pt;word-wrap: break-word;white-space: normal'>hive.metastore.sasl.enabled</div>|<div style='width: 40pt;word-wrap: break-word;white-space: normal'>If true, the metastore thrift interface will be secured with SASL.Clients must authenticate with Kerberos.</div>|<div style='width: 40pt;word-wrap: break-word;white-space: normal'>true</div>
<div style='width: 40pt;word-wrap: break-word;white-space: normal'>hive.metastore.kerberos.principal</div>|<div style='width: 40pt;word-wrap: break-word;white-space: normal'>The service principal for the metastore thrift server. The special string _HOST will be replaced automatically with the correct host name.</div>|<div style='width: 40pt;word-wrap: break-word;white-space: normal'>for example hive/_HOST@${realm}</div>
<div style='width: 40pt;word-wrap: break-word;white-space: normal'>hive.metastore.kerberos.keytab.file</div>|<div style='width: 40pt;word-wrap: break-word;white-space: normal'>The path to the Kerberos Keytab file containing the metastore thrift server's service principal.</div>|<div style='width: 40pt;word-wrap: break-word;white-space: normal'>for example /etc/security/keytabs/hive.service.keytab</div>
<code>hadoop.security.authentication</code>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>Set the authentication for the cluster</div>|<div style='width: 120pt;word-wrap: break-word;white-space: normal'>kerberos</div>
<code>hive.metastore.uris</code>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>URI for client to contact metastore server</div>|<div style='width: 120pt;word-wrap: break-word;white-space: normal'>thrift://{metastoreHost}:{metastorePort}}</div>
<code>hive.metastore.sasl.enabled</code>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>If true, the metastore thrift interface will be secured with SASL.Clients must authenticate with Kerberos.</div>|<div style='width: 120pt;word-wrap: break-word;white-space: normal'>true</div>
<code>hive.metastore.kerberos.principal</code>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>The service principal for the metastore thrift server. The special string _HOST will be replaced automatically with the correct host name.</div>|<div style='width: 120pt;word-wrap: break-word;white-space: normal'>for example hive/_HOST@${realm}</div>
<code>hive.metastore.kerberos.keytab.file</code>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>The path to the Kerberos Keytab file containing the metastore thrift server's service principal.</div>|<div style='width: 120pt;word-wrap: break-word;white-space: normal'>for example /etc/security/keytabs/hive.service.keytab</div>

8
docs/security/kinit.md
View File

@ -77,10 +77,10 @@ They are valid for relatively short period. So, we always need to refresh it for
Key | Default | Meaning | Since
--- | --- | --- | ---
kyuubi\.kinit<br>\.principal|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>&lt;undefined&gt;</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Name of the Kerberos principal.</div>|<div style='width: 20pt'>1.0.0</div>
kyuubi\.kinit\.keytab|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>&lt;undefined&gt;</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Location of Kyuubi server's keytab.</div>|<div style='width: 20pt'>1.0.0</div>
kyuubi\.kinit\.interval|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>PT1H</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>How often will Kyuubi server run `kinit -kt [keytab] [principal]` to renew the local Kerberos credentials cache</div>|<div style='width: 20pt'>1.0.0</div>
kyuubi\.kinit\.max<br>\.attempts|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>10</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>How many times will `kinit` process retry</div>|<div style='width: 20pt'>1.0.0</div>
<code>kyuubi.kinit.principal</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>&lt;undefined&gt;</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Name of the Kerberos principal.</div>|<div style='width: 20pt'>1.0.0</div>
<code>kyuubi.kinit.keytab</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>&lt;undefined&gt;</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Location of Kyuubi server's keytab.</div>|<div style='width: 20pt'>1.0.0</div>
<code>kyuubi.kinit.interval</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>PT1H</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>How often will Kyuubi server run `kinit -kt [keytab] [principal]` to renew the local Kerberos credentials cache</div>|<div style='width: 20pt'>1.0.0</div>
<code>kyuubi.kinit.max.attempts</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>10</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>How many times will `kinit` process retry</div>|<div style='width: 20pt'>1.0.0</div>
When working with a Kerberos-enabled Hadoop cluster, we should ensure that `hadoop.security.authentication`
is set to `KERBEROS` in `$HADOOP_CONF_DIR/core-site.xml` or `$KYUUBI_HOME/conf/kyuubi-defaults.conf`.