3641d9fb0a
### Why are the changes needed? The PR fixes multiple `Pygments lexer name` issues and resolves the following warnings during the documentation build process: ``` ../kyuubi/docs/client/advanced/kerberos.md:37: WARNING: Pygments lexer name 'cmd' is not known ../kyuubi/docs/client/bi_tools/hue.md:26: WARNING: Lexing literal_block "Welcome to\n __ __ __\n /\\ \\/\\ \\ /\\ \\ __\n \\ \\ \\/'/' __ __ __ __ __ __\\ \\ \\____/\\_\\\n \\ \\ , < /\\ \\/\\ \\/\\ \\/\\ \\/\\ \\/\\ \\\\ \\ '__`\\/\\ \\\n \\ \\ \\\\`\\\\ \\ \\_\\ \\ \\ \\_\\ \\ \\ \\_\\ \\\\ \\ \\L\\ \\ \\ \\\n \\ \\_\\ \\_\\/`____ \\ \\____/\\ \\____/ \\ \\_,__/\\ \\_\\\n \\/_/\\/_/`/___/> \\/___/ \\/___/ \\/___/ \\/_/\n /\\___/\n \\/__/" as "bash" resulted in an error at token: "'". Retrying in relaxed mode. [misc.highlighting_failure] ../kyuubi/docs/client/jdbc/hive_jdbc.md:27: WARNING: Pygments lexer name 'gradle' is not known ../kyuubi/docs/client/jdbc/kyuubi_jdbc.rst:111: WARNING: Pygments lexer name 'jdbc' is not known ../kyuubi/docs/client/jdbc/kyuubi_jdbc.rst:134: WARNING: Pygments lexer name 'jdbc' is not known ../kyuubi/docs/client/jdbc/kyuubi_jdbc.rst:143: WARNING: Pygments lexer name 'jdbc' is not known ../kyuubi/docs/client/jdbc/kyuubi_jdbc.rst:163: WARNING: Pygments lexer name 'jdbc' is not known ../kyuubi/docs/connector/spark/delta_lake_with_azure_blob.rst:191: WARNING: Pygments lexer name 'log' is not known ../kyuubi/docs/deployment/hive_metastore.md:38: WARNING: Pygments lexer name 'shell script' is not known ../kyuubi/docs/deployment/hive_metastore.md:207: WARNING: Lexing literal_block "Caused by: org.apache.thrift.TApplicationException: Invalid method name: 'get_table_req'\n\tat org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:79)\n\tat org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_table_req(ThriftHiveMetastore.java:1567)\n\tat org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_table_req(ThriftHiveMetastore.java:1554)\n\tat org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:1350)\n\tat org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.getTable(SessionHiveMetaStoreClient.java:127)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:498)\n\tat org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:173)\n\tat com.sun.proxy.$Proxy37.getTable(Unknown Source)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:498)\n\tat org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke(HiveMetaStoreClient.java:2336)\n\tat com.sun.proxy.$Proxy37.getTable(Unknown Source)\n\tat org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1274)\n\t... 93 more" as "java" resulted in an error at token: "'". Retrying in relaxed mode. [misc.highlighting_failure] ../kyuubi/docs/extensions/server/authentication.rst:75: WARNING: Pygments lexer name 'property' is not known ../kyuubi/docs/extensions/server/events.rst:76: WARNING: Pygments lexer name 'property' is not known ../kyuubi/docs/monitor/logging.md:38: WARNING: Pygments lexer name 'log' is not known ../kyuubi/docs/monitor/logging.md:86: WARNING: Pygments lexer name 'log' is not known ../kyuubi/docs/monitor/logging.md:222: WARNING: Pygments lexer name 'log' is not known ../kyuubi/docs/security/kerberos.rst:104: WARNING: Pygments lexer name 'property' is not known ../kyuubi/docs/security/ldap.md:24: WARNING: Pygments lexer name 'properties example' is not known ../kyuubi/docs/security/ldap.md:40: WARNING: Pygments lexer name 'properties example' is not known ``` Supported languages: [Pygments lexers](https://pygments.org/docs/lexers) and [highlightjs](https://github.com/highlightjs/highlight.js/blob/main/SUPPORTED_LANGUAGES.md). ### How was this patch tested? Built documentation locally and checked there are related warnings. ### Was this patch authored or co-authored using generative AI tooling? No Closes #6986 from dnskr/fix-unknown-Pygments-lexer-name. Closes #6986 f5b62f52d [dnskr] [DOC] Fix multiple Pygments lexer name issues Authored-by: dnskr <dnskrv88@gmail.com> Signed-off-by: Cheng Pan <chengpan@apache.org>
120 lines
4.6 KiB
ReStructuredText
120 lines
4.6 KiB
ReStructuredText
.. Licensed to the Apache Software Foundation (ASF) under one or more
|
|
contributor license agreements. See the NOTICE file distributed with
|
|
this work for additional information regarding copyright ownership.
|
|
The ASF licenses this file to You under the Apache License, Version 2.0
|
|
(the "License"); you may not use this file except in compliance with
|
|
the License. You may obtain a copy of the License at
|
|
|
|
.. http://www.apache.org/licenses/LICENSE-2.0
|
|
|
|
.. Unless required by applicable law or agreed to in writing, software
|
|
distributed under the License is distributed on an "AS IS" BASIS,
|
|
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
See the License for the specific language governing permissions and
|
|
limitations under the License.
|
|
|
|
Configure Kyuubi to use Kerberos Authentication
|
|
===============================================
|
|
|
|
If you are deploying Kyuubi with a kerberized Hadoop cluster, it is strongly
|
|
recommended that ``kyuubi.authentication`` should be set to `KERBEROS` too.
|
|
|
|
Kerberos Overview
|
|
-----------------
|
|
|
|
Kerberos is a network authentication protocol that provides the tools of
|
|
authentication and strong cryptography over the network.
|
|
The Kerberos protocol uses strong cryptography so that a client or a server
|
|
can prove its identity to its server or client across an insecure network connection.
|
|
After a client and server have used Kerberos to prove their identity, they can
|
|
also encrypt all of their communications to assure privacy and data integrity as
|
|
they go about their business.
|
|
|
|
The Kerberos architecture is centered around a trusted authentication service
|
|
called the key distribution center, or KDC.
|
|
Users and services in a Kerberos environment are referred to as principals;
|
|
each principal shares a secret, such as a password, with the KDC.
|
|
|
|
Enable Kerberos Authentication
|
|
------------------------------
|
|
|
|
To enable the Kerberos authentication method, we need to
|
|
|
|
Create a Kerberos principal and keytab
|
|
**************************************
|
|
|
|
You can use the following commands in a Linux-based Kerberos environment to set up
|
|
the identity and update the keytab file:
|
|
|
|
The ``kyuubi.keytab`` file must be owned and readable by the Linux login user.
|
|
|
|
.. code-block::
|
|
|
|
# kadmin
|
|
: addprinc -randkey superuser/FQDN@REALM
|
|
: ktadd -k ./kyuubi.keytab superuser/FQDN@REALM
|
|
|
|
.. note:: A widespread use case of kyuubi is to replace HiveServer2/Hive QL with
|
|
Kyuubi/Spark SQL. If an existing HiveServer2 environment is already there,
|
|
copying the environment and reusing the keytab and principal of HiveServer2 is
|
|
a convenient way.
|
|
|
|
Enable `Hadoop Impersonation`_
|
|
*******************************
|
|
|
|
If background cluster is also an kerberized Hadoop cluster, we need to enable the
|
|
impersonation capability of the superuser we use to start kyuubi server.
|
|
|
|
You can configure proxy user using properties ``hadoop.proxyuser.$superuser.hosts``
|
|
along with either or both of ``hadoop.proxyuser.$superuser.groups`` and ``hadoop.proxyuser.$superuser.users``.
|
|
|
|
For instance, by specifying as below in ``core-site.xml``, the ``superuser`` named ``admin`` can connect
|
|
only from ``host1`` and ``host2`` to impersonate a user belonging to ``group1`` and ``group2``.
|
|
|
|
.. code-block:: xml
|
|
|
|
<property>
|
|
<name>hadoop.proxyuser.admin.hosts</name>
|
|
<value>host1,host2</value>
|
|
</property>
|
|
<property>
|
|
<name>hadoop.proxyuser.admin.groups</name>
|
|
<value>group1,group2</value>
|
|
</property>
|
|
|
|
|
|
Here,
|
|
|
|
- ``admin`` is the principal(short name) used to start kyuubi servers
|
|
- ``host1`` and ``host2`` are node addresses of kyuubi servers
|
|
- ``group1`` and ``group2`` are groups of client users
|
|
|
|
.. note:: These configurations need to be configured in the Hadoop cluster
|
|
and refreshed to take effect.
|
|
|
|
.. note:: If you are using the keytab of existing HiveServer2, this step can
|
|
also be omitted
|
|
|
|
Configure the authentication properties
|
|
***************************************
|
|
|
|
Configure the following properties to ``$KYUUBI_HOME/conf/kyuubi-defaults.conf``
|
|
on each node where kyuubi server is installed.
|
|
|
|
.. code-block:: properties
|
|
|
|
kyuubi.authentication=KERBEROS
|
|
kyuubi.kinit.principal=superuser/FQDN@REALM
|
|
kyuubi.kinit.keytab=/path/to/kyuubi.keytab
|
|
|
|
These `configurations`_ also need to be set to enable KERBEROS authentication.
|
|
|
|
Refresh all the kyuubi server instances
|
|
***************************************
|
|
|
|
Restart all the kyuubi server instances or `Refresh Configurations`_ to activate the settings.
|
|
|
|
.. _Hadoop Impersonation: https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/Superusers.html
|
|
.. _configurations: ../client/advanced/kerberos.html
|
|
.. _Refresh Configurations: ../tools/kyuubi-admin.html#refresh-config
|