kyuubi/docs/index.rst

.. Licensed to the Apache Software Foundation (ASF) under one or more
   contributor license agreements.  See the NOTICE file distributed with
   this work for additional information regarding copyright ownership.
   The ASF licenses this file to You under the Apache License, Version 2.0
   (the "License"); you may not use this file except in compliance with
   the License.  You may obtain a copy of the License at

..    http://www.apache.org/licenses/LICENSE-2.0

.. Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.

.. Kyuubi documentation master file, created by
   sphinx-quickstart on Wed Oct 28 14:23:28 2020.
   You can adapt this file completely to your liking, but it should at least
   contain the root `toctree` directive.

.. image:: imgs/kyuubi_logo.png
   :align: center

Welcome to Kyuubi's documentation!
==================================

Kyuubi™ is a unified multi-tenant JDBC interface for large-scale data processing and analytics, built on top of `Apache Spark™ <http://spark.apache.org/>`_.

.. image:: ./imgs/kyuubi_layers.png

In general, the complete ecosystem of Kyuubi falls into the hierarchies shown in the above figure, with each layer loosely coupled to the other.

For example, you can use Kyuubi, Spark and `Apache Iceberg <https://iceberg.apache.org/>`_ to build and manage Data Lake with pure SQL for both data processing e.g. ETL, and analytics e.g. BI.
All workloads can be done on one platform, using one copy of data, with one SQL interface.

Kyuubi provides the following features:

Multi-tenancy
-------------

Kyuubi supports the end-to-end multi-tenancy,
and this is why we want to create this project despite that the Spark `Thrift JDBC/ODBC server <http://spark.apache.org/docs/latest/sql-distributed-sql-engine.html#running-the-thrift-jdbcodbc-server>`_ already exists.

1. Supports multi-client concurrency and authentication
2. Supports one Spark application per account(SPA).
3. Supports QUEUE/NAMESPACE Access Control Lists (ACL)
4. Supports metadata & data Access Control Lists

Users who have valid accounts could use all kinds of client tools, e.g.
Hive Beeline, `HUE <https://gethue.com/>`_, `DBeaver <https://dbeaver.io/>`_,
`SQuirreL SQL Client <http://squirrel-sql.sourceforge.net/>`_, etc,
to operate with Kyuubi server concurrently.

The SPA policy makes sure 1) a user account can only get computing resource with managed ACLs, e.g.
`Queue Access Control Lists <https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/FairScheduler.html#Queue_Access_Control_Lists>`_,
from cluster managers, e.g.
`Apache Hadoop YARN <https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YARN.html>`_,
`Kubernetes (K8s) <https://kubernetes.io/>`_ to create the Spark application;
2) a user account can only access data and metadata from a storage system, e.g.
`Apache Hadoop HDFS <https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html>`_,
with permissions.

Ease of Use
------------

You only need to be familiar with Structured Query Language (SQL) and Java Database Connectivity (JDBC) to handle massive data.
It helps you focus on the design and implementation of your business system.

Run Anywhere
------------

Kyuubi can submit Spark applications to all supported cluster managers, including YARN, Mesos, Kubernetes, Standalone, and local.

The SPA policy also make it possible for you to launch different applications against different cluster managers.

High Performance
----------------

Kyuubi is built on the Apache Spark, a lightning-fast unified analytics engine.

 - **Concurrent execution**: multiple Spark applications work together
 - **Quick response**: long-running Spark applications without startup cost
 - **Optimal execution plan**: fully supports Spark SQL Catalyst Optimizer,

Authentication & Authorization
------------------------------

With strong authentication and fine-grained column/row level authorization,
Kyuubi keeps your system and data secure.

High Availability
-----------------

Kyuubi provides both high availability and load balancing solutions based on Zookeeper.

.. toctree::
   :caption: Usage Guide
   :maxdepth: 2
   :glob:

   quick_start/index
   deployment/index
   Security <security/index>
   client/index
   integrations/index
   monitor/index
   sql/index
   tools/index

.. toctree::
   :caption: Kyuubi Insider
   :maxdepth: 2

   overview/index

.. toctree::
   :caption: Contributing
   :maxdepth: 2

   develop_tools/index
   community/index

.. toctree::
   :caption: Appendix
   :maxdepth: 2

   appendix/index