Refine documentaion architecture
This commit is contained in:
Kent Yao
Kent Yao
parent
ca1b452e9a
commit
aaaeddeb48
@ -1,10 +1,10 @@
|
||||
<script async defer src="https://buttons.github.io/buttons.js"></script>
|
||||
|
||||
# Community [](https://gitter.im/kyuubi-on-spark/Lobby?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge)
|
||||
# <a class="github-button" href="https://github.com/yaooqinn/kyuubi/subscription" data-color-scheme="no-preference: light; light: dark; dark: light;" data-icon="octicon-eye" data-show-count="true" aria-label="Watch yaooqinn/kyuubi on GitHub">Watch</a>
|
||||
# <a class="github-button" href="https://github.com/yaooqinn/kyuubi" data-color-scheme="no-preference: light; light: dark; dark: light;" data-icon="octicon-star" data-show-count="true" aria-label="Star yaooqinn/kyuubi on GitHub">Star</a>
|
||||
# <a class="github-button" href="https://github.com/yaooqinn/kyuubi/fork" data-color-scheme="no-preference: light; light: dark; dark: light;" data-icon="octicon-repo-forked" data-show-count="true" aria-label="Fork yaooqinn/kyuubi on GitHub">Fork</a>
|
||||
# <a class="github-button" href="https://github.com/yaooqinn/kyuubi/issues" data-color-scheme="no-preference: light; light: dark; dark: light;" data-icon="octicon-issue-opened" data-show-count="true" aria-label="Issue yaooqinn/kyuubi on GitHub">Issue</a>
|
||||
# <a class="github-button" href="https://github.com/yaooqinn/kyuubi/releases" data-color-scheme="no-preference: light; light: dark; dark: light;" data-icon="octicon-download" aria-label="Download yaooqinn/kyuubi on GitHub">Download</a>
|
||||
# <a class="github-button" href="https://github.com/yaooqinn" data-color-scheme="no-preference: light; light: dark; dark: light;" data-show-count="true" aria-label="Follow @yaooqinn on GitHub">Follow @yaooqinn</a>
|
||||
# <a class="github-button" href="https://github.com/sponsors/yaooqinn" data-color-scheme="no-preference: light; light: dark; dark: light;" data-icon="octicon-heart" aria-label="Sponsor @yaooqinn on GitHub">Sponsor</a>
|
||||
<a class="github-button" href="https://github.com/yaooqinn/kyuubi/subscription" data-color-scheme="no-preference: light; light: dark; dark: light;" data-icon="octicon-eye" data-show-count="true" aria-label="Watch yaooqinn/kyuubi on GitHub">Watch</a>
|
||||
<a class="github-button" href="https://github.com/yaooqinn/kyuubi" data-color-scheme="no-preference: light; light: dark; dark: light;" data-icon="octicon-star" data-show-count="true" aria-label="Star yaooqinn/kyuubi on GitHub">Star</a>
|
||||
<a class="github-button" href="https://github.com/yaooqinn/kyuubi/fork" data-color-scheme="no-preference: light; light: dark; dark: light;" data-icon="octicon-repo-forked" data-show-count="true" aria-label="Fork yaooqinn/kyuubi on GitHub">Fork</a>
|
||||
<a class="github-button" href="https://github.com/yaooqinn/kyuubi/issues" data-color-scheme="no-preference: light; light: dark; dark: light;" data-icon="octicon-issue-opened" data-show-count="true" aria-label="Issue yaooqinn/kyuubi on GitHub">Issue</a>
|
||||
<a class="github-button" href="https://github.com/yaooqinn/kyuubi/releases" data-color-scheme="no-preference: light; light: dark; dark: light;" data-icon="octicon-download" aria-label="Download yaooqinn/kyuubi on GitHub">Download</a>
|
||||
<a class="github-button" href="https://github.com/yaooqinn" data-color-scheme="no-preference: light; light: dark; dark: light;" data-show-count="true" aria-label="Follow @yaooqinn on GitHub">Follow @yaooqinn</a>
|
||||
<a class="github-button" href="https://github.com/sponsors/yaooqinn" data-color-scheme="no-preference: light; light: dark; dark: light;" data-icon="octicon-heart" aria-label="Sponsor @yaooqinn on GitHub">Sponsor</a>
|
||||
|
||||
@ -10,16 +10,95 @@
|
||||
Welcome to Kyuubi's documentation!
|
||||
==================================
|
||||
|
||||
Kyuubi™ is a unified multi-tenant JDBC interface for large-scale data processing and analytics, built on top of `Apache Spark™ <http://spark.apache.org/>`_.
|
||||
|
||||
.. image:: ./imgs/kyuubi_layers.*
|
||||
|
||||
In general, the complete ecosystem of Kyuubi falls into the hierarchies shown in the above figure, with each layer loosely coupled to the other.
|
||||
|
||||
For example, you can use Kyuubi, Spark and `Apache Iceberg <https://iceberg.apache.org/>`_ to build and manage Data Lake with pure SQL for both data processing e.g. ETL, and analytics e.g. BI.
|
||||
All workloads can be done on one platform, using one copy of data, with one SQL interface.
|
||||
|
||||
Kyuubi provides the following features:
|
||||
|
||||
Multi-tenancy
|
||||
-------------
|
||||
|
||||
Kyuubi supports the end-to-end multi-tenancy,
|
||||
and this is why we want to create this project despite that the Spark `Thrift JDBC/ODBC server <http://spark.apache.org/docs/latest/sql-distributed-sql-engine.html#running-the-thrift-jdbcodbc-server>`_ already exists.
|
||||
|
||||
1. Supports multi-client concurrency and authentication
|
||||
2. Supports one Spark application per account(SPA).
|
||||
3. Supports QUEUE/NAMESPACE Access Control Lists (ACL)
|
||||
4. Supports metadata & data Access Control Lists
|
||||
|
||||
Users who have valid accounts could use all kinds of client tools, e.g.
|
||||
Hive Beeline, `HUE <https://gethue.com/>`_, `DBeaver <https://dbeaver.io/>`_,
|
||||
`SQuirreL SQL Client <http://squirrel-sql.sourceforge.net/>`_, etc,
|
||||
to operate with Kyuubi server concurrently.
|
||||
|
||||
The SPA policy makes sure 1) a user account can only get computing resource with managed ACLs, e.g.
|
||||
`Queue Access Control Lists <https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/FairScheduler.html#Queue_Access_Control_Lists>`_,
|
||||
from cluster managers, e.g.
|
||||
`Apache Hadoop YARN <https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YARN.html>`_,
|
||||
`Kubernetes (K8s) <https://kubernetes.io/>`_ to create the Spark application;
|
||||
2) a user account can only access data and metadata from a storage system, e.g.
|
||||
`Apache Hadoop HDFS <https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html>`_,
|
||||
with permissions.
|
||||
|
||||
Ease of Use
|
||||
------------
|
||||
|
||||
You only need to be familiar with Structured Query Language (SQL) and Java Database Connectivity (JDBC) to handle massive data.
|
||||
It helps you focus on the design and implementation of your business system.
|
||||
|
||||
Run Anywhere
|
||||
------------
|
||||
|
||||
Kyuubi can submit Spark applications to all supported cluster managers, including YARN, Mesos, Kubernetes, Standalone, and local.
|
||||
|
||||
The SPA policy also make it possible for you to launch different applications against different cluster managers.
|
||||
|
||||
High Performance
|
||||
----------------
|
||||
|
||||
Kyuubi is built on the Apache Spark, a lightning-fast unified analytics engine.
|
||||
|
||||
- **Concurrent execution**: multiple Spark applications work together
|
||||
- **Quick response**: long-running Spark applications without startup cost
|
||||
- **Optimal execution plan**: fully supports Spark SQL Catalyst Optimizer,
|
||||
|
||||
Authentication & Authorization
|
||||
------------------------------
|
||||
|
||||
With strong authentication and fine-grained column/row level authorization,
|
||||
Kyuubi keeps your system and data secure.
|
||||
|
||||
High Availability
|
||||
-----------------
|
||||
|
||||
Kyuubi provides both high availability and load balancing solutions based on Zookeeper.
|
||||
|
||||
.. toctree::
|
||||
:caption: Usage Guide
|
||||
:maxdepth: 2
|
||||
:glob:
|
||||
|
||||
quick_start/index
|
||||
Overview <overview/index>
|
||||
Deploy <deployment/index>
|
||||
Security <security/index>
|
||||
client/index
|
||||
integrations/index
|
||||
|
||||
.. toctree::
|
||||
:caption: Kyuubi Insider
|
||||
:maxdepth: 2
|
||||
|
||||
overview/index
|
||||
|
||||
.. toctree::
|
||||
:caption: Contributing
|
||||
:maxdepth: 2
|
||||
|
||||
tools/index
|
||||
community/index
|
||||
|
||||
|
||||
@ -8,6 +8,5 @@ Overview
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
summary
|
||||
Architecture <architecture>
|
||||
kyuubi_vs_hive
|
||||
|
||||
@ -1,64 +0,0 @@
|
||||
# Kyuubi™
|
||||
|
||||
Kyuubi™ is a unified multi-tenant JDBC interface for large-scale data processing and analytics, built on top of [Apache Spark™](http://spark.apache.org/).
|
||||
|
||||
|
||||
|
||||
In general, the complete ecosystem of Kyuubi falls into the hierarchies shown in the above figure, with each layer loosely coupled to the other.
|
||||
|
||||
For example, you can use Kyuubi, Spark and [Apache Iceberg](https://iceberg.apache.org/) to build and manage Data Lake with pure SQL for both data processing e.g. ETL, and analytics e.g. BI.
|
||||
All workloads can be done on one platform, using one copy of data, with one SQL interface.
|
||||
|
||||
Kyuubi provides the following features:
|
||||
|
||||
## Multi-tenancy
|
||||
|
||||
Kyuubi supports the end-to-end multi-tenancy,
|
||||
and this is why we want to create this project despite that the Spark [Thrift JDBC/ODBC server](http://spark.apache.org/docs/latest/sql-distributed-sql-engine.html#running-the-thrift-jdbcodbc-server) already exists.
|
||||
|
||||
1. Supports multi-client concurrency and authentication
|
||||
2. Supports one Spark application per account(SPA).
|
||||
3. Supports QUEUE/NAMESPACE Access Control Lists (ACL)
|
||||
4. Supports metadata & data Access Control Lists
|
||||
|
||||
Users who have valid accounts could use all kinds of client tools, e.g.
|
||||
Hive Beeline, [HUE](https://gethue.com/), [DBeaver](https://dbeaver.io/),
|
||||
[SQuirreL SQL Client](http://squirrel-sql.sourceforge.net/), etc,
|
||||
to operate with Kyuubi server concurrently.
|
||||
|
||||
The SPA policy makes sure 1) a user account can only get computing resource with managed ACLs, e.g.
|
||||
[Queue Access Control Lists](https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/FairScheduler.html#Queue_Access_Control_Lists),
|
||||
from cluster managers, e.g.
|
||||
[Apache Hadoop YARN](https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YARN.html),
|
||||
[Kubernetes (K8s)](https://kubernetes.io/) to create the Spark application;
|
||||
2) a user account can only access data and metadata from a storage system, e.g.
|
||||
[Apache Hadoop HDFS](https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html),
|
||||
with permissions.
|
||||
|
||||
## Ease of Use
|
||||
|
||||
You only need to be familiar with Structured Query Language (SQL) and Java Database Connectivity (JDBC) to handle massive data.
|
||||
It helps you focus on the design and implementation of your business system.
|
||||
|
||||
- SQL is the standard language for accessing relational databases, and very popular in big data eco too.
|
||||
It turns out that everybody knows SQL.
|
||||
- JDBC provides a standard API for tool/database developers and makes it possible to write database applications using a pure Java API.
|
||||
- There are plenty of free or commercial JDBC tools out there.
|
||||
|
||||
## Run Anywhere
|
||||
|
||||
Kyuubi can submit Spark applications to all supported cluster managers, including YARN, Mesos, Kubernetes, Standalone, and local.
|
||||
|
||||
The SPA policy also make it possible for you to launch different applications against different cluster managers.
|
||||
|
||||
## High Performance
|
||||
|
||||
Kyuubi is built on the Apache Spark, a lightning-fast unified analytics engine.
|
||||
|
||||
- **Concurrent execution**: multiple Spark applications work together
|
||||
- **Quick response**: long-running Spark applications without startup cost
|
||||
- **Optimal execution plan**: fully supports Spark SQL Catalyst Optimizer,
|
||||
|
||||
## Authentication & Authorization
|
||||
|
||||
## High Availability
|
||||
Loading…
Reference in New Issue
Block a user