kyuubi/docs/deployment
Fei Wang 73c6b1b14a [KYUUBI #2717] [SUB-TASK][KPIP-4] Introduce jdbc session state store for batch session multiple HA
### _Why are the changes needed?_

Close #2715
Close #2374
Introduce jdbc based session state store to share the batch state information across kyuubi instances.

This pr is based on docs: https://docs.google.com/document/d/1_BMekMT88P2vCHCl4ZacWEeWslpIv4pxsJH32PAnWSI and  refactor it during implementation.

In this PR, I introduce `StateStore` interface and implement the default `JDBCStateStore`.

The data structures:
1. BatchState - the batch state info for multiple-HA, including batch operation state and application information
2. BatchMeta - the meta data used for batch recovery, including sessionConf, batchRequest information

For JDBCStateStore:
- support to specify data base type:
  - 1. DERBY - it will use derby jdbc driver by default
  - 2. MYSQL - it will use mysql jdbc by default
  - 3. CUSTOM - the user need specify the jdbc driver
- support to init schema

By default, the jdbc url is a derby in-memory database url for single kyuubi instance.

To support multiple HA in production env, we should config the production jdbc url to share batch info across kyuubi instances.

The datasource is managed by HiKariCP.

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #2717 from turboFei/kyuubi_2715_hikaricp_v2.

Closes #2717

367b944a3 [Fei Wang] comments
9a7963dbd [Fei Wang] post session type in event
b1c130853 [Fei Wang] move session type to session package
66def0cb6 [Fei Wang] add JDBCStateStoreConf
839b72bf6 [Fei Wang] refactor
644cbcd28 [Fei Wang] filter by session type
58f1c7854 [Fei Wang] refactor
1c0035208 [Fei Wang] add session type
34a087ee5 [Fei Wang] refactor
fba38e473 [Fei Wang] refactor
fcbde25a0 [Fei Wang] refactor
332a67537 [Fei Wang] comments
135ba9b6d [Fei Wang] Revert "revert class utils change"
0e045a483 [Fei Wang] Revert "Revert "move""
05048416e [Fei Wang] Revert "test again"
01a50c35b [Fei Wang] test again
b89159d13 [Fei Wang] Revert "move"
3764a9051 [Fei Wang] move
eef17822c [Fei Wang] revert
4166db261 [Fei Wang] adjust engine idle timeout
5fc95a824 [Fei Wang] revert class utils change
23f88d63d [Fei Wang] refactor
a66cb811d [Fei Wang] refactor
89fa0edb4 [Fei Wang] fix
e5228fefd [Fei Wang] refactor
581908d86 [Fei Wang] exclude sql
3446aaa06 [Fei Wang] UPPER
ab980c677 [Fei Wang] [KYUUBI #2715] Introduce session state store for batch session multiple HA

Authored-by: Fei Wang <fwang12@ebay.com>
Signed-off-by: Fei Wang <fwang12@ebay.com>
2022-05-31 15:06:39 +08:00
..
spark [KYUUBI #2751] [DOC] Replace sphinx_rtd_theme with sphinx_book_theme 2022-05-27 02:22:36 +08:00
engine_lifecycle.md [KYUUBI #2751] [DOC] Replace sphinx_rtd_theme with sphinx_book_theme 2022-05-27 02:22:36 +08:00
engine_on_kubernetes.md [KYUUBI #2751] [DOC] Replace sphinx_rtd_theme with sphinx_book_theme 2022-05-27 02:22:36 +08:00
engine_on_yarn.md [KYUUBI #2751] [DOC] Replace sphinx_rtd_theme with sphinx_book_theme 2022-05-27 02:22:36 +08:00
engine_share_level.md [KYUUBI #2751] [DOC] Replace sphinx_rtd_theme with sphinx_book_theme 2022-05-27 02:22:36 +08:00
high_availability_guide.md [KYUUBI #2751] [DOC] Replace sphinx_rtd_theme with sphinx_book_theme 2022-05-27 02:22:36 +08:00
hive_metastore.md [KYUUBI #2751] [DOC] Replace sphinx_rtd_theme with sphinx_book_theme 2022-05-27 02:22:36 +08:00
index.rst [KYUUBI #2751] [DOC] Replace sphinx_rtd_theme with sphinx_book_theme 2022-05-27 02:22:36 +08:00
kyuubi_on_kubernetes.md [KYUUBI #2751] [DOC] Replace sphinx_rtd_theme with sphinx_book_theme 2022-05-27 02:22:36 +08:00
settings.md [KYUUBI #2717] [SUB-TASK][KPIP-4] Introduce jdbc session state store for batch session multiple HA 2022-05-31 15:06:39 +08:00