apache/kyuubi
1
0
Fork 0
Code Issues Pull Requests Actions 5 Packages Projects Releases Wiki Activity
768b8ff7ca
kyuubi/docs/deployment/migration-guide.md
Cheng Pan 06a915aff4 [KYUUBI #4950] Migrate Kyuubi embedded database from Derby to SQLite 
### _Why are the changes needed?_

Apache Derby is no longer active. One major drawback of Derby is that it does not support multiple connections to single db files, making it hard to analyze the data on local development.

SQLite may be the most popular embedded DBMS in the world. It lives almost in every smartphone (at least Android and iOS integrate SQLite), which means SQLite is quite stable and may be a good choice for standalone production deployment.

SQLite provides a CLI command `sqlite3` which is easy to use to connect a data file and run queries. Multi connections to a single db file is allowed, which helps a lot to analyze the data when the Kyuubi server is running.

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4950 from pan3793/sqlite.

Closes #4950

738c39e6a [Cheng Pan] Update docs/deployment/migration-guide.md
5facdad9c [Cheng Pan] Update docs/deployment/migration-guide.md
b9883489c [Cheng Pan] migrate default metadata store to sqlite
c785e1a77 [Cheng Pan] migrate jdbc auth test to sqlite
23f63b932 [Cheng Pan] introduce sqlite deps

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2023-06-12 21:01:51 +08:00

3.4 KiB
Raw Blame History

Kyuubi Migration Guide

Upgrading from Kyuubi 1.7 to 1.8

  • Since Kyuubi 1.8, SQLite is added and becomes the default database type of Kyuubi metastore, as Derby has been deprecated. Both Derby and SQLite are mainly for testing purposes, and they're not supposed to be used in production. To restore previous behavior, set kyuubi.metadata.store.jdbc.database.type=DERBY and kyuubi.metadata.store.jdbc.url=jdbc:derby:memory:kyuubi_state_store_db;create=true.

Upgrading from Kyuubi 1.7.0 to 1.7.1

  • Since Kyuubi 1.7.1, protocolVersion is removed from the request parameters of the REST API Open(create) a session. All removed or unknown parameters will be silently ignored and affects nothing.
  • Since Kyuubi 1.7.1, confOverlay is supported in the request parameters of the REST API Create an operation with EXECUTE_STATEMENT type.

Upgrading from Kyuubi 1.6 to 1.7

  • In Kyuubi 1.7, kyuubi.ha.zookeeper.engine.auth.type does not fallback to kyuubi.ha.zookeeper.auth.type.
    When Kyuubi engine does Kerberos authentication with Zookeeper, user needs to explicitly set kyuubi.ha.zookeeper.engine.auth.type to KERBEROS.
  • Since Kyuubi 1.7, Kyuubi returns engine's information for GetInfo request instead of server. To restore the previous behavior, set kyuubi.server.info.provider to SERVER.
  • Since Kyuubi 1.7, Kyuubi session type SQL is refactored to INTERACTIVE, because Kyuubi supports not only SQL session, but also SCALA and PYTHON sessions. User need to use INTERACTIVE sessionType to look up the session event.
  • Since Kyuubi 1.7, the REST API of Open(create) a session will not contain parameters user password and IpAddr. User and password should be set in Authorization of http request if needed.

Upgrading from Kyuubi 1.6.0 to 1.6.1

  • Since Kyuubi 1.6.1, kyuubi.ha.zookeeper.engine.auth.type does not fallback to kyuubi.ha.zookeeper.auth.type.
    When Kyuubi engine does Kerberos authentication with Zookeeper, user needs to explicitly set kyuubi.ha.zookeeper.engine.auth.type to KERBEROS.

Upgrading from Kyuubi 1.5 to 1.6

  • Kyuubi engine gets Zookeeper principal & keytab from kyuubi.ha.zookeeper.auth.principal & kyuubi.ha.zookeeper.auth.keytab.
    kyuubi.ha.zookeeper.auth.principal & kyuubi.ha.zookeeper.auth.keytab fallback to kyuubi.kinit.principal & kyuubi.kinit.keytab when not set.
    Since Kyuubi 1.6, kyuubi.kinit.principal & kyuubi.kinit.keytab are filtered out from Kyuubi engine's conf for better security.
    When Kyuubi engine does Kerberos authentication with Zookeeper, user needs to explicitly set kyuubi.ha.zookeeper.auth.principal & kyuubi.ha.zookeeper.auth.keytab.