9db0088ae1
 [](https://github.com/yaooqinn/kyuubi/pull/333)     [❨?❩](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info)<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT --> <!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> the service discovery should stop before the embedded zk server. ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [x] Add screenshots for manual tests if appropriate ```before 2021-01-31 11:26:15.800 ERROR server.KyuubiServer: RECEIVED SIGNAL 15: TERM 2021-01-31 11:26:15.801 INFO server.NIOServerCnxn: Closed socket connection for client /127.0.0.1:58707 which had sessionid 0x1000fbbfad70002 2021-01-31 11:26:15.801 INFO server.NIOServerCnxn: Closed socket connection for client /127.0.0.1:58490 which had sessionid 0x1000fbbfad70000 2021-01-31 11:26:15.801 INFO server.NIOServerCnxnFactory: NIOServerCnxn factory exited run method 2021-01-31 11:26:15.801 INFO zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x1000fbbfad70000, likely server has closed socket, closing socket connection and attempting reconnect 2021-01-31 11:26:15.801 INFO server.ZooKeeperServer: shutting down 2021-01-31 11:26:15.801 INFO server.SessionTrackerImpl: Shutting down 2021-01-31 11:26:15.801 INFO server.PrepRequestProcessor: Shutting down 2021-01-31 11:26:15.802 INFO server.KyuubiServer: Service: [ServiceDiscovery] is stopping. 2021-01-31 11:26:15.802 INFO server.SyncRequestProcessor: Shutting down 2021-01-31 11:26:15.802 INFO server.PrepRequestProcessor: PrepRequestProcessor exited loop! 2021-01-31 11:26:15.802 INFO server.SyncRequestProcessor: SyncRequestProcessor exited! 2021-01-31 11:26:15.802 INFO server.FinalRequestProcessor: shutdown of request processor complete 2021-01-31 11:26:15.804 INFO server.EmbeddedZkServer: Service[EmbeddedZkServer] is stopped. 2021-01-31 11:26:15.906 INFO state.ConnectionStateManager: State change: SUSPENDED 2021-01-31 11:26:15.907 INFO client.ServiceDiscovery: Zookeeper client connection state changed to: SUSPENDED 2021-01-31 11:26:17.615 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) 2021-01-31 11:26:17.615 INFO zookeeper.ClientCnxn: Socket error occurred: localhost/127.0.0.1:2181: Connection refused 2021-01-31 11:26:18.793 INFO server.SessionTrackerImpl: SessionTrackerImpl exited loop! 2021-01-31 11:26:19.126 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) 2021-01-31 11:26:19.126 INFO zookeeper.ClientCnxn: Socket error occurred: localhost/127.0.0.1:2181: Connection refused 2021-01-31 11:26:20.815 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) 2021-01-31 11:26:20.815 INFO zookeeper.ClientCnxn: Socket error occurred: localhost/127.0.0.1:2181: Connection refused 2021-01-31 11:26:21.947 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) 2021-01-31 11:26:21.948 INFO zookeeper.ClientCnxn: Socket error occurred: localhost/127.0.0.1:2181: Connection refused 2021-01-31 11:26:23.288 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) 2021-01-31 11:26:23.289 INFO zookeeper.ClientCnxn: Socket error occurred: localhost/127.0.0.1:2181: Connection refused 2021-01-31 11:26:25.075 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) 2021-01-31 11:26:25.075 INFO zookeeper.ClientCnxn: Socket error occurred: localhost/127.0.0.1:2181: Connection refused 2021-01-31 11:26:26.965 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) 2021-01-31 11:26:26.965 INFO zookeeper.ClientCnxn: Socket error occurred: localhost/127.0.0.1:2181: Connection refused 2021-01-31 11:26:28.412 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) 2021-01-31 11:26:28.412 INFO zookeeper.ClientCnxn: Socket error occurred: localhost/127.0.0.1:2181: Connection refused 2021-01-31 11:26:30.356 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) 2021-01-31 11:26:30.356 INFO zookeeper.ClientCnxn: Socket error occurred: localhost/127.0.0.1:2181: Connection refused 2021-01-31 11:26:31.751 ERROR curator.ConnectionState: Connection timed out for connection string (127.0.0.1:2181) and timeout (15000) / elapsed (15841) org.apache.curator.CuratorConnectionLossException: KeeperErrorCode = ConnectionLoss ``` ```after 2021-01-31 11:49:15.070 ERROR server.KyuubiServer: RECEIVED SIGNAL 15: TERM 2021-01-31 11:49:15.071 INFO server.KyuubiServer: Service: [ServiceDiscovery] is stopping. 2021-01-31 11:49:15.074 INFO server.PrepRequestProcessor: Processed session termination for sessionid: 0x1000fd1db370000 2021-01-31 11:49:15.074 WARN client.ServiceDiscovery: This Kyuubi instance localhost:10009 is now de-registered from ZooKeeper. The server will be shut down after the last client session completes. 2021-01-31 11:49:15.074 INFO client.ServiceDiscovery: Service[ServiceDiscovery] is stopped. 2021-01-31 11:49:15.075 INFO server.NIOServerCnxn: Closed socket connection for client /127.0.0.1:63940 which had sessionid 0x1000fd1db370000 2021-01-31 11:49:15.075 INFO zookeeper.ZooKeeper: Session: 0x1000fd1db370000 closed 2021-01-31 11:49:15.076 WARN client.ServiceDiscovery: Service[ServiceDiscovery] is not started(STOPPED) yet. 2021-01-31 11:49:15.076 INFO server.KyuubiServer: Service: [FrontendService] is stopping. 2021-01-31 11:49:15.078 INFO service.FrontendService: FrontendService has stopped 2021-01-31 11:49:15.078 INFO service.FrontendService: Service[FrontendService] is stopped. 2021-01-31 11:49:15.078 INFO server.KyuubiServer: Service: [KyuubiBackendService] is stopping. 2021-01-31 11:49:15.078 INFO server.KyuubiBackendService: Service: [KyuubiSessionManager] is stopping. 2021-01-31 11:49:15.078 INFO session.KyuubiSessionManager: Service: [KyuubiOperationManager] is stopping. 2021-01-31 11:49:15.078 INFO operation.KyuubiOperationManager: Service[KyuubiOperationManager] is stopped. 2021-01-31 11:49:15.078 INFO session.KyuubiSessionManager: Service[KyuubiSessionManager] is stopped. 2021-01-31 11:49:15.079 INFO server.KyuubiBackendService: Service[KyuubiBackendService] is stopped. 2021-01-31 11:49:15.079 INFO server.KyuubiServer: Service: [KinitAuxiliaryService] is stopping. 2021-01-31 11:49:15.079 INFO service.KinitAuxiliaryService: Service[KinitAuxiliaryService] is stopped. 2021-01-31 11:49:15.079 INFO server.KyuubiServer: Service[KyuubiServer] is stopped. 2021-01-31 11:49:15.079 INFO server.NIOServerCnxn: Closed socket connection for client /127.0.0.1:64039 which had sessionid 0x1000fd1db370002 2021-01-31 11:49:15.079 INFO server.NIOServerCnxnFactory: NIOServerCnxn factory exited run method 2021-01-31 11:49:15.079 INFO server.ZooKeeperServer: shutting down 2021-01-31 11:49:15.079 INFO server.SessionTrackerImpl: Shutting down 2021-01-31 11:49:15.079 INFO server.PrepRequestProcessor: Shutting down 2021-01-31 11:49:15.080 INFO server.SyncRequestProcessor: Shutting down 2021-01-31 11:49:15.080 INFO server.PrepRequestProcessor: PrepRequestProcessor exited loop! 2021-01-31 11:49:15.080 INFO server.SyncRequestProcessor: SyncRequestProcessor exited! 2021-01-31 11:49:15.080 INFO server.FinalRequestProcessor: shutdown of request processor complete 2021-01-31 11:49:15.081 INFO server.EmbeddedZkServer: Service[EmbeddedZkServer] is stopped. ``` - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request Closes #333 from yaooqinn/hang. 945e1ba [jhx1008] Fix hanging due to disordered service stopping Authored-by: jhx1008 <jhx1008@gmail.com> Signed-off-by: Kent Yao <yao@apache.org> |
||
|---|---|---|
| .github | ||
| bin | ||
| build | ||
| conf | ||
| dev | ||
| docs | ||
| externals | ||
| kyuubi-assembly | ||
| kyuubi-common | ||
| kyuubi-ha | ||
| kyuubi-main | ||
| _config.yml | ||
| .gitignore | ||
| .readthedocs.yml | ||
| .travis.yml | ||
| CODE_OF_CONDUCT.md | ||
| codecov.yml | ||
| LICENSE | ||
| pom.xml | ||
| README.md | ||
| scalastyle-config.xml | ||
Kyuubi
Kyuubi is a high-performance universal JDBC and SQL execution engine, built on top of Apache Spark. The goal of Kyuubi is to facilitate users to handle big data like ordinary data.
It provides a standardized JDBC interface with easy-to-use data access in big data scenarios. End-users can focus on developing their own business systems and mining data value without having to be aware of the underlying big data platform (compute engines, storage services, metadata management, etc.).
Kyuubi relies on Apache Spark to provide high-performance data query capabilities,
and every improvement in the engine's capabilities can help Kyuubi's performance make a qualitative leap.
In addition, Kyuubi improves ad-hoc responsiveness through the engine caching,
and enhances concurrency through horizontal scaling and load balancing.
It provides complete authentication and authentication services to ensure data and metadata security.
It provides robust high availability and load balancing to help you guarantee the SLA commitments.
It provides a two-level elastic resource management architecture to effectively improve resource utilization while covering the performance and response requirements of all scenarios including interactive,
or batch processing and point queries, or full table scans.
It embraces Spark and builds an ecosystem on top of it,
which allows Kyuubi to quickly expand its existing ecosystem and introduce new features,
such as cloud-native support and Data Lake/Lake House support.
The vision of Kyuubi vision is to build on top of Apache Spark and Data Lake technologies to unify the portal and become an ideal data lake management platform. It can support data processing e.g. ETL, and analytics e.g. BI in a pure SQL way. All workloads can be done on one platform, using one copy of data, with one SQL interface.
Online Documentation
Since Kyuubi 1.0.0, the Kyuubi online documentation is hosted by https://readthedocs.org/. You can find the specific version of Kyuubi documentation as listed below.
For 0.8 and earlier versions, please check the Github Pages directly.
Quick Start
Ready? Getting Started with Kyuubi.
Contributing
All bits of help are welcome. You can make various types of contributions to Kyuubi, including the following but not limited to,
- Help new users in chat channel or share your success stories w/ us -
- Improve Documentation -
- Test releases -
- Improve test coverage -
- Report bugs and better help developers to reproduce
- Review changes
- Make a pull request
- Promote to others
- Click the star button if you like this project
Before you start, we recommend that you check the Contribution Guidelines first.
Aside
The project took its name from a character of a popular Japanese manga - Naruto.
The character is named Kyuubi Kitsune/Kurama, which is a nine-tailed fox in mythology.
Kyuubi spread the power and spirit of fire, which is used here to represent the powerful Apache Spark.
Its nine tails stands for end-to-end multi-tenancy support of this project.
License
This project is licensed under the Apache 2.0 License. See the LICENSE file for details.