[KYUUBI #632] solve the timeout problem when stop the kyuubiServer

### _Why are the changes needed?_
When you stop the KyuubiServiceDiscovery, you need to close node in zk and delete localNodePath at first. But when EmbeddedZookeeper is closed before KyuubiServiceDiscovery, the operation that stop kyuubiServer will timeout.
Now use 'org.apache.hadoop.util.ShutdownHookManager' to close each service by different priority.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request

Closes #632 from zhang1002/dev.

Closes #632

ee21e68 [张宇翔] Change the relevant format
f68fc7a [张宇翔] Merge branch 'master' into dev
128f244 [张宇翔] solve the timeout problem when stop the kyuubiServer
79b3d5a [张宇翔] ## solve the folder naming problem For starting kyuubiServer, the script need to get 'SPARK_HOME' from the file: RELEASE. The rule is: spark-${SPARK_VERSION}-bin-hadoop${HADOOP_VERSION}. But when we package project from source, the script named dist will create the directory about 'SPARK_HOME' by the rule: spark-${SPARK_VERSION}-bin-hadoop${SPARK_HADOOP_VERSION}. So it's the differences between two rules. This change let ${HADOOP_VERSION} as same as ${SPARK_HADOOP_VERSION}.

Lead-authored-by: 张宇翔 <zhang1002@126.com>
Co-authored-by: 张宇翔 <zhangyuxiang03@corp.netease.com>
Signed-off-by: Kent Yao <yao@apache.org>

kyuubi-common/src/main/scala/org/apache/kyuubi/Utils.scala

@@ -26,6 +26,7 @@ import scala.collection.JavaConverters._
 
 import org.apache.commons.lang3.SystemUtils
 import org.apache.hadoop.security.UserGroupInformation
+import org.apache.hadoop.util.ShutdownHookManager
 
 import org.apache.kyuubi.config.internal.Tests.IS_TESTING
 
@@ -165,4 +166,13 @@ private[kyuubi] object Utils extends Logging {
   def isTesting: Boolean = {
     System.getProperty(IS_TESTING.key) != null
   }
+
+  /**
+   * Add some operations that you want into ShutdownHook
+   * @param hook
+   * @param priority: 0~100
+   */
+  def addShutdownHook(hook: Runnable, priority: Int): Unit = {
+    ShutdownHookManager.get().addShutdownHook(hook, priority)
+  }
 }

kyuubi-main/src/main/scala/org/apache/kyuubi/server/KyuubiServer.scala

@@ -44,7 +44,9 @@ object KyuubiServer extends Logging {
     val server = new KyuubiServer()
     server.initialize(conf)
     server.start()
-    sys.addShutdownHook(server.stop())
+    Utils.addShutdownHook(new Runnable {
+      override def run(): Unit = server.stop()
+    }, 100)
     server
   }
 

kyuubi-zookeeper/src/main/scala/org/apache/kyuubi/zookeeper/EmbeddedZookeeper.scala

@@ -23,6 +23,7 @@ import scala.collection.JavaConverters._
 
 import org.apache.curator.test.{InstanceSpec, TestingServer}
 
+import org.apache.kyuubi.Utils
 import org.apache.kyuubi.config.KyuubiConf
 import org.apache.kyuubi.service.{AbstractService, ServiceState}
 import org.apache.kyuubi.zookeeper.ZookeeperConf._
@@ -71,7 +72,9 @@ class EmbeddedZookeeper extends AbstractService("EmbeddedZookeeper") {
   override def start(): Unit = synchronized {
     server.start()
     info(s"$getName is started at $getConnectString")
-    sys.addShutdownHook(server.close())
+    Utils.addShutdownHook(new Runnable {
+      override def run(): Unit = server.close()
+    }, 50)
     super.start()
   }