[KYUUBI #991] [MINOR] improve the error message of SPARK package not found.
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ <!-- Please clarify why the changes are needed. For instance, 1. If you add a feature, you can talk about the use case of it. 2. If you fix a bug, you can clarify why it is a bug. --> As `File.listFiles` may return null when spark package not found in folder `${PROJECT_ROOT_DIR}/kyuubi-server/externals/kyuubi-download/target`, the NPE will throw. Improve the error message. Before thie pr: ``` Caused by: java.lang.RuntimeException: org.apache.kyuubi.KyuubiSQLException:Error opening session SessionHandle [9749221f-5b30-457b-b8e5-25affba25061] for fchen due to null at org.apache.kyuubi.KyuubiSQLException$.apply(KyuubiSQLException.scala:68) at org.apache.kyuubi.session.KyuubiSessionManager.openSession(KyuubiSessionManager.scala:74) at org.apache.kyuubi.service.AbstractBackendService.openSession(AbstractBackendService.scala:45) at org.apache.kyuubi.service.ThriftFrontendService.getSessionHandle(ThriftFrontendService.scala:190) at org.apache.kyuubi.service.ThriftFrontendService.OpenSession(ThriftFrontendService.scala:199) at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1377) at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1362) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.kyuubi.service.authentication.TSetIpAddressProcessor.process(TSetIpAddressProcessor.scala:36) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.NullPointerException: null at scala.collection.mutable.ArrayOps$ofRef$.length$extension(ArrayOps.scala:204) at scala.collection.mutable.ArrayOps$ofRef.length(ArrayOps.scala:204) at scala.collection.IndexedSeqOptimized.isEmpty(IndexedSeqOptimized.scala:30) at scala.collection.IndexedSeqOptimized.isEmpty$(IndexedSeqOptimized.scala:30) at scala.collection.mutable.ArrayOps$ofRef.isEmpty(ArrayOps.scala:198) at scala.collection.TraversableLike.headOption(TraversableLike.scala:608) at scala.collection.TraversableLike.headOption$(TraversableLike.scala:608) at scala.collection.mutable.ArrayOps$ofRef.headOption(ArrayOps.scala:198) at org.apache.kyuubi.engine.spark.SparkProcessBuilder.$anonfun$executable$1(SparkProcessBuilder.scala:50) at scala.Option.orElse(Option.scala:447) at org.apache.kyuubi.engine.spark.SparkProcessBuilder.<init>(SparkProcessBuilder.scala:41) at org.apache.kyuubi.engine.EngineRef.$anonfun$create$1(EngineRef.scala:140) at org.apache.kyuubi.engine.EngineRef.tryWithLock(EngineRef.scala:116) at org.apache.kyuubi.engine.EngineRef.create(EngineRef.scala:128) at org.apache.kyuubi.engine.EngineRef.$anonfun$getOrCreate$1(EngineRef.scala:182) at scala.Option.getOrElse(Option.scala:189) at org.apache.kyuubi.engine.EngineRef.getOrCreate(EngineRef.scala:182) at org.apache.kyuubi.session.KyuubiSessionImpl.$anonfun$open$2(KyuubiSessionImpl.scala:63) at org.apache.kyuubi.session.KyuubiSessionImpl.$anonfun$open$2$adapted(KyuubiSessionImpl.scala:62) at org.apache.kyuubi.ha.client.ZooKeeperClientProvider$.withZkClient(ZooKeeperClientProvider.scala:74) at org.apache.kyuubi.session.KyuubiSessionImpl.open(KyuubiSessionImpl.scala:62) at org.apache.kyuubi.session.KyuubiSessionManager.openSession(KyuubiSessionManager.scala:58) ... 12 more ``` After this pr: ``` Caused by: java.lang.RuntimeException: org.apache.kyuubi.KyuubiSQLException:Error opening session SessionHandle [5ea6a8b3-1727-4e0d-b542-992f84afcde8] for fchen due to SPARK_HOME is not set! at org.apache.kyuubi.KyuubiSQLException$.apply(KyuubiSQLException.scala:68) at org.apache.kyuubi.session.KyuubiSessionManager.openSession(KyuubiSessionManager.scala:74) at org.apache.kyuubi.service.AbstractBackendService.openSession(AbstractBackendService.scala:45) at org.apache.kyuubi.service.ThriftFrontendService.getSessionHandle(ThriftFrontendService.scala:190) at org.apache.kyuubi.service.ThriftFrontendService.OpenSession(ThriftFrontendService.scala:199) at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1377) at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1362) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.kyuubi.service.authentication.TSetIpAddressProcessor.process(TSetIpAddressProcessor.scala:36) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.RuntimeException: org.apache.kyuubi.KyuubiSQLException:SPARK_HOME is not set! at org.apache.kyuubi.KyuubiSQLException$.apply(KyuubiSQLException.scala:68) at org.apache.kyuubi.engine.spark.SparkProcessBuilder.$anonfun$executable$5(SparkProcessBuilder.scala:61) at scala.Option.getOrElse(Option.scala:189) at org.apache.kyuubi.engine.spark.SparkProcessBuilder.<init>(SparkProcessBuilder.scala:61) at org.apache.kyuubi.engine.EngineRef.$anonfun$create$1(EngineRef.scala:140) at org.apache.kyuubi.engine.EngineRef.tryWithLock(EngineRef.scala:116) at org.apache.kyuubi.engine.EngineRef.create(EngineRef.scala:128) at org.apache.kyuubi.engine.EngineRef.$anonfun$getOrCreate$1(EngineRef.scala:182) at scala.Option.getOrElse(Option.scala:189) at org.apache.kyuubi.engine.EngineRef.getOrCreate(EngineRef.scala:182) at org.apache.kyuubi.session.KyuubiSessionImpl.$anonfun$open$2(KyuubiSessionImpl.scala:63) at org.apache.kyuubi.session.KyuubiSessionImpl.$anonfun$open$2$adapted(KyuubiSessionImpl.scala:62) at org.apache.kyuubi.ha.client.ZooKeeperClientProvider$.withZkClient(ZooKeeperClientProvider.scala:74) at org.apache.kyuubi.session.KyuubiSessionImpl.open(KyuubiSessionImpl.scala:62) at org.apache.kyuubi.session.KyuubiSessionManager.openSession(KyuubiSessionManager.scala:58) ... 12 more ``` ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #991 from cfmcgrady/improve-error-msg. Closes #991 1e8da683 [Fu Chen] imporove the error message Authored-by: Fu Chen <cfmcgrady@gmail.com> Signed-off-by: ulysses-you <ulyssesyou18@gmail.com>
This commit is contained in:
Fu Chen
ulysses-you
parent
3afd20e03a
commit
9695f75889
@ -42,16 +42,17 @@ class SparkProcessBuilder(
|
||||
val cwd = getClass.getProtectionDomain.getCodeSource.getLocation.getPath
|
||||
.split("kyuubi-server")
|
||||
assert(cwd.length > 1)
|
||||
Paths.get(cwd.head)
|
||||
.resolve("externals")
|
||||
.resolve("kyuubi-download")
|
||||
.resolve("target")
|
||||
.toFile
|
||||
.listFiles(new FilenameFilter {
|
||||
override def accept(dir: File, name: String): Boolean = {
|
||||
dir.isDirectory && name.startsWith("spark-")
|
||||
}
|
||||
}).headOption.map(_.getAbsolutePath)
|
||||
Option(
|
||||
Paths.get(cwd.head)
|
||||
.resolve("externals")
|
||||
.resolve("kyuubi-download")
|
||||
.resolve("target")
|
||||
.toFile
|
||||
.listFiles(new FilenameFilter {
|
||||
override def accept(dir: File, name: String): Boolean = {
|
||||
dir.isDirectory && name.startsWith("spark-")}}))
|
||||
.flatMap(_.headOption)
|
||||
.map(_.getAbsolutePath)
|
||||
}
|
||||
|
||||
sparkHomeOpt.map{ dir =>
|
||||
|
||||
Loading…
Reference in New Issue
Block a user