b23c87c318
### _Why are the changes needed?_ The following discussion assumes using Spark cluster mode w/ `waitCompletion=false`. In Spark on Yarn, the application is visible immediately after `spark-submit` is returned, but things are different in Spark on K8s, Driver Pod is ephemerally invisible after submitting, so NOT_FOUND is returned instead of UNKNOWN or PENDING. To tolerate the above case, `kyuubi.engine.submit.timeout` is introduced, ApplicationManager will report UNKNOWN instead of NOT_FOUND during the Driver Pod scheduling period. More detail in #4467 1. Remove `KubernetesApplicationOperation`'s `JpsApplicationOperation` for handle Client Deploy Mode(`YarnApplicationOperation` doesn't handle this either) 2. Add engine submit timeout for `KubernetesApplicationOperation` to return Unknown status when not found driver pod in time range. 3. GetApplicationInfo with it's submit time ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request Closes #4469 from zwangsheng/4467. Closes #4467 562b67463 [zwangsheng] [KYUUBI #4467] Fix Setting.md 362c43d1b [zwangsheng] [KYUUBI #4467] Fix Setting.md ac69f4d81 [zwangsheng] [KYUUBI #4467] Add Config Desc d2b9fb660 [zwangsheng] [KYUUBI #4467] save tab eac880fcf [zwangsheng] [KYUUBI #4467] Ingnore Kubernetes Operation for client mode test 7a20b97a4 [Cheng Pan] Update kyuubi-server/src/main/scala/org/apache/kyuubi/engine/KubernetesApplicationOperation.scala aa4c7716a [zwangsheng] [KYUUBI #4467] Ingnore Kubernetes Operation for client mode test c5bd888ab [zwangsheng] [KYUUBI #4467] note it test a86dcefba [zwangsheng] [KYUUBI #4467] Using default none aed7f8794 [Cheng Pan] Update docs/deployment/settings.md 490df7dc0 [zwangsheng] [KYUUBI #4467] fix complie 33f3a5be8 [zwangsheng] [KYUUBI #4467] fix comments 4745790cf [zwangsheng] [KYUUBI #4467] Fix IT Test 924cfe38e [zwangsheng] [KYUUBI #4467] Fix Setting.md 5f8aeaacc [zwangsheng] [KYUUBI #4467] KubernetesApplicationOperation Wait if not fount driver pod in limit time range Lead-authored-by: zwangsheng <2213335496@qq.com> Co-authored-by: Cheng Pan <pan3793@gmail.com> Signed-off-by: Cheng Pan <chengpan@apache.org> |
||
|---|---|---|
| .. | ||
| spark | ||
| engine_lifecycle.md | ||
| engine_on_kubernetes.md | ||
| engine_on_yarn.md | ||
| engine_share_level.md | ||
| high_availability_guide.md | ||
| hive_metastore.md | ||
| index.rst | ||
| kyuubi_on_kubernetes.md | ||
| migration-guide.md | ||
| settings.md | ||