apache/kyuubi
1
0
Fork 0
Code Issues Pull Requests Actions 5 Packages Projects Releases Wiki Activity

[KYUUBI #4519] Update metadata when batch application state changed

Browse Source 
### _Why are the changes needed?_

Kyuubi Server should update the metadata store when the batch application state changes, otherwise the peers see the outdated application state.

The following inconsistent happened on a dual Kyuubi Server deployment
```
23/03/15 11:15:36 batchId: e0b06788-46c0-484e-8648-deb08fc5dbd2, appId: null
23/03/15 11:15:41 batchId: e0b06788-46c0-484e-8648-deb08fc5dbd2, appId: null, appStatus: null
23/03/15 11:15:46 batchId: e0b06788-46c0-484e-8648-deb08fc5dbd2, appId: spark-934bd65c1de14fceb22411986dc5fe99, appStatus: PENDING
23/03/15 11:15:51 batchId: e0b06788-46c0-484e-8648-deb08fc5dbd2, appId: spark-934bd65c1de14fceb22411986dc5fe99, appStatus: PENDING
23/03/15 11:15:56 batchId: e0b06788-46c0-484e-8648-deb08fc5dbd2, appId: spark-934bd65c1de14fceb22411986dc5fe99, appStatus: PENDING
23/03/15 11:16:01 batchId: e0b06788-46c0-484e-8648-deb08fc5dbd2, appId: spark-934bd65c1de14fceb22411986dc5fe99, appStatus: PENDING
23/03/15 11:16:06 batchId: e0b06788-46c0-484e-8648-deb08fc5dbd2, appId: spark-934bd65c1de14fceb22411986dc5fe99, appStatus: PENDING
23/03/15 11:16:11 batchId: e0b06788-46c0-484e-8648-deb08fc5dbd2, appId: spark-934bd65c1de14fceb22411986dc5fe99, appStatus: PENDING
23/03/15 11:16:16 batchId: e0b06788-46c0-484e-8648-deb08fc5dbd2, appId: spark-934bd65c1de14fceb22411986dc5fe99, appStatus: PENDING
23/03/15 11:16:21 batchId: e0b06788-46c0-484e-8648-deb08fc5dbd2, appId: spark-934bd65c1de14fceb22411986dc5fe99, appStatus: RUNNING
23/03/15 11:16:26 batchId: e0b06788-46c0-484e-8648-deb08fc5dbd2, appId: spark-934bd65c1de14fceb22411986dc5fe99, appStatus: PENDING
23/03/15 11:16:31 batchId: e0b06788-46c0-484e-8648-deb08fc5dbd2, appId: spark-934bd65c1de14fceb22411986dc5fe99, appStatus: RUNNING
23/03/15 11:16:37 batchId: e0b06788-46c0-484e-8648-deb08fc5dbd2, appId: spark-934bd65c1de14fceb22411986dc5fe99, appStatus: PENDING
23/03/15 11:16:42 batchId: e0b06788-46c0-484e-8648-deb08fc5dbd2, appId: spark-934bd65c1de14fceb22411986dc5fe99, appStatus: RUNNING
23/03/15 11:16:47 batchId: e0b06788-46c0-484e-8648-deb08fc5dbd2, appId: spark-934bd65c1de14fceb22411986dc5fe99, appStatus: PENDING
23/03/15 11:16:52 batchId: e0b06788-46c0-484e-8648-deb08fc5dbd2, appId: spark-934bd65c1de14fceb22411986dc5fe99, appStatus: RUNNING
```

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4519 from pan3793/batch-metadata.

Closes #4519

6b9b3e706 [Cheng Pan] Update metadata when batch application state changed

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
This commit is contained in:
Cheng Pan 2023-03-15 15:08:11 +08:00
parent a027b6e0a1
commit 7b8ca1235c
No known key found for this signature in database
GPG Key ID: 8001952629BCC75D
1 changed files with 1 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
kyuubi-server/src/main/scala/org/apache/kyuubi/operation/BatchJobSubmission.scala
View File

@ -300,6 +300,7 @@ class BatchJobSubmission(
val newApplicationStatus = currentApplicationInfo
if (newApplicationStatus.map(_.state) != _applicationInfo.map(_.state)) {
_applicationInfo = newApplicationStatus
updateBatchMetadata()
info(s"Batch report for $batchId, ${_applicationInfo}")
}
}