apache/kyuubi
1
0
Fork 0
Code Issues Pull Requests Actions 5 Packages Projects Releases Wiki Activity

[KYUUBI #3725] [BATCH] Improve submit batch job print engine builder command

Browse Source 
### _Why are the changes needed?_

old version
```log
2022-10-31 11:37:49 [INFO] [KyuubiSessionManager-exec-pool: Thread-174] org.apache.kyuubi.operation.BatchJobSubmission#59 - Submitting SPARK batch[5515091e-9eac-4466-a547-be4ba1352b6a] job: /opt/spark/bin/spark-submit \
	--class org.apache.spark.examples.SparkPi \
	--conf spark.jars=xxx/spark-examples_2.12-3.3.1.jar \
	--conf spark.kubernetes.driver.label.kyuubi-unique-tag=5515091e-9eac-4466-a547-be4ba1352b6a \
	--conf spark.app.name=xxx-k8s测试 \
	--conf spark.executorEnv.SPARK_USER_NAME=xxx \
	--conf spark.kubernetes.driverEnv.SPARK_USER_NAME=xxx \
	--conf spark.hive.server2.proxy.user=xxx \
	--conf spark.kyuubi.client.ipAddress=xxx \
	--conf spark.kubernetes.driverEnv.SPARK_USER_NAME=xxx \
	--conf spark.executorEnv.SPARK_USER_NAME=xxx \
	--proxy-user xxx xxx/spark-examples_2.12-3.3.1.jar 1000
```
new version
```log
2022-10-31 11:37:49 [INFO] [KyuubiSessionManager-exec-pool: Thread-174] org.apache.kyuubi.operation.BatchJobSubmission#59 - Submitting SPARK batch[5515091e-9eac-4466-a547-be4ba1352b6a] job:
/opt/spark/bin/spark-submit \
	--class org.apache.spark.examples.SparkPi \
	--conf spark.jars=xxx/spark-examples_2.12-3.3.1.jar \
	--conf spark.kubernetes.driver.label.kyuubi-unique-tag=5515091e-9eac-4466-a547-be4ba1352b6a \
	--conf spark.app.name=xxx-k8s测试 \
	--conf spark.executorEnv.SPARK_USER_NAME=xxx \
	--conf spark.kubernetes.driverEnv.SPARK_USER_NAME=xxx \
	--conf spark.hive.server2.proxy.user=xxx \
	--conf spark.kyuubi.client.ipAddress=xxx \
	--conf spark.kubernetes.driverEnv.SPARK_USER_NAME=xxx \
	--conf spark.executorEnv.SPARK_USER_NAME=xxx \
	--proxy-user xxx xxx/spark-examples_2.12-3.3.1.jar 1000
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3725 from zwangsheng/improve/batch_log_builder.

Closes #3725

87f8bad1 [zwangsheng] improve

Authored-by: zwangsheng <2213335496@qq.com>
Signed-off-by: hongdongdong <hongdd@apache.org>
This commit is contained in:
zwangsheng 2022-10-31 16:54:02 +08:00 committed by hongdongdong
parent 292457edf8
commit 81d9a8f1ca
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
kyuubi-server/src/main/scala/org/apache/kyuubi/operation/BatchJobSubmission.scala
View File

@ -196,7 +196,7 @@ class BatchJobSubmission(
private def submitAndMonitorBatchJob(): Unit = {
var appStatusFirstUpdated = false
try {
info(s"Submitting $batchType batch[$batchId] job: $builder")
info(s"Submitting $batchType batch[$batchId] job:\n$builder")
val process = builder.start
applicationInfo = currentApplicationInfo
while (!applicationFailed(applicationInfo) && process.isAlive) {