ed0d9976ae
### _Why are the changes needed?_ current version, when set `kyuubi.session.engine.spark.showProgress=true`, it will show stage's progress info,but the info only show stage's detail, now we need to add job info in this, just like ``` [Stage 1:> (0 + 1) / 2] ``` to ``` [Job 1 (0 / 1) Stages] [Stage 1:> (0 + 1) / 2] ``` **this update is useful when user want know their sql execute detail** closes #4186 ### _How was this patch tested?_ - [x] Add screenshots for manual tests if appropriate **The photo show match log**  - [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request ### _Was this patch authored or co-authored using generative AI tooling?_ No Closes #5410 from davidyuan1223/improvement_add_job_log. Closes #4186 d8d03c4c0 [Cheng Pan] Update externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/spark/kyuubi/SQLOperationListener.scala a06e9a17c [david yuan] Update SparkConsoleProgressBar.scala 854408416 [david yuan] Merge branch 'apache:master' into improvement_add_job_log 963ff18b9 [david yuan] Update SparkConsoleProgressBar.scala 9e4635653 [david yuan] Update SparkConsoleProgressBar.scala 8c04dca7d [david yuan] Update SQLOperationListener.scala 39751bffa [davidyuan] fix 4f657e728 [davidyuan] fix deleted files 86756eba7 [david yuan] Merge branch 'apache:master' into improvement_add_job_log 0c9ac27b5 [davidyuan] add showProgress with jobInfo Unit Test d4434a0de [davidyuan] Revert "add showProgress with jobInfo Unit Test" 84b1aa005 [davidyuan] Revert "improvement_add_job_log fix" 66126f96e [davidyuan] Merge remote-tracking branch 'origin/improvement_add_job_log' into improvement_add_job_log 228fd9cf3 [davidyuan] add showProgress with jobInfo Unit Test 055e0ac96 [davidyuan] add showProgress with jobInfo Unit Test e4aac34bd [davidyuan] Merge remote-tracking branch 'origin/improvement_add_job_log' into improvement_add_job_log b226adad8 [davidyuan] Merge remote-tracking branch 'origin/improvement_add_job_log' into improvement_add_job_log a08799ca0 [david yuan] Update externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/spark/kyuubi/StageStatus.scala a991b68c4 [david yuan] Update externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/spark/kyuubi/StageStatus.scala d12046dac [davidyuan] add showProgress with jobInfo Unit Test 10a56b159 [davidyuan] add showProgress with jobInfo Unit Test a973cdde6 [davidyuan] improvement_add_job_log fix 1. provide Option[Int] with JobId e8a510891 [davidyuan] improvement_add_job_log fix 1. provide Option[Int] with JobId 7b9e874f2 [davidyuan] improvement_add_job_log fix 1. provide Option[Int] with JobId 5b4aaa8b5 [davidyuan] improvement_add_job_log fix 1. fix new end line 2. provide Option[Int] with JobId 780f9d15e [davidyuan] improvement_add_job_log fix 1. remove duplicate synchronized 2. because the activeJobs is ConcurrentHashMap, so reduce synchronized 3. fix scala code style 4. change forEach to asScala code style 5. change conf str to KyuubiConf.XXX.key 59340b713 [davidyuan] add showProgress with jobInfo Unit Test af05089d4 [davidyuan] add showProgress with jobInfo Unit Test c07535a01 [davidyuan] [Improvement] spark showProgress can briefly output info of the job #4186 d4bdec798 [yuanfuyuan] fix_4186 9fa8e73fc [davidyuan] add showProgress with jobInfo Unit Test 49debfbe3 [davidyuan] improvement_add_job_log fix 1. provide Option[Int] with JobId 5cf8714e0 [davidyuan] improvement_add_job_log fix 1. provide Option[Int] with JobId 249a422b6 [davidyuan] improvement_add_job_log fix 1. provide Option[Int] with JobId e15fc7195 [davidyuan] improvement_add_job_log fix 1. fix new end line 2. provide Option[Int] with JobId 4564ef98f [davidyuan] improvement_add_job_log fix 1. remove duplicate synchronized 2. because the activeJobs is ConcurrentHashMap, so reduce synchronized 3. fix scala code style 4. change forEach to asScala code style 5. change conf str to KyuubiConf.XXX.key 32ad0759b [davidyuan] add showProgress with jobInfo Unit Test d30492e46 [davidyuan] add showProgress with jobInfo Unit Test 6209c344e [davidyuan] [Improvement] spark showProgress can briefly output info of the job #4186 56b91a321 [yuanfuyuan] fix_4186 Lead-authored-by: davidyuan <yuanfuyuan@mafengwo.com> Co-authored-by: davidyuan <davidyuan1223@gmail.com> Co-authored-by: david yuan <51512358+davidyuan1223@users.noreply.github.com> Co-authored-by: yuanfuyuan <1406957364@qq.com> Co-authored-by: Cheng Pan <pan3793@gmail.com> Signed-off-by: Cheng Pan <chengpan@apache.org> |
||
|---|---|---|
| .. | ||
| kyuubi-chat-engine | ||
| kyuubi-download | ||
| kyuubi-flink-sql-engine | ||
| kyuubi-hive-sql-engine | ||
| kyuubi-jdbc-engine | ||
| kyuubi-spark-sql-engine | ||
| kyuubi-trino-engine | ||