[KYUUBI #3539] [FEATURE][TPCDS] Add white list help run the specified queries

### _Why are the changes needed?_

Add `white-list` help run the specified queries.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

  `--white-list q4,q15,q19,q25,q42,q49,q60,q66,q68,q72,q76,q78,q79,q89,q97`

![popo_2022-09-22  15-01-05](https://user-images.githubusercontent.com/52876270/191679906-76639468-a67d-408f-8b83-a18b34ef80e7.jpg)

Closes #3539 from zwangsheng/feature/tpcds_white_list.

Closes #3539

74657957 [zwangsheng] fix
26ade916 [Binjie Yang] Update dev/kyuubi-tpcds/src/main/scala/org/apache/kyuubi/tpcds/benchmark/RunBenchmark.scala
11a5b5c9 [Binjie Yang] Update dev/kyuubi-tpcds/src/main/scala/org/apache/kyuubi/tpcds/benchmark/RunBenchmark.scala
780410a3 [zwangsheng] fix
19018932 [zwangsheng] naming
1a4d1a25 [zwangsheng] naming
e1bb0069 [zwangsheng] fix
1be2bcb9 [zwangsheng] white

Lead-authored-by: zwangsheng <2213335496@qq.com>
Co-authored-by: Binjie Yang <52876270+zwangsheng@users.noreply.github.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
This commit is contained in:
zwangsheng 2022-09-27 20:08:31 +08:00 committed by Cheng Pan
parent 00aa49c5c0
commit 17289b13b5
No known key found for this signature in database
GPG Key ID: 8001952629BCC75D

View File

@ -29,7 +29,8 @@ case class RunConfig(
filter: Option[String] = None,
iterations: Int = 3,
breakdown: Boolean = false,
resultsDir: String = "/spark/sql/performance")
resultsDir: String = "/spark/sql/performance",
queries: Set[String] = Set.empty)
// scalastyle:off
/**
@ -65,6 +66,11 @@ object RunBenchmark {
opt[String]('r', "results-dir")
.action((x, c) => c.copy(resultsDir = x))
.text("dir to store benchmark results, e.g. hdfs://hdfs-nn:9870/pref")
opt[String]('q', "queries")
.action { case (x, c) =>
c.copy(queries = x.split(",").map(_.trim).filter(_.nonEmpty).toSet)
}
.text("name of the queries to run, use , split multiple name")
help("help")
.text("prints this usage text")
}
@ -96,11 +102,18 @@ object RunBenchmark {
benchmark.tpcds2_4Queries
}
val runQueries =
if (config.queries.nonEmpty) {
allQueries.filter(q => config.queries.contains(q.name.split('-')(0)))
} else {
allQueries
}
println("== QUERY LIST ==")
allQueries.foreach(q => println(q.name))
runQueries.foreach(q => println(q.name))
val experiment = benchmark.runExperiment(
executionsToRun = allQueries,
executionsToRun = runQueries,
includeBreakdown = config.breakdown,
iterations = config.iterations,
tags = Map("host" -> InetAddress.getLocalHost.getHostName))