github/spark-sql-perf
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

put additionalTests to mlmetrics (#153)

Browse Source 
The time for additionalTests is missing in MLMetrics. Now add it back to MLMetrics so that we can test the time for other methods.
This commit is contained in:
ludatabricks 2018-06-13 15:21:50 -07:00 committed by Joseph Bradley
parent a4e1c790ba
commit 49717a72dd
1 changed files with 4 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
src/main/scala/com/databricks/spark/sql/perf/mllib/MLPipelineStageBenchmarkable.scala
View File

@ -91,10 +91,11 @@ class MLPipelineStageBenchmarkable(
val additionalTests = test.testAdditionalMethods(param, model).map {
tuple =>
val (additionalMethodTime, _) = measureTime { tuple._2() }
tuple._1 -> additionalMethodTime.toMillis.toDouble
}
MLMetric(tuple._1, additionalMethodTime.toMillis, false)
}.toArray
val mlMetrics = Array(metricTrainingTime, metricTraining, metricTestTime, metricTest)
val mlMetrics = Array(metricTrainingTime, metricTraining, metricTestTime, metricTest) ++
additionalTests
val paramsMap = params.toMap
val benchmarkId = name.split('.').last + "_" + paramsMap.hashCode.abs