Yin Huai
8674c153b7
Merge pull request #10 from marmbrus/updateSBT
...
Update SBT
2015-08-20 16:52:29 -07:00
Michael Armbrust
3f33db31c0
Update SBT
2015-08-20 16:46:45 -07:00
Michael Armbrust
00aa49e8e4
Add support for CPU Profiling.
2015-08-20 16:46:12 -07:00
Yin Huai
249157f6a6
Fix typo.
2015-08-17 12:56:35 -07:00
Michael Armbrust
d6f89c862d
Merge pull request #8 from yhuai/hashSum
...
Add a new ExecutionMode to calculate the sum of hash values of result rows.
2015-08-14 12:39:51 -07:00
Yin Huai
d5c3104ec6
address comments.
2015-08-14 11:39:06 -07:00
Yin Huai
51546868f4
You can specific perf result location.
2015-08-13 18:43:50 -07:00
Yin Huai
11bfdc7c5a
Add an ExecutionMode to check query results.
2015-08-13 18:43:49 -07:00
Yin Huai
1fe1729331
Merge pull request #7 from marmbrus/fixBreakdown
...
Fixes to breakdown calculation and table creation.
2015-08-13 18:39:53 -07:00
Michael Armbrust
ed8ddfedcd
yins comments
2015-08-13 17:54:00 -07:00
Michael Armbrust
4101a1e968
Fixes to breakdown calculation and table creation.
2015-08-13 15:47:01 -07:00
Yin Huai
ff19051b0e
Merge pull request #6 from marmbrus/refactor
...
Remove deprecated parquet writing code and add some micro benchmarks
2015-08-11 16:10:11 -07:00
Michael Armbrust
a239da90a2
more cleanup, update readme
2015-08-11 15:51:34 -07:00
Michael Armbrust
51b9dcb5b5
Merge remote-tracking branch 'origin/master' into refactor
...
Conflicts:
src/main/scala/com/databricks/spark/sql/perf/bigdata/Queries.scala
src/main/scala/com/databricks/spark/sql/perf/query.scala
src/main/scala/com/databricks/spark/sql/perf/runBenchmarks.scala
src/main/scala/com/databricks/spark/sql/perf/table.scala
src/main/scala/com/databricks/spark/sql/perf/tpcds/queries/ImpalaKitQueries.scala
src/main/scala/com/databricks/spark/sql/perf/tpcds/queries/SimpleQueries.scala
2015-08-07 15:31:32 -07:00
Yin Huai
e650da3533
Merge pull request #3 from jystephan/master
...
Closing bracket typo
2015-07-22 15:06:15 -07:00
Jean-Yves Stephan
9421522820
Closing bracket
2015-07-22 15:03:43 -07:00
Yin Huai
a50fedd5bc
Merge pull request #2 from jystephan/master
...
Allow saving benchmark queries results as parquet files
2015-07-22 13:40:39 -07:00
Jean-Yves Stephan
653d82134d
No collect before saveAsParquet
2015-07-22 13:30:40 -07:00
Yin Huai
b10fa582ea
Merge pull request #1 from Nosfe/patch-1
...
Reading hadoopConfiguration directly from SparkContext.
2015-07-22 10:27:24 -07:00
Michael Armbrust
f00ad77985
with data generation
2015-07-22 00:29:58 -07:00
Jean-Yves Stephan
a4a53b8a73
Took Aaron's comments
2015-07-21 20:05:53 -07:00
Jean-Yves Stephan
d866cce1a1
Format
2015-07-21 13:27:50 -07:00
Jean-Yves Stephan
933f3f0bb5
Removed queryOutputLocation parameter
2015-07-21 13:26:50 -07:00
Jean-Yves Stephan
9640cd8c1e
The execution mode (collect results / foreach results / writeparquet) is now specified as an argument to Query.
2015-07-21 13:23:11 -07:00
Jean-Yves Stephan
8e62e4fdbd
Added optional parameters to runBenchmark to specify a location to save queries outputs as parquet files.
...
+ Removed the hardcoded baseDir/parquet/ structure
2015-07-20 17:09:20 -07:00
Michael Armbrust
eba8cea93c
Basic join performance tests
2015-07-13 16:20:36 -07:00
Michael Armbrust
eb3dd30c35
Refactor to work in notebooks
2015-07-03 11:26:06 -07:00
Pace Francesco
4f4b08a122
Reading hadoopConfiguration from Spark.
...
Read hadoopConfiguration from SparkContext instead of creating a new Configuration directly from Hadoop config files.
This allow us to use hadoop parameters inserted or modified in one of Spark config files. (e.g.: Swift credentials).
2015-06-19 15:01:57 +02:00
Yin Huai
3eca8d2947
Add a method to wait for the finish of the experiment (waitForFinish).
2015-05-22 12:41:55 -07:00
Yin Huai
70da4f490e
Move dataframe into benchmark.
2015-05-16 19:31:55 -07:00
Yin Huai
9156e14f4b
Provide userSpecifiedBaseDir to access a dataset that is not in the path with the default format.
2015-05-07 11:01:38 -07:00
Yin Huai
fb9939b136
includeBreakdown is a parameter of runExperiment.
2015-04-20 10:03:41 -07:00
Yin Huai
6c5657b609
Refactoring and doc.
2015-04-16 18:10:57 -07:00
Yin Huai
930751810e
Initial port.
2015-04-15 20:03:14 -07:00
Yin Huai
e81669ab3b
Initial commit.
2015-04-15 20:02:32 -07:00
