Yin Huai
a50fedd5bc
Merge pull request #2 from jystephan/master
...
Allow saving benchmark queries results as parquet files
2015-07-22 13:40:39 -07:00
Jean-Yves Stephan
653d82134d
No collect before saveAsParquet
2015-07-22 13:30:40 -07:00
Yin Huai
b10fa582ea
Merge pull request #1 from Nosfe/patch-1
...
Reading hadoopConfiguration directly from SparkContext.
2015-07-22 10:27:24 -07:00
Jean-Yves Stephan
a4a53b8a73
Took Aaron's comments
2015-07-21 20:05:53 -07:00
Jean-Yves Stephan
d866cce1a1
Format
2015-07-21 13:27:50 -07:00
Jean-Yves Stephan
933f3f0bb5
Removed queryOutputLocation parameter
2015-07-21 13:26:50 -07:00
Jean-Yves Stephan
9640cd8c1e
The execution mode (collect results / foreach results / writeparquet) is now specified as an argument to Query.
2015-07-21 13:23:11 -07:00
Jean-Yves Stephan
8e62e4fdbd
Added optional parameters to runBenchmark to specify a location to save queries outputs as parquet files.
...
+ Removed the hardcoded baseDir/parquet/ structure
2015-07-20 17:09:20 -07:00
Pace Francesco
4f4b08a122
Reading hadoopConfiguration from Spark.
...
Read hadoopConfiguration from SparkContext instead of creating a new Configuration directly from Hadoop config files.
This allow us to use hadoop parameters inserted or modified in one of Spark config files. (e.g.: Swift credentials).
2015-06-19 15:01:57 +02:00
Yin Huai
3eca8d2947
Add a method to wait for the finish of the experiment (waitForFinish).
2015-05-22 12:41:55 -07:00
Yin Huai
70da4f490e
Move dataframe into benchmark.
2015-05-16 19:31:55 -07:00
Yin Huai
9156e14f4b
Provide userSpecifiedBaseDir to access a dataset that is not in the path with the default format.
2015-05-07 11:01:38 -07:00
Yin Huai
fb9939b136
includeBreakdown is a parameter of runExperiment.
2015-04-20 10:03:41 -07:00
Yin Huai
6c5657b609
Refactoring and doc.
2015-04-16 18:10:57 -07:00
Yin Huai
930751810e
Initial port.
2015-04-15 20:03:14 -07:00
Yin Huai
e81669ab3b
Initial commit.
2015-04-15 20:02:32 -07:00
