5b49061d86
### _Why are the changes needed?_ Add Docs for kyuubi tools spark-block-cleaner. * Explain the parameters * Introduction to basic startup * Give an example ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [X] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request Closes #815 from zwangsheng/doc/spark_block_cleaner. Closes #815 1ec6795f [Binjie Yang] delete todo bbf4d6e2 [Binjie Yang] make it common 9cf3e159 [Binjie Yang] format 0803995a [Binjie Yang] straighten out the article f834b382 [Binjie Yang] refactor 25be318f [Binjie Yang] fix 7304e595 [Binjie Yang] docs for spark-block-cleaner Authored-by: Binjie Yang <2213335496@qq.com> Signed-off-by: ulysses-you <ulyssesyou18@gmail.com>
77 lines
2.0 KiB
Markdown
77 lines
2.0 KiB
Markdown
<div align=center>
|
|
|
|
|
|
|
|
</div>
|
|
|
|
# Building Kyuubi
|
|
|
|
## Building Kyuubi with Apache Maven
|
|
|
|
**Kyuubi** is built based on [Apache Maven](http://maven.apache.org),
|
|
|
|
```bash
|
|
./build/mvn clean package -DskipTests
|
|
```
|
|
|
|
This results in the creation of all sub-modules of Kyuubi project without running any unit test.
|
|
|
|
If you want to test it manually, you can start Kyuubi directly from the Kyuubi project root by running
|
|
|
|
```bash
|
|
bin/kyuubi start
|
|
```
|
|
|
|
## Building a Submodule Individually
|
|
|
|
For instance, you can build the Kyuubi Common module using:
|
|
|
|
```bash
|
|
build/mvn clean package -pl :kyuubi-common -DskipTests
|
|
```
|
|
|
|
## Building Submodules Individually
|
|
|
|
For instance, you can build the Kyuubi Common module using:
|
|
|
|
```bash
|
|
build/mvn clean package -pl :kyuubi-common,:kyuubi-ha -DskipTests
|
|
```
|
|
|
|
## Skipping Some modules
|
|
|
|
For instance, you can build the Kyuubi modules without Kyuubi Codecov and Assembly modules using:
|
|
|
|
```bash
|
|
mvn clean install -pl '!:kyuubi-codecov,!:kyuubi-assembly' -DskipTests
|
|
```
|
|
|
|
## Building Kyuubi against Different Apache Spark versions
|
|
|
|
Since v1.1.0, Kyuubi support building with different Spark profiles,
|
|
|
|
Profile | Default | Since
|
|
--- | --- | ---
|
|
-Pspark-3.0 | Yes | 1.0.0
|
|
-Pspark-3.1 | No | 1.1.0
|
|
|
|
|
|
## Defining the Apache Mirror for Spark
|
|
|
|
By default, we use `https://archive.apache.org/dist/spark/` to download the built-in Spark release package,
|
|
but if you find it hard to reach, or the downloading speed is too slow, you can define the `spark.archive.mirror`
|
|
property to a suitable Apache mirror site. For instance,
|
|
|
|
```bash
|
|
build/mvn clean package -Dspark.archive.mirror=https://mirrors.bfsu.edu.cn/apache/spark/spark-3.0.1
|
|
```
|
|
|
|
Visit [Apache Mirrors](http://www.apache.org/mirrors/) and choose a mirror based on your region.
|
|
|
|
Specifically for developers in China mainland, you can use the pre-defined profile named `mirror-cn` which use
|
|
`mirrors.bfsu.edu.cn` to speed up Spark Binary downloading. For instance,
|
|
|
|
```bash
|
|
build/mvn clean package -Pmirror-cn
|
|
```
|