2a39d697c0
# 🔍 Description ## Issue References 🔗 ## Describe Your Solution 🔧 Add docs for spark-3.4 and spark-3.5 as supported Spark profiles. ## Types of changes 🔖 - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) ## Test Plan 🧪 #### Behavior Without This Pull Request ⚰️ <img width="774" alt="image" src="https://github.com/apache/kyuubi/assets/1935105/29b0690d-574f-4b3c-ac70-50d5fb67a75e"> #### Behavior With This Pull Request 🎉 <img width="766" alt="image" src="https://github.com/apache/kyuubi/assets/1935105/58af356b-0f9c-4fe9-8f2b-8769ccebcc3a"> #### Related Unit Tests --- # Checklists ## 📝 Author Self Checklist - [x] My code follows the [style guidelines](https://kyuubi.readthedocs.io/en/master/contributing/code/style.html) of this project - [x] I have performed a self-review - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html) ## 📝 Committer Pre-Merge Checklist - [ ] Pull request title is okay. - [ ] No license issues. - [ ] Milestone correctly set? - [ ] Test coverage is ok - [ ] Assignees are selected. - [ ] Minimum number of approvals - [ ] No changes are requested **Be nice. Be informative.** Closes #5801 from bowenliang123/doc-spark-profiles. Closes #5801 35254df09 [liangbowen] style 2b55824fd [liangbowen] add spark-3.4 and spark-3.5 to the docs of building from code Authored-by: liangbowen <liangbowen@gf.com.cn> Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
3.1 KiB
Building From Source
Building With Maven
Kyuubi is built based on Apache Maven,
./build/mvn clean package -DskipTests
This results in the creation of all sub-modules of Kyuubi project without running any unit test.
If you want to test it manually, you can start Kyuubi directly from the Kyuubi project root by running
bin/kyuubi start
Building A Submodule Individually
For instance, you can build the Kyuubi Common module using:
build/mvn clean package -pl kyuubi-common -DskipTests
Building Submodules Individually
For instance, you can build the Kyuubi Common module using:
build/mvn clean package -pl kyuubi-common,kyuubi-ha -DskipTests
Skipping Some Modules
For instance, you can build the Kyuubi modules without Kyuubi Codecov and Assembly modules using:
mvn clean install -pl '!dev/kyuubi-codecov,!kyuubi-assembly' -DskipTests
Building Kyuubi Against Different Apache Spark Versions
Since v1.1.0, Kyuubi support building with different Spark profiles,
| Profile | Default | Since |
|---|---|---|
| -Pspark-3.1 | 1.1.0 | |
| -Pspark-3.2 | 1.4.0 | |
| -Pspark-3.3 | 1.6.0 | |
| -Pspark-3.4 | ✓ | 1.8.0 |
| -Pspark-3.5 | 1.8.0 |
Building With Apache dlcdn Site
By default, we use https://archive.apache.org/dist/ to download the built-in release packages of engines,
such as Spark or Flink.
But sometimes, you may find it hard to reach, or the download speed is too slow,
then you can define the apache.archive.dist by -Pmirror-cdn to accelerate to download speed.
For example,
build/mvn clean package -Pmirror-cdn
The profile migrates your download repo to the Apache officially suggested site - https://dlcdn.apache.org.
Note that, this site only holds the latest versions of Apache releases. You may fail if the specific version
defined by spark.version or flink.version is overdue.
Building with the fast profile
The fast profile helps to significantly reduce build time, which is useful for development or compilation validation, by skipping running the tests, code style checks, building scaladoc, enforcer rules and downloading engine archives used for tests.
build/mvn clean package -Pfast