Commit Graph

15 Commits

Author SHA1 Message Date
huangxiaoping
083f3c1e29
[KYUUBI #6010] Improve Gluten usage documentation
# 🔍 Description
## Issue References 🔗

This pull request fixes #6010

## Describe Your Solution 🔧

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6011 from huangxiaopingRD/gluten-doc.

Closes #6010

88bce5865 [huangxiaoping] [KYUUBI #6010] Improve Gluten usage documentation

Authored-by: huangxiaoping <1754789345@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2024-01-24 11:47:32 +08:00
wangjunbo
5b6a729fa8
[KYUUBI #5800] [KYUUBI#5467] Integrate Intel Gluten with Spark engine
# 🔍 Description
## Issue References 🔗

This pull request fixes #5467

## Describe Your Solution 🔧

1. Add Gluten UTs.
2. Setup CI for Gluten testing
3. Write docs to guide users in setting up Kyuubi with Spark plus Gluten.

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

github action ci tests: [Gluten Test CI](https://github.com/Kwafoor/incubator-kyuubi/actions/runs/7111586978)

---

# Checklists
## 📝 Author Self Checklist

- [ ] My code follows the [style guidelines](https://kyuubi.readthedocs.io/en/master/contributing/code/style.html) of this project
- [x] I have performed a self-review
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

## 📝 Committer Pre-Merge Checklist

- [ ] Pull request title is okay.
- [ ] No license issues.
- [ ] Milestone correctly set?
- [ ] Test coverage is ok
- [ ] Assignees are selected.
- [ ] Minimum number of approvals
- [ ] No changes are requested

**Be nice. Be informative.**

Closes #5800 from Kwafoor/kyuubi_5467.

Closes #5800

c6dd26f93 [wangjunbo] fix
7818ae0c5 [wangjunbo] fix Scala Test
296f08c8c [wangjunbo] remove spark-3.2 gluten test
5a704675d [wangjunbo] [KYUUBI#5467] Integrate Intel Gluten with Spark engine

Authored-by: wangjunbo <wangjunbo@qiyi.com>
Signed-off-by: ulyssesyou <ulyssesyou@apache.org>
2023-12-07 10:47:00 +08:00
senmiaoliu
998b5d5349
[KYUUBI #5099] Document link failure
### _Why are the changes needed?_

close #5099

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request

Closes #5103 from lsm1/features/kyuubi_5099.

Closes #5099

84a1ecad0 [senmiaoliu] fix doc

Authored-by: senmiaoliu <senmiaoliu@trip.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2023-07-28 10:46:53 +08:00
liangbowen
22e9fd7d68 [KYUUBI #4226] Fix word spelling typos in docs
### _Why are the changes needed?_

- fix word spelling typos in docs

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4226 from bowenliang123/doc-word-typo.

Closes #4226

393de90d [liangbowen] update
365cdc4b [liangbowen] fix word typos in docs

Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
2023-02-02 11:43:03 +08:00
liangbowen
69d625a1be [KYUUBI #4200] [Improvement] [Docs] Introduce Markdown formatting with spotless-maven-plugin and flexmark for docs
### _Why are the changes needed?_

- to consolidate styles in markdown files from manual written or auto-generated
- apply markdown formatting rules with flexmark from [spotless-maven-plugin](https://github.com/diffplug/spotless/tree/main/plugin-maven#markdown) to *.md files in `/docs`
- use `flexmark` to format markdown generation in `TestUtils` of common module used by `AllKyuubiConfiguration` and `KyuubiDefinedFunctionSuite`, as the same way in `FlexmarkFormatterFunc ` of `spotless-maven-plugin` using with `COMMONMARK` as `FORMATTER_EMULATION_PROFILE` (https://github.com/diffplug/spotless/blob/maven/2.30.0/lib/src/flexmark/java/com/diffplug/spotless/glue/markdown/FlexmarkFormatterFunc.java)
- using `flexmark` of` 0.62.2`, as the last version requiring Java 8+ (checked from pom file and bytecode version)

```
<markdown>
    <includes>
        <include>docs/**/*.md</include>
    </includes>
    <flexmark></flexmark>
</markdown>
```

- Changes applied to markdown doc files,
  -  no style change or breakings in built docs by `make html`
  - removal all the first blank in licences and comments to conform markdown style rules
  - tables regenerated by flexmark following as in [GitHub Flavored Markdown](https://help.github.com/articles/organizing-information-with-tables/) (https://github.com/vsch/flexmark-java/wiki/Extensions#tables)

### _How was this patch tested?_
- [x] regenerate docs using `make html` successfully and check all the markdown pages available
- [x] regenerate `settings.md` and `functions.md` by `AllKyuubiConfiguration` and `KyuubiDefinedFunctionSuite`, and pass the checks by both themselves and spotless check via `dev/reformat`
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4200 from bowenliang123/markdown-formatting.

Closes #4200

1eeafce4 [liangbowen] revert minor changes in AllKyuubiConfiguration
4f892857 [liangbowen] use flexmark in markdown doc generation
8c978abd [liangbowen] changes on markdown files
a9190556 [liangbowen] apply markdown formatting rules with `spotless-maven-plugin` to markdown files with in `/docs`

Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
2023-01-30 11:14:41 +08:00
Kent Yao
e1921fc8c0
[KYUUBI #2751] [DOC] Replace sphinx_rtd_theme with sphinx_book_theme
### _Why are the changes needed?_

make documentation style to fit official web site

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [x] Add screenshots for manual tests if appropriate
![image](https://user-images.githubusercontent.com/8326978/170434595-d5ec5068-ab2d-484e-9371-9331311cc0f1.png)
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #2751 from yaooqinn/doc2.

Closes #2751

b8123525 [Kent Yao] Switch

Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-05-27 02:22:36 +08:00
KenjiFujima
f80d1a8be3
[KYUUBI #2241] Remove unused deployment documents of Spark
### _Why are the changes needed?_

Remove unused deployment documents of Spark.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #2242 from KenjiFujima/KYUUBI-2241.

Closes #2241

b9264590 [KenjiFujima] [KYUUBI #2241] Remove unused deployment documents of Spark

Authored-by: KenjiFujima <thanosxnicholas@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-03-29 12:30:43 +08:00
Cheng Pan
54dfb4bbc0
[KYUUBI #1215][DOC] Document incremental collection
### _Why are the changes needed?_

Document incremental collection, close #1215

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [x] Add screenshots for manual tests if appropriate

<img width="1914" alt="1" src="https://user-images.githubusercontent.com/26535726/157191008-451bee2a-eb6b-4bb6-869b-5b0f75a21448.png">
<img width="1919" alt="2" src="https://user-images.githubusercontent.com/26535726/157191016-e183bbf5-aa4a-496d-a250-5d14cf04101d.png">
<img width="1920" alt="3" src="https://user-images.githubusercontent.com/26535726/157191026-343a39d7-0ab8-4886-9a51-3670394ef6be.png">

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #2057 from pan3793/doc.

Closes #1215

82677a03 [Cheng Pan] grammar
3be0e3f1 [Cheng Pan] fix
b467e975 [Cheng Pan] Update
d256cbaf [Cheng Pan] compress picture
b3c5fb64 [Cheng Pan] Fix
4c06307c [Cheng Pan] [KYUUBI #1215][DOC] Document incremental collection

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
2022-03-08 20:49:51 +08:00
Kent Yao
d805f49fe7
[KYUUBI #1557] [DOC] The TTL Of Kyuubi Engines
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

![image](https://user-images.githubusercontent.com/8326978/146000343-44c7b30c-28ba-4434-ab6e-9fb4d0a97d75.png)

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [x] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1557 from yaooqinn/ttk.

Closes #1557

8446180a [Kent Yao] [DOC] The TTL Of Kyuubi Engines

Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
2021-12-15 10:02:02 +08:00
zhenjiaguo
0f1d4ec0e0
[KYUUBI #1347] [DOC] Fix miscellaneous doc typos
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

Some spelling & grammar format fix.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1347 from zhenjiaguo/typo-fix.

Closes #1347

5c20ae28 [zhenjiaguo] change serial to several
db3fa969 [zhenjiaguo] typo fix

Authored-by: zhenjiaguo <zhenjia_guo@163.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
2021-11-08 09:27:37 +08:00
Cheng Pan
a76c344042
[KYUUBI #951] [LICENSE] Add license header on all docs
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #951 from pan3793/license.

Closes #951

4629eecd [Cheng Pan] Fix
c45a0784 [Cheng Pan] nit
b9a46b42 [Cheng Pan] pin license header at first line
80d1a71b [Cheng Pan] nit
b2a46e4c [Cheng Pan] Update
f6acaaf8 [Cheng Pan] minor
ef99183f [Cheng Pan] Add license header on all docs

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2021-08-19 09:53:52 +08:00
watermen
4fe409562a
[KYUUBI #454] Fix metrics Versions
![watermen](https://badgen.net/badge/Hello/watermen/green) [![Closes #454](https://badgen.net/badge/Preview/Closes%20%23454/blue)](https://github.com/yaooqinn/kyuubi/pull/454) ![27](https://badgen.net/badge/%2B/27/red) ![27](https://badgen.net/badge/-/27/green) ![1](https://badgen.net/badge/commits/1/yellow) ![Target Issue](https://badgen.net/badge/Missing/Target%20Issue/ff0000) ![Test Plan](https://badgen.net/badge/Missing/Test%20Plan/ff0000) [<img width="16" alt="Powered by Pull Request Badge" src="https://user-images.githubusercontent.com/1393946/111216524-d2bb8e00-85d4-11eb-821b-ed4c00989c02.png">](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info)<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->

Closes #454 from watermen/version.

c141344 [watermen] Fix metrics Versions

Authored-by: watermen <qiyadong2010@gmail.com>
Signed-off-by: Kent Yao <yao@apache.org>
2021-03-26 11:56:20 +08:00
Kent Yao
3ccbc505b1
logo 2021-03-19 21:08:52 +08:00
Kent Yao
183e18f1bc
[KYUUBI #404] How To Use Spark Dynamic Resource Allocation (DRA) in Kyuubi
![yaooqinn](https://badgen.net/badge/Hello/yaooqinn/green) [![Closes #404](https://badgen.net/badge/Preview/Closes%20%23404/blue)](https://github.com/yaooqinn/kyuubi/pull/404) ![518](https://badgen.net/badge/%2B/518/red) ![443](https://badgen.net/badge/-/443/green) ![3](https://badgen.net/badge/commits/3/yellow) ![Target Issue](https://badgen.net/badge/Missing/Target%20Issue/ff0000) [&#10088;?&#10089;](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info)<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->

<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request

Closes #404 from yaooqinn/dra.

b8b755c [Kent Yao] How To Use Spark Dynamic Resource Allocation (DRA) in Kyuubi
9bb47b1 [Kent Yao] How To Use Spark Dynamic Resource Allocation (DRA) in Kyuubi
e786787 [Kent Yao] How To Use Spark Dynamic Resource Allocation (DRA) in Kyuubi

Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
2021-03-08 18:27:49 +08:00
Kent Yao
3d0f050034
[KYUUBI #388] How To Use Spark Adaptive Query Execution (AQE) in Kyuubi
![yaooqinn](https://badgen.net/badge/Hello/yaooqinn/green) [![Closes #388](https://badgen.net/badge/Preview/Closes%20%23388/blue)](https://github.com/yaooqinn/kyuubi/pull/388) ![298](https://badgen.net/badge/%2B/298/red) ![28](https://badgen.net/badge/-/28/green) ![1](https://badgen.net/badge/commits/1/yellow) ![Target Issue](https://badgen.net/badge/Missing/Target%20Issue/ff0000) [&#10088;?&#10089;](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info)<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->

<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

User guide

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request

Closes #388 from yaooqinn/aqe.

dc2833f [Kent Yao] How To Use Spark Adaptive Query Execution (AQE) in Kyuubi

Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
2021-03-03 13:55:52 +08:00