[KYUUBI #388 ] How To Use Spark Adaptive Query Execution (AQE) in Kyuubi

![yaooqinn](https://badgen.net/badge/Hello/yaooqinn/green) [![Closes #388](https://badgen.net/badge/Preview/Closes%20%23388/blue)](https://github.com/yaooqinn/kyuubi/pull/388) ![298](https://badgen.net/badge/%2B/298/red) ![28](https://badgen.net/badge/-/28/green) ![1](https://badgen.net/badge/commits/1/yellow) ![Target Issue](https://badgen.net/badge/Missing/Target%20Issue/ff0000) [&#10088;?&#10089;](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info)<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->

<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

User guide

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request

Closes #388 from yaooqinn/aqe.

dc2833f [Kent Yao] How To Use Spark Adaptive Query Execution (AQE) in Kyuubi

Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>

2021-03-03 13:55:52 +08:00

868 B

Raw Blame History

How To Use Spark Dynamic Allocation in Kyuubi

TO BE DONE


## Dynamic Allocation
# spark.dynamicAllocation.enabled                           true
# spark.dynamicAllocation.initialExecutors                  10
# spark.dynamicAllocation.minExecutors                      10
# spark.dynamicAllocation.maxExecutors                      500
# spark.dynamicAllocation.executorAllocationRatio           0.8
# spark.dynamicAllocation.executorIdleTimeout               60s
# spark.dynamicAllocation.cachedExecutorIdleTimeout         1h
# spark.dynamicAllocation.shuffleTracking.enabled           false
# spark.dynamicAllocation.shuffleTracking.timeout           30min
# spark.dynamicAllocation.schedulerBacklogTimeout           1s
# spark.dynamicAllocation.sustainedSchedulerBacklogTimeout  1s

868 B Raw Blame History

How To Use Spark Dynamic Allocation in Kyuubi

868 B

Raw Blame History