kyuubi/dev
ulysses-you 43f40dcb8a
[KYUUBI #631] Add kyuubi-spark-extensions module
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/NetEase/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
The added sql module structure looks like:
```
kyuubi
  |
  - dev
      |
      - kyuubi-extension-spark_3.1
```

This PR mainly add 3 feature:
* merging small files automatically (include dynamic partition insertion case)
* insert shuffle node before Join to make AQE `OptimizeSkewedJoin` work
* stage level config isolation in AQE

Note that,  the sql rule  depend on the Apache Spark interface so we need make the sql module verion independence. Currently, this PR only supports the Spark 3.1.1.

Due to the version issue, we need to check and deploy this extension manually currently.

### _How was this patch tested?_
Add new test.

Closes #631 from ulysses-you/add-sql-module.

Closes #631

2cf12f1 [ulysses-you] version
cfbf72c [ulysses-you] address comment
7740ca6 [ulysses-you] module name
0f723eb [ulysses-you] workflow
45c23d8 [ulysses-you] line
80378f5 [ulysses-you] assembly
95528aa [ulysses-you] move module
5fe5d87 [ulysses-you] license
6578440 [ulysses-you] init work

Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: Kent Yao <yao@apache.org>
2021-05-13 16:03:32 +08:00
..
kyuubi-codecov Prepare Kyuubi v1.3.0 2021-05-13 13:58:20 +08:00
kyuubi-extension-spark_3.1 [KYUUBI #631] Add kyuubi-spark-extensions module 2021-05-13 16:03:32 +08:00
kyuubi-tpcds Prepare Kyuubi v1.3.0 2021-05-13 13:58:20 +08:00
dependencyList [KYUUBI #630] [METRICS] Support expose Prometheus metrics 2021-05-13 12:32:27 +08:00
merge_kyuubi_pr.py [KYUUBI #531] Refactor repo key words (#531) 2021-04-15 17:13:02 +08:00