Commit Graph

12 Commits

Author SHA1 Message Date
hongdongdong
3fa3004aef
[KYUUBI #1397] Move bind host and port to configmap
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
Setting `frontend.bind.host` and `frontend.bind.port` in `kyuubi-configmap.yaml`, instead of put them in ENV.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1398 from hddong/move-bind.

Closes #1397

0741f08a [hongdongdong] [KYUUBI#1397] Move bind host and port to configmap

Authored-by: hongdongdong <hongdongdong@cmss.chinamobile.com>
Signed-off-by: Kent Yao <yao@apache.org>
2021-11-16 18:05:12 +08:00
sunfangbin
f5e8152a9e
[KYUUBI #1389] Refine helm service template to support LoadBalancer type
### _Why are the changes needed?_
The helm service template is currently hard-coded with `NodePort`, we should make it configurable to support other service type such as `LoadBalancer`.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1389 from murong00/branch-1389.

Closes #1389

5855c546 [sunfangbin] Refine toYaml usage
d659ad4c [sunfangbin] Refine helm service template to support LoadBalancer type

Authored-by: sunfangbin <sunfangbin@cmss.chinamobile.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
2021-11-16 12:04:55 +08:00
hongdongdong
1f87aec04e
[KYUUBI #1325] [KYUUBI#1324] Add resource limit for kyuubi server on kubernetes
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
Add resource limit config for kyuubi server on kubernetes.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1325 from hddong/add-resource-limit.

Closes #1325

8ec3d987 [hongdongdong] fix
51f5fbeb [hongdongdong] [KYUUBI#1324] Add resource limit for kyuubi server on kubernetes

Authored-by: hongdongdong <hongdongdong@cmss.chinamobile.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
2021-11-03 15:01:23 +08:00
hongdongdong
23d821950e
[KYUUBI #1255] [KYUUBI#1253] Support use helm to deploy kyuubi server on k8s
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
We usally use helm to manager application yaml on k8s, we can use helm to delopy kyuubi server too.
Usages:
```
helm install kyuubi ${kyuubi_helm_url} -n kyuubi
```
```
helm uninstall kyuubi -n kyuubi
```

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1255 from hddong/add-helm.

Closes #1255

e197de34 [hongdongdong] Add doc
d0f74bbb [hongdongdong] rm hel tgz
e9a0a2e4 [hongdongdong] update and fix
f666d331 [hongdongdong] [KYUUBI#1253] Support use helm to deploy kyuubi server on k8s

Authored-by: hongdongdong <hongdongdong@cmss.chinamobile.com>
Signed-off-by: Kent Yao <yao@apache.org>
2021-10-20 18:16:28 +08:00
ulysses-you
95b884e9b7
[KYUUBI #1199] Refine docker image file
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
* Use apache/kyuubi docker hub repo instead of personal repo
* change workflow env to ubuntu-20.04

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1199 from ulysses-you/refine-docker.

Closes #1199

2ec73286 [ulysses-you] always
d6ed71b5 [ulysses-you] refine

Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
2021-10-09 14:22:35 +08:00
Cheng Pan
c8d9d8be21
[KYUUBI #1155] [BUILD] Support passthrough env var CI into build container
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

Continuous integration(aka. CI) services like GitHub Actions, Travis always provide an environment variable `CI` in runners, and we detect this variable to run some specific actions.

In [KYUUBI #742], we add a `CI` env var detection in `build/mvn` to suppress noisy maven logs in CI scenes.

This PR introduces a new build arg `CI` in `docker/Dockerfile`, which is intent to passthrough env var CI into the build container to suppress noisy maven logs in GitHub Action and Travis.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1155 from pan3793/ci.

Closes #1155

0f66b37e [Cheng Pan] typo
e1b9d35e [Cheng Pan] nit
d78bb563 [Cheng Pan] update workflow
4209ae67 [Cheng Pan] Update comments
ffc3a1e1 [Cheng Pan] nit
761c9e6c [Cheng Pan] Update usage
2f74b977 [Cheng Pan] Add build arg CI
509da2aa [Cheng Pan] [BUILD] Suppress maven log in Minikube Integration Test

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
2021-09-26 18:04:34 +08:00
hongdongdong
6509a6b925
[KYUUBI #1047] Use deployment to run kyuubi server HA on k8s
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
kyuubi-pod.yaml start a single kyuubi server, use deployment to start multiple servers.

### _How was this patch tested?_
- [X] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1048 from hddong/add-k8s-deployment.

Closes #1047

661463b9 [hongdongdong] [KYUUBI#1047] Use deployment to run kyuubi server HA on k8s

Authored-by: hongdongdong <hongdongdong@cmss.chinamobile.com>
Signed-off-by: Kent Yao <yao@apache.org>
2021-09-08 11:27:36 +08:00
Fu Chen
c83c38b350 [KYUUBI #954] Refactor Dockerfile
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

1. As the binary distributions contain all things that we want, we don't need a pre-build spark as a docker base image, use java as the base image instead.

2. Build kyuubi from source code.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #955 from cfmcgrady/kyuubi-954.

Closes #954

c1c4ed31 [Fu Chen] only compile module kyuubi-compile/kyuubi-integration-tests
af96c7df [Fu Chen] update master.yaml
4c395c22 [Fu Chen] sleep for debug GA
770eef1f [Fu Chen] add local docker registry
749db9c4 [Fu Chen] build docker image
feb86b01 [Fu Chen] fix Dockerfile arg bug
0a599749 [Fu Chen] add .dockerignore
49dc6e31 [Fu Chen] review
dd1d73ce [Fu Chen] refactor Dockerfile

Authored-by: Fu Chen <cfmcgrady@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou18@gmail.com>
2021-08-20 09:24:38 +08:00
timothy65535
eb96c6ea64
[KYUUBI #877] Add kubernetes configmap yaml file
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

Add kubernetes configmap yaml file
```
kubectl apply -f docker/kyuubi-configmap.yaml
```

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request

Closes #878 from timothy65535/ky-877.

Closes #877

dff774de [timothy65535] improve patch
79e79d03 [timothy65535] keep origin configmap usage
ed28c78a [timothy65535] [KYUUBI #877] Add kubernetes configmap yaml file

Authored-by: timothy65535 <timothy65535@163.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2021-08-02 14:55:10 +08:00
hongdongdong
d90c21e0b0
[KYUUBI #859] Add kubernetes configmap
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
Now, we need to rebuild docker image when config changed. Use configmap to save kyuubi config file, then we can change config by edit configmap and restart pod.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [X] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request

Closes #866 from hddong/kyuubi-859.

Closes #859

caae67d1 [hongdongdong] fix comments
13fc08aa [hongdongdong] [KYUUBI#859]Add kubernetes configmap

Authored-by: hongdongdong <hongdongdong@cmss.chinamobile.com>
Signed-off-by: Kent Yao <yao@apache.org>
2021-07-27 15:00:54 +08:00
hongdongdong
ab18262d19
[KYUUBI #844] Exposing Kyuubi as a NodePort service
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
`kubectl port-forward` usually use to test, use `NodePort` to visit kyuubi service in pod.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [X] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request

Closes #858 from hddong/kyuubi-844.

Closes #844

d1492262 [hongdongdong] modify comments
72d51d61 [hongdongdong] fix comments
91727986 [hongdongdong] fix
a169332e [hongdongdong] fix comments
74cec930 [hongdongdong] [KYUUBI#844]Use NodePort to expose kyuubi port on k8s

Authored-by: hongdongdong <hongdongdong@cmss.chinamobile.com>
Signed-off-by: Kent Yao <yao@apache.org>
2021-07-22 18:17:43 +08:00
Kent Yao
5e16ddb1b5
Run kyuubi on kubernetes - make basic dockerfile and example (#795)
* Run kyuubi on kubernetes - make basic dockerfile and example

* Run kyuubi on kubernetes - make basic dockerfile and example

* Run kyuubi on kubernetes - make basic dockerfile and example

* Run kyuubi on kubernetes - make basic dockerfile and example

* Run kyuubi on kubernetes - make basic dockerfile and example

* Revert "Set protected_branches explicitly in a nonstrict way (#793)"

This reverts commit b16533a972.

* Revert "Revert "Set protected_branches explicitly in a nonstrict way (#793)""

This reverts commit 57f5f6cf9c010ed177bdf0cd446902feb68c14dd.

* adress comments
2021-07-14 20:00:37 +08:00