Commit Graph

1144 Commits

Author SHA1 Message Date
Cheng Pan
b31ce17750
Revert "[KYUUBI #1176] InvalidACL appears in the engine when zookeeper acl is turned on"
This reverts commit 9ba2aac9f5.
2021-10-12 21:17:50 +08:00
Cheng Pan
f2871e3143
Revert "[KYUUBI #1176][FOLLOWUP] change configuration(ha.zookeeper.acl.engine.enabled) version"
This reverts commit 547b270f0e.
2021-10-12 21:17:29 +08:00
simon
1a9bf3600d
[KYUUBI #1190] Enhance error log detection
### Why are the changes needed?
When line2 is read, it does not contain `at` or `cause by`, so all of the following messages are discarded.

#1190

```
2021-10-09 09:35:14.528 ERROR spark.SparkSQLEngine: Failed to instantiate SparkSession: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient;
org.apache.spark.sql.AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient;
	at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:113)
	at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:225)
	at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:137)
```

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1200 from simon824/fixut.

Closes #1190

d239df0e [simon] codestyle
83cb7d83 [simon] fixut
2d399883 [simon] Merge remote-tracking branch 'upstream/master'
3d9c12d3 [simon] Merge remote-tracking branch 'upstream/master'
5b905dbd [simon] Merge remote-tracking branch 'upstream/master'

Authored-by: simon <zhangshiming@cvte.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
(cherry picked from commit 4f8f4d9c6b)
Signed-off-by: Cheng Pan <chengpan@apache.org>
2021-10-09 14:39:18 +08:00
sychen
547b270f0e
[KYUUBI #1176][FOLLOWUP] change configuration(ha.zookeeper.acl.engine.enabled) version
https://github.com/apache/incubator-kyuubi/pull/1177
### _Why are the changes needed?_
change configuration(ha.zookeeper.acl.engine.enabled) version to `1.3.1`

### _How was this patch tested?_
- [] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [] Add screenshots for manual tests if appropriate

- [] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1193 from cxzl25/KYUUBI-1176-followup.

Closes #1176

72753ae5 [sychen] change config version

Authored-by: sychen <sychen@trip.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
(cherry picked from commit 464fdf456b)
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
2021-10-08 16:56:29 +08:00
sychen
9ba2aac9f5
[KYUUBI #1176] InvalidACL appears in the engine when zookeeper acl is turned on
### _Why are the changes needed?_
https://github.com/apache/incubator-kyuubi/issues/1176
When `kyuubi.ha.zookeeper.acl.enabled=true`, both service and engine will use zookeeper acl to create znode, but engine has no keytab information and cannot write information to zookeeper, throwing an exception.

```java
Caused by: org.apache.zookeeper.KeeperException$InvalidACLException: KeeperErrorCode = InvalidACL for /kyuubi_USER/XXXX
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:124)
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
	at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:792)
	at org.apache.kyuubi.shade.org.apache.curator.framework.imps.CreateBuilderImpl$11.call(CreateBuilderImpl.java:740)
```

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [x] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1177 from cxzl25/KYUUBI-1176.

Closes #1176

ecc08fa7 [sychen] fix engine acl
0b7cc2ec [sychen] fix InvalidACL

Authored-by: sychen <sychen@trip.com>
Signed-off-by: Kent Yao <yao@apache.org>
(cherry picked from commit d332534325)
Signed-off-by: Kent Yao <yao@apache.org>
2021-10-08 10:08:47 +08:00
张宇翔
1d2f172db4
[KYUUBI #1067] Catch exception if logRoot is not exist
BUGFIX:
#1067
This PR is just for version_1.3.0

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1154 from zhang1002/branh-1.3_catch-exception.

Closes #1067

2d9aaa5c [张宇翔] catch exception if logRoot is not exist

Authored-by: 张宇翔 <zhang1002@126.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
2021-09-28 13:29:58 +08:00
Cheng Pan
d621e50e58
[KYUUBI #1158] [INFRA] Update ANNOUNCE mail template
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
Update ANNOUNCE mail template to add more detailed descriptions of Kyuubi.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1158 from pan3793/ann.

Closes #1158

46c19975 [Cheng Pan] typo
a87321f5 [Cheng Pan] [INFRA] Update ANNOUNCE mail template

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
2021-09-26 17:57:33 +08:00
Cheng Pan
a03014d5b9
[KYUUBI #1107] [INFRA] Add release mail templates
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1107 from pan3793/template.

Closes #1107

659a2120 [Cheng Pan] Update ANNOUNCE template
971ce74a [Cheng Pan] Update ANNOUNCE template
13465519 [Cheng Pan] Update Links
6a5faf5c [Cheng Pan] typo
0f037c3f [Cheng Pan] Update rat excludes
d8e4cbb9 [Cheng Pan] Add license headers and remove incorrect words
8872a34a [Cheng Pan] Update template
d7f01643 [Cheng Pan] Add mail template
2a170071 [Cheng Pan] Add mail template

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
(cherry picked from commit 1489d1dbf5)
Signed-off-by: Cheng Pan <chengpan@apache.org>
2021-09-24 12:33:00 +08:00
timothy65535
f3b6609e5d
[KYUUBI #1141] Remove redundant unit
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

Remove redundant unit. `MB`

```
2021-09-23 08:41:08.889 INFO spark.SparkSQLEngine:
    Spark application name: kyuubi_USER_root_6f67c12c-f362-4a07-87d0-7aa7e5f43171
          application ID:  spark-461363e9cc934d1986f7ae4191dc0410
          application web UI: http://spark-740a727c11ce3da7-driver-svc.default.svc:44692
          master: k8s://https://10.10.9.35:6443
          version: 3.1.2
          driver: [cpu: 1, mem: 1G]
          executor: [cpu: 2, mem: 2G MB, maxNum: 18]
    Start time: Thu Sep 23 08:40:39 UTC 2021

    User: root (shared mode: USER)
    State: LATENT
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1141 from timothy65535/remove.

Closes #1141

af1827e9 [timothy65535] Remove redundant unit

Authored-by: timothy65535 <timothy65535@163.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
2021-09-24 11:23:45 +08:00
Cheng Pan
28adf6cdbc
[KYUUBI #884][FOLLOWUP] Fix catalog in KyuubiDatabaseMetaData#getTables
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
The behavior of `KyuubiDatabaseMetaData#getTables` changed by accident in #884[1], this PR restores the method's previous behavior and adds multi-catalog test case to verify it works as expected.

[1] [This](1a8b4ebaae (diff-c540737d1b293b909d07d50a05f67d35d274bc3163d723409135634a575e5e9dR68)) makes KyuubiDatabaseMetaData#getTables method make no sense, just invokes the superclass.

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1133 from pan3793/jdbc.

Closes #884

f02327d9 [Cheng Pan] add provided scala-library
3702a08a [Cheng Pan] nit
18045a92 [Cheng Pan] Fix test jars scope
edd1ca8f [Cheng Pan] Address comments
3fecbf5b [Cheng Pan] Clean withKyuubiConf instead of extraConfigs
45e7bd19 [Cheng Pan] nit
f19def37 [Cheng Pan] [KYUUBI #884][FOLLOWUP] Fix catalog in KyuubiDatabaseMetaData#getTables

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
(cherry picked from commit 997d72a1cd)
Signed-off-by: Cheng Pan <chengpan@apache.org>
2021-09-24 10:25:26 +08:00
Cheng Pan
1b40443d81
[KYUUBI #1134] [TEST] DataLake suites should clean up withKyuubiConf instead of extraConfigs 2021-09-23 09:33:00 +08:00
Cheng Pan
4dd4e83e91
[KYUUBI #1130] [DOCS] Restore Kyuubi logo image size
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1130 from pan3793/logo.

Closes #1130

e79da016 [Cheng Pan] [DOCS] Restore Kyuubi logo image size

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
(cherry picked from commit 126ba514dd)
Signed-off-by: Cheng Pan <chengpan@apache.org>
2021-09-22 13:54:47 +08:00
Cheng Pan
0843c82d4d
[KYUUBI #1125][FOLLOWUP] Compress and remove duplicated new logo and keep legacy logo
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1128 from pan3793/logo.

Closes #1125

affa2b1a [Cheng Pan] [KYUUBI #1125][FOLLOWUP] Compress and remove duplicated new logo and keep legacy logo

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
(cherry picked from commit e0f880433f)
Signed-off-by: Cheng Pan <chengpan@apache.org>
2021-09-22 09:20:02 +08:00
happytofly
f6dd8a0607
[KYUUBI #1125] A new version of logo flie, apache information included.
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1125 from happytofly/master.

Closes #1125

9e3ffc42 [happytofly] Add files via upload
f91316d7 [happytofly] Add files via upload

Authored-by: happytofly <systemvv@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
(cherry picked from commit a5e9c83040)
Signed-off-by: Cheng Pan <chengpan@apache.org>
2021-09-20 12:32:15 +08:00
Cheng Pan
f9ae303ece
[KYUUBI #1108] Register KyuubiDriver in ServiceLoader
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1108 from pan3793/jdbc.

Closes #1108

0f14d12a [Cheng Pan] Add License header
18e19c5f [Cheng Pan] Exclude META-INF/services from rat
5048c471 [Cheng Pan] Register KyuubiDriver in ServiceLoader

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
(cherry picked from commit 861bf6af31)
Signed-off-by: Cheng Pan <chengpan@apache.org>
2021-09-17 11:44:56 +08:00
Cheng Pan
bfdf6e1f3b
[KYUUBI #1104] Fix release script finalize_svn
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1104 from pan3793/release.

Closes #1104

f73706e1 [Cheng Pan] Fix release script finalize_svn

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
(cherry picked from commit 9e8b974c23)
Signed-off-by: Cheng Pan <chengpan@apache.org>
2021-09-15 09:21:07 +08:00
Cheng Pan
8f0a7aacff
[KYUUBI #1096] Remove never expired declaration in NOTICE
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
https://www.mail-archive.com/generalincubator.apache.org/msg75717.html

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1096 from pan3793/notice.

Closes #1096

c35e86b1 [Cheng Pan] Remove never expired declaration in NOTICE

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
(cherry picked from commit 7010b6e165)
Signed-off-by: Cheng Pan <chengpan@apache.org>
2021-09-14 10:50:46 +08:00
张宇翔
9b1fa5e0c5
[KYUUBI #1094] [BUGFIX #1068]Support upload file to HDFS
#1068
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1094 from zhang1002/branch-1.3_support-upload-file-to-hdfs.

Closes #1094

e96794ed [张宇翔] Support upload file to hdfs
ff237744 [张宇翔] Support the eventLog upload to hdfs

Authored-by: 张宇翔 <zhang1002@126.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
2021-09-13 20:10:37 +08:00
hongdongdong
6a183f9d1e
[KYUUBI #1057] [KYUUBI#1056] Output error log when currentEngine is None
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
When createSpark in SparkSqlEngine get sparksession or execute initialize.sql failed, we cannot get any error info in logs. At this time, currentEngine is None.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1057 from hddong/engine-fix.

Closes #1057

44ead3b3 [hongdongdong] fix
3462d36c [hongdongdong] [KYUUBI#1056] Output error log when currentEngine is None

Authored-by: hongdongdong <hongdongdong@cmss.chinamobile.com>
Signed-off-by: Kent Yao <yao@apache.org>
(cherry picked from commit 8886513534)
Signed-off-by: Kent Yao <yao@apache.org>
2021-09-09 13:37:49 +08:00
qiuliang
8be8c0f314
[KYUUBI #1034] Engine may deadlock when close operationLog
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1062 from qiuliang988/1034.

Closes #1034

41d510f5 [qiuliang] add comments
dbd7c770 [qiuliang] [KYUUBI #1034]Engine may deadlock when close operationLog

Authored-by: qiuliang <qlcumt@163.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
(cherry picked from commit d83674248e)
Signed-off-by: Cheng Pan <chengpan@apache.org>
2021-09-09 11:48:30 +08:00
Cheng Pan
cdd478400e
[RELEASE] Bump 1.3.1-SNAPSHOT 2021-09-06 15:14:05 +08:00
Cheng Pan
ff9bab6b7a
[RELEASE] Bump 1.3.0-incubating 2021-09-06 14:26:35 +08:00
Cheng Pan
c19e861709
[KYUUBI #1024][FOLLOWUP] Fix source release folder name
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1030 from pan3793/pkg.

Closes #1024

4fad69f9 [Cheng Pan] [KYUUBI #1024][FOLLOWUP] Fix source release folder name

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
(cherry picked from commit 057bac1913)
Signed-off-by: Cheng Pan <chengpan@apache.org>
2021-09-06 10:26:10 +08:00
Cheng Pan
92cc4cd058
[RELEASE] Bump 1.3.1-SNAPSHOT 2021-09-05 00:50:18 +08:00
Cheng Pan
fb3990c647
[RELEASE] Bump 1.3.0-incubating 2021-09-05 00:27:17 +08:00
Cheng Pan
1bcc494a63
[KYUUBI #1027] [BUILD] Use relative path to generate sha512 for release tarballs
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
Close #1025

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1027 from pan3793/sha.

Closes #1027

3e384b80 [Cheng Pan] [BUILD] Use relative path to generate sha512 for release tarballs

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
(cherry picked from commit 05fc1c0ba8)
Signed-off-by: Cheng Pan <chengpan@apache.org>
2021-09-05 00:23:15 +08:00
Cheng Pan
3312bbb2c0
[KYUUBI #1026] [BUILD] Rename tarball to apache-kyuubi
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
Close #1024

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1026 from pan3793/tarball.

Closes #1026

bc515aab [Cheng Pan] [BUILD] Rename tarball to apache-kyuubi

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
(cherry picked from commit fdd51d55f8)
Signed-off-by: Cheng Pan <chengpan@apache.org>
2021-09-05 00:21:42 +08:00
Cheng Pan
2bafbf38a2
[KYUUBI #1019] [BUILD] Remove developers from pom
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

> I would recommend not doing this and in particular not not adding organisation as could be taken as misrepresenting who owns the project. People must act as individuals in the interest of the project when working on a project and not the company they are working for.
>
> Kind Regards,
Justin

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1019 from pan3793/pom.

Closes #1019

11369b7d [Cheng Pan] [BUILD] Remove developers from pom

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
(cherry picked from commit 134da16f36)
Signed-off-by: Kent Yao <yao@apache.org>
2021-09-03 19:15:50 +08:00
ulysses-you
0a8f19b789
[KYUUBI #1020] Fix incubating issue
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
Make it more cleaner.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1020 from ulysses-you/incubating.

Closes #1020

5ae2d0d4 [ulysses-you] fix

Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: Kent Yao <yao@apache.org>
(cherry picked from commit 8217f3d92f)
Signed-off-by: Kent Yao <yao@apache.org>
2021-09-03 19:14:07 +08:00
Cheng Pan
d8f4be3b1c
[KYUUBI #1013] Enhance bootstrap shell script
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

To make the error message more clear when `SPARK_HOME` is invalid.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1013 from pan3793/sh.

Closes #1013

c9a573b1 [Cheng Pan] nit
e1628640 [Cheng Pan] indent with 2 spaces
68a767f5 [Cheng Pan] Address comments
59d4f076 [Cheng Pan] Enhance load-kyuubi-env.sh

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
(cherry picked from commit 5edc5066f2)
Signed-off-by: Kent Yao <yao@apache.org>
2021-09-02 10:15:20 +08:00
zhouyifan279
2a080d374b
[KYUUBI #1005] ServiceDiscoverySuite fails if hostname contains uppercase letter
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
Executing test case "ServiceDiscoverySuite set up zookeeper auth" on a Linux host named "KYUUBI" fails.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1006 from zhouyifan279/KYUUBI#1005.

Closes #1005

045c6b34 [zhouyifan279] [KYUUBI #1005] ServiceDiscoverySuite fails if hostname contains uppercase letter

Authored-by: zhouyifan279 <zhouyifan279@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
(cherry picked from commit 866a63d7a7)
Signed-off-by: Cheng Pan <chengpan@apache.org>
2021-08-31 20:36:46 +08:00
ulysses-you
7cef74e85d Revert "[KYUUBI #1000] Use underscore instead hyphen in path"
This reverts commit 92c2ebcbf3.
2021-08-30 17:09:16 +08:00
hongdongdong
92c2ebcbf3 [KYUUBI #1000] Use underscore instead hyphen in path
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
Use underscore instead hyphen in path, friendly to sql.

- [X] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1000 from hddong/fix-underscore.

Closes #1000

8b3459a5 [hongdongdong] Use underscore instead hyphen in path

Authored-by: hongdongdong <hongdongdong@cmss.chinamobile.com>
Signed-off-by: ulysses-you <ulyssesyou18@gmail.com>
(cherry picked from commit 94d60ea505)
Signed-off-by: ulysses-you <ulyssesyou18@gmail.com>
2021-08-30 17:06:12 +08:00
Cheng Pan
328d5b2d47
[RELEASE] Bump 1.3.1-SNAPSHOT 2021-08-30 15:28:36 +08:00
Cheng Pan
f1319e5098
[RELEASE] Bump 1.3.0-incubating 2021-08-30 15:04:29 +08:00
Cheng Pan
eb1832c703
[KYUUBI #998] Update release doc
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #998 from pan3793/gpg.

Closes #998

f32c561e [Cheng Pan] Update doc
07d5fe3d [Cheng Pan] Update release doc

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2021-08-30 14:58:39 +08:00
fwang12
0d5ccab9e9
[KYUUBI #997] Fix build error when project path contains kyuubi-
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
The building will fail if the project name contains `kyuubi-`.
For example: for our jenkins job, its name is `kyuubi-binary-build`, and the TMP dir won't be created because all the jar files are filtered because their absolute path contains `kyuubi-`.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #997 from turboFei/fix_collect_license.

Closes #997

eef47b79 [fwang12] follow up
45fe7834 [fwang12] replace tab with 2 spaces
f6589403 [fwang12] refactor
c0cddd96 [fwang12] fix build error

Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
(cherry picked from commit 7141f2f13e)
Signed-off-by: Cheng Pan <chengpan@apache.org>
2021-08-30 13:41:54 +08:00
Cheng Pan
4c47981626
[RELEASE] Bump 1.3.1-SNAPSHOT 2021-08-30 12:49:28 +08:00
Cheng Pan
2536309355
[RELEASE] Bump 1.3.0-incubating 2021-08-30 10:04:28 +08:00
Cheng Pan
0f6e76cd2d
[RELEASE] Bump 1.3.1-SNAPSHOT 2021-08-27 15:09:37 +08:00
Cheng Pan
1ec75b3ef2
[RELEASE] Bump 1.3.0-incubating 2021-08-27 14:54:31 +08:00
ulysses-you
6020b831a7
[KYUUBI #992] Fix SPARK_EXTENSION_VERSION_ is seen as a variable
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
fix dist bug

`$SPARK_EXTENSION_VERSION_` is empty
`${SPARK_EXTENSION_VERSION}_` is `3-1_`

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #992 from ulysses-you/fix-SPARK_EXTENSION_VERSION_.

Closes #992

4379093e [ulysses-you] fix

Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
(cherry picked from commit c59e54a675)
Signed-off-by: Cheng Pan <chengpan@apache.org>
2021-08-26 19:20:56 +08:00
Fu Chen
083d051bcc [KYUUBI #991] [MINOR] improve the error message of SPARK package not found.
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

As `File.listFiles` may return null when spark package not found in folder `${PROJECT_ROOT_DIR}/kyuubi-server/externals/kyuubi-download/target`, the NPE will throw. Improve the error message.

Before thie pr:

```
Caused by: java.lang.RuntimeException: org.apache.kyuubi.KyuubiSQLException:Error opening session SessionHandle [9749221f-5b30-457b-b8e5-25affba25061] for fchen due to null
	at org.apache.kyuubi.KyuubiSQLException$.apply(KyuubiSQLException.scala:68)
	at org.apache.kyuubi.session.KyuubiSessionManager.openSession(KyuubiSessionManager.scala:74)
	at org.apache.kyuubi.service.AbstractBackendService.openSession(AbstractBackendService.scala:45)
	at org.apache.kyuubi.service.ThriftFrontendService.getSessionHandle(ThriftFrontendService.scala:190)
	at org.apache.kyuubi.service.ThriftFrontendService.OpenSession(ThriftFrontendService.scala:199)
	at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1377)
	at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1362)
	at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
	at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
	at org.apache.kyuubi.service.authentication.TSetIpAddressProcessor.process(TSetIpAddressProcessor.scala:36)
	at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException: null
	at scala.collection.mutable.ArrayOps$ofRef$.length$extension(ArrayOps.scala:204)
	at scala.collection.mutable.ArrayOps$ofRef.length(ArrayOps.scala:204)
	at scala.collection.IndexedSeqOptimized.isEmpty(IndexedSeqOptimized.scala:30)
	at scala.collection.IndexedSeqOptimized.isEmpty$(IndexedSeqOptimized.scala:30)
	at scala.collection.mutable.ArrayOps$ofRef.isEmpty(ArrayOps.scala:198)
	at scala.collection.TraversableLike.headOption(TraversableLike.scala:608)
	at scala.collection.TraversableLike.headOption$(TraversableLike.scala:608)
	at scala.collection.mutable.ArrayOps$ofRef.headOption(ArrayOps.scala:198)
	at org.apache.kyuubi.engine.spark.SparkProcessBuilder.$anonfun$executable$1(SparkProcessBuilder.scala:50)
	at scala.Option.orElse(Option.scala:447)
	at org.apache.kyuubi.engine.spark.SparkProcessBuilder.<init>(SparkProcessBuilder.scala:41)
	at org.apache.kyuubi.engine.EngineRef.$anonfun$create$1(EngineRef.scala:140)
	at org.apache.kyuubi.engine.EngineRef.tryWithLock(EngineRef.scala:116)
	at org.apache.kyuubi.engine.EngineRef.create(EngineRef.scala:128)
	at org.apache.kyuubi.engine.EngineRef.$anonfun$getOrCreate$1(EngineRef.scala:182)
	at scala.Option.getOrElse(Option.scala:189)
	at org.apache.kyuubi.engine.EngineRef.getOrCreate(EngineRef.scala:182)
	at org.apache.kyuubi.session.KyuubiSessionImpl.$anonfun$open$2(KyuubiSessionImpl.scala:63)
	at org.apache.kyuubi.session.KyuubiSessionImpl.$anonfun$open$2$adapted(KyuubiSessionImpl.scala:62)
	at org.apache.kyuubi.ha.client.ZooKeeperClientProvider$.withZkClient(ZooKeeperClientProvider.scala:74)
	at org.apache.kyuubi.session.KyuubiSessionImpl.open(KyuubiSessionImpl.scala:62)
	at org.apache.kyuubi.session.KyuubiSessionManager.openSession(KyuubiSessionManager.scala:58)
	... 12 more
```
After this pr:

```
Caused by: java.lang.RuntimeException: org.apache.kyuubi.KyuubiSQLException:Error opening session SessionHandle [5ea6a8b3-1727-4e0d-b542-992f84afcde8] for fchen due to SPARK_HOME is not set!
	at org.apache.kyuubi.KyuubiSQLException$.apply(KyuubiSQLException.scala:68)
	at org.apache.kyuubi.session.KyuubiSessionManager.openSession(KyuubiSessionManager.scala:74)
	at org.apache.kyuubi.service.AbstractBackendService.openSession(AbstractBackendService.scala:45)
	at org.apache.kyuubi.service.ThriftFrontendService.getSessionHandle(ThriftFrontendService.scala:190)
	at org.apache.kyuubi.service.ThriftFrontendService.OpenSession(ThriftFrontendService.scala:199)
	at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1377)
	at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1362)
	at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
	at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
	at org.apache.kyuubi.service.authentication.TSetIpAddressProcessor.process(TSetIpAddressProcessor.scala:36)
	at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: org.apache.kyuubi.KyuubiSQLException:SPARK_HOME is not set!
	at org.apache.kyuubi.KyuubiSQLException$.apply(KyuubiSQLException.scala:68)
	at org.apache.kyuubi.engine.spark.SparkProcessBuilder.$anonfun$executable$5(SparkProcessBuilder.scala:61)
	at scala.Option.getOrElse(Option.scala:189)
	at org.apache.kyuubi.engine.spark.SparkProcessBuilder.<init>(SparkProcessBuilder.scala:61)
	at org.apache.kyuubi.engine.EngineRef.$anonfun$create$1(EngineRef.scala:140)
	at org.apache.kyuubi.engine.EngineRef.tryWithLock(EngineRef.scala:116)
	at org.apache.kyuubi.engine.EngineRef.create(EngineRef.scala:128)
	at org.apache.kyuubi.engine.EngineRef.$anonfun$getOrCreate$1(EngineRef.scala:182)
	at scala.Option.getOrElse(Option.scala:189)
	at org.apache.kyuubi.engine.EngineRef.getOrCreate(EngineRef.scala:182)
	at org.apache.kyuubi.session.KyuubiSessionImpl.$anonfun$open$2(KyuubiSessionImpl.scala:63)
	at org.apache.kyuubi.session.KyuubiSessionImpl.$anonfun$open$2$adapted(KyuubiSessionImpl.scala:62)
	at org.apache.kyuubi.ha.client.ZooKeeperClientProvider$.withZkClient(ZooKeeperClientProvider.scala:74)
	at org.apache.kyuubi.session.KyuubiSessionImpl.open(KyuubiSessionImpl.scala:62)
	at org.apache.kyuubi.session.KyuubiSessionManager.openSession(KyuubiSessionManager.scala:58)
	... 12 more
```

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #991 from cfmcgrady/improve-error-msg.

Closes #991

1e8da683 [Fu Chen] imporove the error message

Authored-by: Fu Chen <cfmcgrady@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou18@gmail.com>
(cherry picked from commit 9695f75889)
Signed-off-by: ulysses-you <ulyssesyou18@gmail.com>
2021-08-26 16:53:42 +08:00
Cheng Pan
57dd860721
[RELEASE] Bump 1.3.1-SNAPSHOT 2021-08-25 20:23:40 +08:00
Cheng Pan
95557b5302
[RELEASE] Bump 1.3.0-incubating 2021-08-25 20:02:43 +08:00
ulysses-you
3c8cce414d
[KYUUBI #985] Add logCaptureThreadReleased flag
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
`isInterrupted` is always false if some `InterruptedException` are throwed. So we should use a new flag to check if log capture thread is released.

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #985 from ulysses-you/flaky-test.

Closes #985

1fcc0a99 [ulysses-you] empty
463a996f [ulysses-you] empty
d1f2ea54 [ulysses-you] eventually
f9f37343 [ulysses-you] fix

Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
(cherry picked from commit 3afd20e03a)
Signed-off-by: Cheng Pan <chengpan@apache.org>
2021-08-25 19:21:53 +08:00
hongdongdong
e624e7b8f7 [KYUUBI #982] Fix the typo issue
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
Fix the typo issue

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #982 from hddong/fix-typo-issue.

Closes #982

d6f6315d [hongdongdong] Fix the typo issue

Authored-by: hongdongdong <hongdongdong@cmss.chinamobile.com>
Signed-off-by: ulysses-you <ulyssesyou18@gmail.com>
2021-08-25 15:29:15 +08:00
Cheng Pan
6ccc00a791
[RELEASE] Bump 1.3.1-SNAPSHOT 2021-08-25 13:59:33 +08:00
Cheng Pan
504b10384b
[RELEASE] Bump 1.3.0-incubating 2021-08-25 13:47:58 +08:00
Cheng Pan
0bcef281b8
[KYUUBI #976] [BUILD] Fix source release build
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->
Support build, run unit tests, and create binary tarball from source release.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

The verify steps:

1. create a source release tarball
```
RELEASE_VERSION=1.4.0-SNAPSHOT build/release/create-package.sh source
```
2. untar tarball
```
cd work/release
tar -xzf kyuubi-1.4.0-SNAPSHOT-bin.tgz
cd kyuubi-1.4.0-SNAPSHOT-bin
```
3. compile and run unit test
```
build/mvn clean install
```
4. create binary release tarball
```
build/dist --tgz --spark-provided -Pkyuubi-extension-spark-3-1
```

Closes #976 from pan3793/source-build.

Closes #976

354968ae [Cheng Pan] Compress imgs
9c47b64f [Cheng Pan] Restore docs/imgs in source release
5d86ecc7 [Cheng Pan] Clean up imgs
a3399279 [Cheng Pan] Support create binary from source release
bb279863 [Cheng Pan] [BUILD] Fix source release ut failures

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
(cherry picked from commit d58a092245)
Signed-off-by: Cheng Pan <chengpan@apache.org>
2021-08-25 13:35:36 +08:00