Commit Graph

71 Commits

Author SHA1 Message Date
Cheng Pan
0559fda948 [RELEASE] Bump 1.10.3-SNAPSHOT 2025-05-22 16:26:33 +00:00
Cheng Pan
494c42517c [RELEASE] Bump 1.10.2 2025-05-22 16:25:38 +00:00
Cheng Pan
b2d540f664 [RELEASE] Bump 1.10.2-SNAPSHOT 2024-12-23 02:14:06 +00:00
Cheng Pan
ba7ad038c6 [RELEASE] Bump 1.10.1 2024-12-23 02:13:52 +00:00
Bowen Liang
b05ddbec5b Bump 1.10.1-SNAPSHOT 2024-11-01 10:22:38 +08:00
Bowen Liang
21a34caf5a [RELEASE] Bump 1.10.0 2024-10-23 17:27:45 +08:00
Cheng Pan
53b3911935
Revert "[KYUUBI #6526] Kyuubi BeeLine wrongly process JDBC URL that contains --"
This reverts commit e26b57acda.
2024-07-08 16:39:11 +08:00
axiangzheng
e26b57acda
[KYUUBI #6526] Kyuubi BeeLine wrongly process JDBC URL that contains --
# 🔍 Description
## Issue References 🔗

This pull request fixes https://github.com/apache/kyuubi/issues/6522

## Describe Your Solution 🔧

Remove comments in dispatch commands only execute command

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6526 from zcx513566/issues_6522.

Closes #6526

3dfd94364 [axiangzheng] [Bug] Use kyuubi beeline connect kyuubi server error if url contains '--' string
6677497ee [axiangzheng] [Bug] Use kyuubi beeline connect kyuubi server error if url contains '--' string

Authored-by: axiangzheng <axiangzheng@tencent.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2024-07-08 14:17:40 +08:00
Bruce Wong
28a1c735af
[KYUUBI #6506] kyuubi-beeline supports --conf
# 🔍 Description
## Issue References 🔗

This pull request fixes #6506

## Describe Your Solution 🔧

Set the alias `--conf` to `--hiveconf`.

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️
```
# use --hiveconf or --hivevar to set some configurations
kyuubi-beeline -u 'jdbc:kyuubi://kyuubi:10009/' \
    --hiveconf kyuubi.operation.result.format=arrow \
    --hiveconf kyuubi.operation.incremental.collect=true-f \
    --hivevar app_name=xxx \
    xxx.sql
```

#### Behavior With This Pull Request 🎉
```
# use --conf to set some configurations
kyuubi-beeline -u 'jdbc:kyuubi://kyuubi:10009/' \
    --conf kyuubi.operation.result.format=arrow \
    --conf kyuubi.operation.incremental.collect=true-f \
    --conf app_name=xxx \
    xxx.sql
```

#### Related Unit Tests

---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6511 from BruceWong96/support-beeline-conf.

Closes #6506

7a81455d8 [Bruce Wong] fix code style.
8d1fec242 [Bruce Wong] delete some spaces.
d64505dd7 [Bruce Wong] [FEATURE] kyuubi-beeline supports --conf.

Authored-by: Bruce Wong <603334301@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2024-06-27 22:12:25 +08:00
Cheng Pan
845f6c63a9
[KYUUBI #6507] Correct Kyuubi BeeLine help message
# 🔍 Description

Hive 2.3.0 (HIVE-7224) turns it on by default, while Kyuubi BeeLine is a fork from Hive BeeLine 3.1.3, we should update the help message to reflect the change.

## Types of changes 🔖

- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

Pass GHA.

---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6507 from pan3793/beeline-incr.

Closes #6507

8b1ed655a [Cheng Pan] fix
7a872eeec [Cheng Pan] fix
88e751d3a [Cheng Pan] Correct Kyuubi BeeLine help message for incremental

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2024-06-27 17:36:48 +08:00
hezhao2
cfc23a1df9
[KYUUBI #6339] kyuubi-beeline auto constructs JDBC URL from kyuubi-defaults.conf
# 🔍 Description
## Issue References 🔗

as title
This pull request closes  #6255

## Describe Your Solution 🔧

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6339 from zhaohehuhu/dev-0426.

Closes #6339

38b36f260 [Cheng Pan] fix
b2e3fcf3d [Cheng Pan] rewrite

Lead-authored-by: hezhao2 <hezhao2@cisco.com>
Co-authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2024-06-25 13:19:46 +08:00
Cheng Pan
315adda353
[KYUUBI #6499] Rewrite some utility methods in Java
# 🔍 Description

This PR rewrites some utility methods in Java, specifically,
```
Utils.isWindows
Utils.isMac
Utils.findLocalInetAddress
```

and moves them from `kyuubi-common`'s `Utils` to the `kyuubi-util`'s `JavaUtils`, so that they could be used in other modules that do not depend on `kyuubi-common`.

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

Pass GHA.

---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6499 from pan3793/javautils.

Closes #6499

565936def [Cheng Pan] fix
f06a85e9f [Cheng Pan] Move some untiliy methods in Java

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2024-06-25 09:59:03 +08:00
dupeng
37bf244236
[KYUUBI #6251] Improve kyuubi-beeline help message
# 🔍 Description
## Issue References 🔗

This pull request fixes #6251

## Describe Your Solution 🔧
As `kyuubi-beeline` is derived from Hive `beeline`, I haven't made extensive modifications to its help message, only adjusting some formatting to enhance its appearance. Below are the specific changes:
1. Replace `jdbc:hive2//` with `jdbc:kyuubi//`.
2. Replace `beeline` with `kyuubi-beeline`.
3. Capitalize the first letter.
4. If the comment for a parameter spans multiple lines, add an additional `\n` at the end of the comment to separate it from the following line.
5. Append the help information for `--python-mode` to the front of `--help`.
6. Add some examples.
7. Improved letter indentation.

Here is the whole help message:
```
$ ./bin/kyuubi-beeline --help
Usage: kyuubi-beeline <options>.

Options:
  -u <database url>               The JDBC URL to connect to.
  -c <named url>                  The named JDBC URL to connect to,
                                  which should be present in beeline-site.xml
                                  as the value of beeline.kyuubi.jdbc.url.<namedUrl>.

  -r                              Reconnect to last saved connect url (in conjunction with !save).
  -n <username>                   The username to connect as.
  -p <password>                   The password to connect as.
  -d <driver class>               The driver class to use.
  -i <init file>                  Script file for initialization.
  -e <query>                      Query that should be executed.
  -f <exec file>                  Script file that should be executed.
  -w, --password-file <file>      The password file to read password from.
  --hiveconf property=value       Use value for given property.
  --hivevar name=value            Hive variable name and value.
                                  This is Hive specific settings in which variables
                                  can be set at session level and referenced in Hive
                                  commands or queries.

  --property-file=<property file> The file to read connection properties (url, driver, user, password) from.
  --color=[true|false]            Control whether color is used for display.
  --showHeader=[true|false]       Show column names in query results.
  --escapeCRLF=[true|false]       Show carriage return and line feeds in query results as escaped \r and \n.
  --headerInterval=ROWS;          The interval between which heades are displayed.
  --fastConnect=[true|false]      Skip building table/column list for tab-completion.
  --autoCommit=[true|false]       Enable/disable automatic transaction commit.
  --verbose=[true|false]          Show verbose error messages and debug info.
  --showWarnings=[true|false]     Display connection warnings.
  --showDbInPrompt=[true|false]   Display the current database name in the prompt.
  --showNestedErrs=[true|false]   Display nested errors.
  --numberFormat=[pattern]        Format numbers using DecimalFormat pattern.
  --force=[true|false]            Continue running script even after errors.
  --maxWidth=MAXWIDTH             The maximum width of the terminal.
  --maxColumnWidth=MAXCOLWIDTH    The maximum width to use when displaying columns.
  --silent=[true|false]           Be more silent.
  --autosave=[true|false]         Automatically save preferences.
  --outputformat=<format mode>    Format mode for result display.
                                  The available options ars [table|vertical|csv2|tsv2|dsv|csv|tsv|json|jsonfile].
                                  Note that csv, and tsv are deprecated, use csv2, tsv2 instead.

  --incremental=[true|false]      Defaults to false. When set to false, the entire result set
                                  is fetched and buffered before being displayed, yielding optimal
                                  display column sizing. When set to true, result rows are displayed
                                  immediately as they are fetched, yielding lower latency and
                                  memory usage at the price of extra display column padding.
                                  Setting --incremental=true is recommended if you encounter an OutOfMemory
                                  on the client side (due to the fetched result set size being large).
                                  Only applicable if --outputformat=table.

  --incrementalBufferRows=NUMROWS The number of rows to buffer when printing rows on stdout,
                                  defaults to 1000; only applicable if --incremental=true
                                  and --outputformat=table.

  --truncateTable=[true|false]    Truncate table column when it exceeds length.
  --delimiterForDSV=DELIMITER     Specify the delimiter for delimiter-separated values output format (default: |).
  --isolation=LEVEL               Set the transaction isolation level.
  --nullemptystring=[true|false]  Set to true to get historic behavior of printing null as empty string.
  --maxHistoryRows=MAXHISTORYROWS The maximum number of rows to store beeline history.
  --delimiter=DELIMITER           Set the query delimiter; multi-char delimiters are allowed, but quotation
                                  marks, slashes, and -- are not allowed (default: ;).

  --convertBinaryArrayToString=[true|false]
                                  Display binary column data as string or as byte array.

  --python-mode                   Execute python code/script.
  -h, --help                      Display this message.

Examples:
  1. Connect using simple authentication to Kyuubi Server on localhost:10009.
  $ kyuubi-beeline -u jdbc:kyuubi://localhost:10009 -n username

  2. Connect using simple authentication to Kyuubi Server on kyuubi.local:10009 using -n for username and -p for password.
  $ kyuubi-beeline -n username -p password -u jdbc:kyuubi://kyuubi.local:10009

  3. Connect using Kerberos authentication with kyuubi/localhostmydomain.com as Kyuubi Server principal(kinit is required before connection).
  $ kyuubi-beeline -u "jdbc:kyuubi://kyuubi.local:10009/default;kyuubiServerPrincipal=kyuubi/localhostmydomain.com"

  4. Connect using Kerberos authentication using principal and keytab directly.
  $ kyuubi-beeline -u "jdbc:kyuubi://kyuubi.local:10009/default;kyuubiClientPrincipal=usermydomain.com;kyuubiClientKeytab=/local/path/client.keytab;kyuubiServerPrincipal=kyuubi/localhostmydomain.com"

  5. Connect using SSL connection to Kyuubi Server on localhost:10009.
  $ kyuubi-beeline -u "jdbc:kyuubi://localhost:10009/default;ssl=true;sslTrustStore=/usr/local/truststore;trustStorePassword=mytruststorepassword"

  6. Connect using LDAP authentication.
  $ kyuubi-beeline -u jdbc:kyuubi://kyuubi.local:10009/default -n ldap-username -p ldap-password

  7. Connect using the ZooKeeper address to Kyuubi HA cluster.
  $ kyuubi-beeline -u "jdbc:kyuubi://zk1:2181,zk2:2181,zk3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=kyuubi" -n username
```
If you have any suggested revisions, I will promptly respond and make the necessary adjustments.

## Types of changes 🔖

- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6300 from dupen01/issue-6251.

Closes #6251

69f9b3dc9 [dupeng] deleted usage() from KyuubiBeeLine.java;updated some examples
463bcfb85 [dupeng] update --delimiter description
95e13c0b6 [dupeng] update some example message
6625f99b5 [dupeng] update cmd-usage
6570ec255 [dupeng] update cmd-usage
59ea1a495 [dupeng] add kyuubi-beeline help message

Authored-by: dupeng <dunett@163.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2024-04-12 19:21:08 +08:00
Shilun Fan
03fa95138b
[KYUUBI #6266] Kyuubi BeeLine supports JSON output format.
# 🔍 Description
## Issue References 🔗

This pull request fixes #6266

## Describe Your Solution 🔧

Kyuubi BeeLine supports JSON output format.

This pr cherrypick HIVE-20447, during this process, no incompatibility occurred in the code.

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

New UT

---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6282 from slfan1989/beeline_support_json.

Closes #6266

0749500e9 [Shilun Fan] [KYUUBI #6266] Kyuubi BeeLine supports JSON output format.

Authored-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2024-04-10 13:23:38 +08:00
Binjie Yang
eb278c562d
[RELEASE] Bump 1.10.0-SNAPSHOT 2024-03-13 14:24:49 +08:00
Cheng Pan
ddf347ff77
[KYUUBI #6162] Cut out hive-common deps in beeline module
# 🔍 Description
## Issue References 🔗

This PR removes the `hive-common` dep from the beeline module, and it is the final step of cutting out all Hive deps from the beeline module.

## Describe Your Solution 🔧

- Remove support of Hive CLI mode (a.k.a. `beelineMode == false`) on beeline.
- Copy ~7 classes (~600 lines) from `org.apache.hive:hive-common:3.1.3`
- Drop `org.apache.hive:hive-common:3.1.3` from beeline module

Note: Hive CLI mode removal is necessary, because it is useless for Kyuubi cases and seriously coupled with `HiveConf`

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

Pass GA, and manually tested.

```
roothadoop-master1:/opt/kyuubi# beeline
Connecting to jdbc:hive2://hadoop-master1.orb.local:10000/default;password=hive;user=hive
Connected to: Apache Hive (version 2.3.9)
Driver: Kyuubi Project Hive JDBC Client (version 1.9.0-SNAPSHOT)
Beeline version 1.9.0-SNAPSHOT by Apache Kyuubi
0: jdbc:hive2://hadoop-master1.orb.local:1000> select version();
+--------------------------------------------------+
|                       _c0                        |
+--------------------------------------------------+
| 2.3.9 r92dd0159f440ca7863be3232f3a683a510a62b9d  |
+--------------------------------------------------+
1 row selected (1.443 seconds)
0: jdbc:hive2://hadoop-master1.orb.local:1000>
```

---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6162 from pan3793/beeline-3.

Closes #6162

e85f096b2 [Cheng Pan] relocate
b2ce8ef61 [Cheng Pan] Cut out hive-common deps in beeline module

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2024-03-12 11:23:10 +08:00
zhouyifan279
ca53942d13
[KYUUBI #6160] Fix beeline test KyuubiBeeLineTest.testKyuubiBeelineComment
# 🔍 Description
## Issue References 🔗

Shell command `beeline` uses `KyuubiBeeline#initArgs`, and never uses `KyuubiBeeline#initArgsFromCliVars`.
We should replace `KyuubiBeeline#initArgsFromCliVars` with `KyuubiBeeline#initArgs` in test `KyuubiBeeLineTest#testKyuubiBeelineComment`.

## Describe Your Solution 🔧

## Types of changes 🔖

- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6160 from zhouyifan279/beeline-comment-test.

Closes #6160

c250af07f [zhouyifan279] Fix beeline test KyuubiBeeLineTest.testKyuubiBeelineComment
935aa371b [zhouyifan279] Fix flaky test KyuubiBeeLineTest.testKyuubiBeelineComment
29bf670dd [zhouyifan279] Fix flaky test KyuubiBeeLineTest.testKyuubiBeelineComment
f7a07425e [zhouyifan279] Fix flaky test KyuubiBeeLineTest.testKyuubiBeelineComment

Authored-by: zhouyifan279 <zhouyifan279@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2024-03-11 21:57:22 +08:00
Cheng Pan
869400ddb8
Revert "[KYUUBI #5918] Kyuubi BeeLine should check the relocated TTransportException"
This reverts commit 2e817d5be8.
2024-03-11 15:29:35 +08:00
Cheng Pan
9fd392b7e9
[KYUUBI #6153] Cut out Hive JDBC and Hive Service deps in beeline module
# 🔍 Description
## Issue References 🔗

This is the next step of https://github.com/apache/kyuubi/issues/6146, cutting out most Hive deps(excepting `hive-common`) and recovering the skipped tests via minor code tunning.

## Describe Your Solution 🔧

- Drop `hive-jdbc`, `hive-service`, `hive-service-rpc` deps in the beeline module.
- Migrate from `commons-lang` to `commons-lang3` in the beeline module.
- Recover the skipped test `TestClientCommandHookFactory#connectHook`

## Types of changes 🔖

- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

Pass GA, and manually test to ensure the following error has gone.

Before
```
roothadoop-master1:/opt/kyuubi# bin/beeline --version
Warn: Not find kyuubi environment file /etc/kyuubi/conf/kyuubi-env.sh, using default ones...
java.lang.NoClassDefFoundError: org/apache/curator/RetryPolicy
```

After
```
roothadoop-master1:/opt/kyuubi# bin/beeline --version
Connecting to jdbc:hive2://hadoop-master1.orb.local:10000/default;password=hive;user=hive
Connected to: Apache Hive (version 2.3.9)
Driver: Kyuubi Project Hive JDBC Client (version 1.9.0-SNAPSHOT)
Beeline version 1.9.0-SNAPSHOT by Apache Kyuubi
0: jdbc:hive2://hadoop-master1.orb.local:1000>
```

---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6153 from pan3793/beeline-2.

Closes #6153

8cd52e509 [Cheng Pan] notice
d03c72992 [Cheng Pan] minor
5d16bf4ce [Cheng Pan] beeline test pass

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2024-03-11 15:21:14 +08:00
Cheng Pan
64058a0178
[KYUUBI #6109] Fork beeline module from Apache Hive 3.1.3
# 🔍 Description
## Issue References 🔗

This is the first step of #6146, to gain a clear commit history, this PR just simply copied the `hive-beeline` module from Apache Hive 3.1.3, with minimal change to pass the tests and manually test basic functionalities, following PRs are going to remove other Hive deps gradually.

## Describe Your Solution 🔧

- Copy source code and test case from Apache Hive 3.1.3
- Drop `org.apache.hive:hive-beeline:3.1.3`
- Backport HIVE-21584 to support JDK 9+
- Drop `HiveCli`, `HiveSchemaTool` and `BeelineInPlaceUpdateStream`, and the corresponding test cases
- Temporary ignore(will fix later) `TestClientCommandHookFactory#testConnectHook` because of error `NoClassDefFound org/apache/curator/RetryPolicy`
- Tune testing code to pass UT

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

Minimal changes to pass the unit tests.

---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6109 from pan3793/fork-beeline.

Closes #6109

885f9fe2b [Cheng Pan] NOTICE
a2efa1c41 [Cheng Pan] fix
5bb1cc9f6 [Cheng Pan] Copy from Apache Hive 3.1.3

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2024-03-11 11:26:39 +08:00
Cheng Pan
182f4349a7
[KYUUBI #6117] Bump Kyuubi Shaded 0.3.0
# 🔍 Description
## Issue References 🔗

This PR upgrades Kyuubi Shaded to 0.3.0, the major changes are:

- upgrade Thrift from 0.9.3-1 to 0.16.0
- upgrade JLine used by Zookeeper CLI from 0.9.94 to 2.14.6
- split kyuubi-relocated-thrift from kyuubi-relocated-hive-service-rpc
- introduce a light kyuubi-relocated-hive-metastore-client for getting DelegationToken

Due to THRIFT-5297 (fixed in 0.14.0) removes requestTimeout and beBackoffSlotLength, the following Kyuubi configurations are removed too.

- kyuubi.frontend.login.timeout
- kyuubi.frontend.thrift.login.timeout
- kyuubi.frontend.backoff.slot.length
- kyuubi.frontend.thrift.backoff.slot.length

Previously, Zookeeper 3.4 depends on JLine 0.9.94 while Zookeeper 3.6 and Hive Beeline depends on JLine 2.14.6, we pulls different versions of JLine into the binary distribution. Now we align them to a single JLine 2.14.6.

## Describe Your Solution 🔧

Upgrade dependencies and change code to adapt to the breaking changes, also upgrade migration guide to mention the removed configurations.

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

Pass GA.

---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6117 from pan3793/shaded-0.3.

Closes #6117

5639c9fd8 [Cheng Pan] nit
d281cdb4b [Cheng Pan] fix
184e1b95b [Cheng Pan] fix
1628337c8 [Cheng Pan] remove unused conf
24db2d5ad [Cheng Pan] try fix
1e995bb34 [Cheng Pan] nit
1d8e9bce9 [Cheng Pan] dep list
7f33624e8 [Cheng Pan] Bump Kyuubi Shaded 0.3.0

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2024-03-08 10:19:30 +08:00
kandy01.wang
2e817d5be8
[KYUUBI #5918] Kyuubi BeeLine should check the relocated TTransportException
# 🔍 Description
## Issue References 🔗

This pull request fixes #5918, when Kyuubi beeline executes incorrect SQL,it will exit with exception:
```
java.lang.NoClassDefFoundError: org/apache/thrift/transport/TTransportException
```

## Describe Your Solution 🔧
1.Supply `org.apache.thrift:libthrift` to `kyuubi-beeline` module.
2.We should override BeeLine's handleSQLException method,makes it compatible
  As we discussed, only `org.apache.kyuubi.shaded.thrift.transport.TTransportException` will be received by `kyuubi-beeline`,so it‘s  unnecessary  to add libthrift dependency to `kyuubi-beeline`.

## Types of changes 🔖

- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklists
## 📝 Author Self Checklist

- [ ] My code follows the [style guidelines](https://kyuubi.readthedocs.io/en/master/contributing/code/style.html) of this project
- [ ] I have performed a self-review
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

## 📝 Committer Pre-Merge Checklist

- [x] Pull request title is okay.
- [x] No license issues.
- [x] Milestone correctly set?
- [ ] Test coverage is ok
- [x] Assignees are selected.
- [x] Minimum number of approvals
- [x] No changes are requested

**Be nice. Be informative.**

Closes #5919 from hadoopkandy/KYUUBI-5918.

Closes #5918

ee96c4436 [kandy01.wang] [KYUUBI #5918] [Bug] kyuubi beeline exit with exception : java.lang.NoClassDefFoundError: org/apache/thrift/transport/TTransportException

Authored-by: kandy01.wang <kandy01.wang@vipshop.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2023-12-27 21:46:48 +08:00
Cheng Pan
9be0c65fe9
[KYUUBI #5783] Switch to kyuubi-relocated-hive-service-rpc
# 🔍 Description
## Issue References 🔗

TL;DR there are some issues with shading Thrift RPC classes during the engine packaging phase, see details in the PR description of https://github.com/apache/kyuubi-shaded/pull/20.

## Describe Your Solution 🔧

This PR aims to migrate from vanilla `hive-service-rpc`, `libfb303`, `libthrift` to `kyuubi-relocated-hive-service-rpc` introduced in https://github.com/apache/kyuubi-shaded/pull/20, the detailed works are:

- replace imported deps in `pom.xml` and rename the package prefix in all modules, except for
  - `kyuubi-server` there are a few places use vanilla thrift classes to access HMS to get token
  - `kyuubi-hive-sql-engine` Hive method invocation
- update relocations rules in modules that creates shaded jar
- introduce `HiveRpcUtils` in `kyuubi-hive-sql-engine` module for object conversion.

As part of the whole change, this PR upgrades from the Kyuubi Shaded 0.1.0 to 0.2.0, which changes the jars name. see https://kyuubi.apache.org/shaded-release/0.2.0.html

## Types of changes 🔖

- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

Pass all Hive UT with Hive 3.1.3, and IT with Hive 3.1.3 and 2.3.9 (also tested with 2.1.1-cdh6.3.2)

---

# Checklists
## 📝 Author Self Checklist

- [ ] My code follows the [style guidelines](https://kyuubi.readthedocs.io/en/master/contributing/code/style.html) of this project
- [x] I have performed a self-review
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my feature works
- [x] New and existing unit tests pass locally with my changes
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

## 📝 Committer Pre-Merge Checklist

- [x] Pull request title is okay.
- [x] No license issues.
- [x] Milestone correctly set?
- [x] Test coverage is ok
- [x] Assignees are selected.
- [x] Minimum number of approvals
- [x] No changes are requested

**Be nice. Be informative.**

Closes #5783 from pan3793/rpc-shaded.

Closes #5783

b45d4deaa [Cheng Pan] remove staging repo
890076a20 [Cheng Pan] Kyuubi Shaded 0.2.0 RC0
071945d45 [Cheng Pan] Rebase
199794ed9 [Cheng Pan] fix
fc128b170 [Cheng Pan] fix
26d313896 [Cheng Pan] fix
632984c92 [Cheng Pan] fix
428305589 [Cheng Pan] fix
6301e28fd [Cheng Pan] fix
955cdb33b [Cheng Pan] Switch to kyuubi-shaded-hive-service-rpc

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2023-12-07 19:55:10 +08:00
kandy01.wang
5b3a78d7df
[KYUUBI #5822] Fix the usage msg of KyuubiBeeLine
# 🔍 Description
## Issue References 🔗

This pull request fixes #5822

## Describe Your Solution 🔧

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

```
$ $KYUUBI_HOME/bin/beeline --help
...
Usage: java " + KyuubiBeeLine.class.getCanonicalName()
...
```

#### Behavior With This Pull Request 🎉

```
$ $KYUUBI_HOME/bin/beeline --help
...
Usage: java org.apache.hive.beeline.KyuubiBeeLine
...
```

#### Related Unit Tests

---

# Checklists
## 📝 Author Self Checklist

- [x] My code follows the [style guidelines](https://kyuubi.readthedocs.io/en/master/contributing/code/style.html) of this project
- [x] I have performed a self-review
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

## 📝 Committer Pre-Merge Checklist

- [x] Pull request title is okay.
- [x] No license issues.
- [x] Milestone correctly set?
- [ ] Test coverage is ok
- [x] Assignees are selected.
- [x] Minimum number of approvals
- [x] No changes are requested

**Be nice. Be informative.**

Closes #5823 from hadoopkandy/KYUUBI-5822.

Closes #5822

a46b6e468 [kandy01.wang] [KYUUBI #5822][doc] Fix the usage msg of KyuubiBeeLine

Authored-by: kandy01.wang <kandy01.wang@vipshop.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2023-12-07 10:01:42 +08:00
Cheng Pan
6061a05f24
Bump 1.9.0-SNAPSHOT 2023-09-04 14:23:12 +08:00
Cheng Pan
f41a8e5b3d
[KYUUBI #5129] KyuubiBeeline should redirect JDK logging
### _Why are the changes needed?_

Otherwise we can not see JDK logs like Krb5.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request

Closes #5129 from pan3793/beeline-log.

Closes #5129

100094823 [Cheng Pan] KyuubiBeeline should redirect JDK logging

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2023-08-03 19:27:31 +08:00
liangbowen
ab3c333122
[KYUUBI #5117] [Minor] Code improvements for Java and Scala code
### _Why are the changes needed?_

- remove duplicated assignment for the same variable in adjacent lines in `FastHiveDecimalImpl`
- replace redundant `putAll` with collection initialization in `BatchRestApi`
- use `try-with-resources` statement with the reader and avoid declaring two variables in the same line of code in `KyuubiCommands`
- fix `warning: Tag 'return:' is not recognised` compilation warning in `KyuubiGetSqlClassification:L53`

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request

Closes #5117 from bowenliang123/fastsignum.

Closes #5117

595b5747d [liangbowen] simplify
be530fac4 [liangbowen] fix warning: Tag '@return:' is not recognised compilation warning in KyuubiGetSqlClassification:L53
249706905 [liangbowen] use try-with-resources in KyuubiCommands
a54a97fdd [liangbowen] remove redundant addAll call to collection initialization
cc76d5d0f [liangbowen] remove repeated assignment

Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2023-08-01 11:36:18 +08:00
liangbowen
c5238535f6 [KYUUBI #4974] [MINOR] Eliminate rawtypes compilation warning in KyuubiBeeLine
### _Why are the changes needed?_

- Eliminate rawtypes compilation warning in `KyuubiBeeLine.java` by adding `SuppressWarnings` annotation of rule `rawtypes` to BeelineParser implementation
```
Warning:  [Warn] /home/runner/work/kyuubi/kyuubi/kyuubi-hive-beeline/src/main/java/org/apache/hive/beeline/KyuubiBeeLine.java:148:53:  [rawtypes] found raw type: ListIterator
```

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4974 from bowenliang123/beeline-rawtypes.

Closes #4974

994a2d3e0 [liangbowen] Fix rawtypes compilation warning in KyuubiBeeLine

Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2023-06-18 14:18:16 +08:00
liangbowen
de0258be79 [KYUUBI #4969] [TEST] Run JUnit tests on beeline module
### _Why are the changes needed?_

- add `maven-surefire-plugin` to beeline module for running JUnit tests.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4969 from bowenliang123/scalatest-junit.

Closes #4969

995c83c07 [liangbowen] surefire on beeline

Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
2023-06-16 20:22:45 +08:00
fwang12
e59349317f [KYUUBI #4305][Bug] Backport HIVE-15820: comment at the head of beeline -e
### _Why are the changes needed?_

Backport https://github.com/apache/hive/pull/1814

related kyuubi issues
#4305
#4333
#4406

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4972 from turboFei/beeline_head_command.

Closes #4305

04b235fb3 [fwang12] [KYUUBI #4305][Bug] Backport HIVE-15820: comment at the head of beeline -e

Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: fwang12 <fwang12@ebay.com>
2023-06-16 19:44:24 +08:00
fwang12
bfc66042c8 [KYUUBI #4965] [BEELINE] Support --python-mode option and remove comments for non-python mode
### _Why are the changes needed?_

Close #4803

The beeline has regression because of 70590f71ef (diff-993fdbefe9fe3d1c91fcedba99362a8c8d9b94793ec16cbfbc989e750367ea89)

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4965 from turboFei/revert_beeline_python_change.

Closes #4965

856d92391 [fwang12] trim for non python mdoe
f8464606b [fwang12] Revert "[KYUUBI #4619] Fix beeline with -e When there are other SQL statements before the source statement, the source statement cannot be executed normally"
e5e3c31b3 [fwang12] revert trim
bec09c254 [fwang12] migration guide
585da6fc1 [fwang12] fix'
f3fcfe97e [fwang12] save
8cb8cb9d0 [fwang12] save
e1539775a [fwang12] comments
814c970a2 [fwang12] save
b1baa773b [fwang12] save
3337ca8fa [fwang12] options

Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: fwang12 <fwang12@ebay.com>
2023-06-16 17:34:24 +08:00
fwang12
01320c4c2f [KYUUBI #4962] Backport HIVE-19048: Initscript errors are ignored
### _Why are the changes needed?_

FYI:
For hive-3.1.3, if failed to get connection, the return code of `beeline -f` is 0, the correct one should be non-zero.
And it works well for hive-4.0.0-alpha-2.
![image](https://github.com/apache/kyuubi/assets/6757692/35ba2329-6f04-4aed-b40b-3c47534554a5)

And the hive-beeline dependency version for kyuubi is 3.1.3.

This pr backports HIVE-19048: Initscript errors are ignored.

https://issues.apache.org/jira/browse/HIVE-19048
006bf8a1a4

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4962 from turboFei/beeline_exit.

Closes #4962

85f497ad2 [fwang12] add ut
21e22cf6f [fwang12] save
49432c256 [fwang12] save
69f90ae37 [fwang12] save
b1afa9e26 [fwang12] backport

Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2023-06-15 10:09:01 +08:00
liangbowen
5f98539c82 [KYUUBI #4944] [MINOR] Code improvement for Java
### _Why are the changes needed?_

- To satisfied the code scanning suggestion of Java language by CodeQL, with no feature changes
  - Ignored error status of call
    - https://github.com/apache/kyuubi/security/code-scanning/88
  - Inefficient empty string test
      - https://github.com/apache/kyuubi/security/code-scanning/87
  - Inefficient String constructor
      - https://github.com/apache/kyuubi/security/code-scanning/84
  - Missing Override annotation
      - https://github.com/apache/kyuubi/security/code-scanning/78
      - https://github.com/apache/kyuubi/security/code-scanning/79
      - https://github.com/apache/kyuubi/security/code-scanning/80
      - https://github.com/apache/kyuubi/security/code-scanning/81
      - https://github.com/apache/kyuubi/security/code-scanning/82
      - https://github.com/apache/kyuubi/security/code-scanning/83
  - Useless toString on String
      - https://github.com/apache/kyuubi/security/code-scanning/108
  - Use of default toString()
      - https://github.com/apache/kyuubi/security/code-scanning/107
  - Unread local variable
      - https://github.com/apache/kyuubi/security/code-scanning/96
  - Random used only once
      - https://github.com/apache/kyuubi/security/code-scanning/192
      - https://github.com/apache/kyuubi/security/code-scanning/191
  - Missing enum case in switch
      - https://github.com/apache/kyuubi/security/code-scanning/193
- redundant usages of length when calling substring

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4944 from bowenliang123/improve-jdbc.

Closes #4944

b1b4dfa03 [liangbowen] substring
0caefc646 [liangbowen] substring
9dab41b57 [liangbowen] substring
a340df36e [liangbowen] style
94be380e8 [liangbowen] code improvement for java

Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
2023-06-09 20:57:45 +08:00
liangbowen
d0675a35a7 [KYUUBI #4879] Refactor and promote relection utils and cleanup similar reflection methods
### _Why are the changes needed?_

- apply the usage of `ReflectUtils` and `Dyn*` to the modules of engines and plugins (eg. Spark engine, Authz plugin, lineage plugin, beeline)
- remove similar redundant methods for calling reflected methods or getting field values
- unified reflection helper methods with type casting support, as `getField[T]` for getting field values from `getFields`, `invokeAs[T]` for invoking methods in `getMethods`.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4879 from bowenliang123/reflect-use.

Closes #4879

c685fb67d [liangbowen] bug fix for "Cannot bind static field options" when executing "bin/beeline"
fc1fdf1de [liangbowen] import
59c3dd032 [liangbowen] comment
c435c131d [liangbowen] reflect util usage

Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
2023-06-06 18:59:18 +08:00
Tianlin Liao
b0d07f7086
[KYUUBI #3887][FOLLOWUP] Fix kyuubiServerPrincipal logic in KyuubiCommands
### _Why are the changes needed?_

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4764 from lightning-L/kyuubi-3887-followup.

Closes #3887

42a123282 [Tianlin Liao] [KYUUBI #3887][FOLLOWUP] fix kyuubiServerPrincipal logic in KyuubiCommands

Authored-by: Tianlin Liao <tiliao@ebay.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2023-04-25 02:17:38 +08:00
xu.guo
d7c1c94f23 [KYUUBI #4619] Fix beeline with -e When there are other SQL statements before the source statement, the source statement cannot be executed normally
### _Why are the changes needed?_

When using beeline -e, if there is a newline or space between the source statement and the previous sql, the source statement will be misjudged as a sql statement and sent to the server for execution
![image](https://user-images.githubusercontent.com/20243868/227915694-42445ecb-7d0e-4ebe-8e8c-b2a25fe069a1.png)

![image](https://user-images.githubusercontent.com/20243868/227915440-fcc540d8-a265-4050-82de-3ffa25c7abc2.png)

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4619 from thomasg19930417/master.

Closes #4619

a2bfc0f0a [xu.guo] Fix When there are other sql before the source statement, the source statement cannot be executed normally
1ec0b8124 [xu.guo] Fix When there are other sql before the source statement, the source statement cannot be executed normally
9abba3cf5 [thomasgx] Merge branch 'apache:master' into master
f8405eb93 [Cheng Pan] Update kyuubi-hive-beeline/src/main/java/org/apache/hive/beeline/KyuubiCommands.java
3ad46c1d6 [Cheng Pan] Update kyuubi-hive-beeline/src/main/java/org/apache/hive/beeline/KyuubiCommands.java
31fa80771 [xu.guo] Merge branch 'master' of https://github.com/thomasg19930417/kyuubi
41e090d66 [thomasgx] Merge branch 'apache:master' into master
ba82229a8 [xu.guo] Move initializeConsoleReader to KyuubiCommands#connect
e06927b78 [xu.guo] Replace create new  consoleReader instance with  call initializeConsoleReader
86f2e5078 [xu.guo] Fix code style
b0c472229 [xu.guo] Fix Beeline with -i run sql faild
042987aa6 [xu.guo] Fix Beeline with -i run sql faild

Lead-authored-by: xu.guo <xu.guo@brgroup.com>
Co-authored-by: thomasgx <570736711@qq.com>
Co-authored-by: Cheng Pan <pan3793@gmail.com>
Signed-off-by: fwang12 <fwang12@ebay.com>
2023-04-04 09:36:05 +08:00
Fu Chen
acdfa6ce1d [KYUUBI #4581] [TEST] Add KyuubiCommands parse python snippets unit test
### _Why are the changes needed?_

to resolve https://github.com/apache/kyuubi/pull/4333#issuecomment-1442793799

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4581 from cfmcgrady/beeline-parser-python-ut.

Closes #4581

9bffa033b [Fu Chen] add KyuubiCommands parse python snippets unit test

Authored-by: Fu Chen <cfmcgrady@gmail.com>
Signed-off-by: Fu Chen <cfmcgrady@gmail.com>
2023-03-22 23:09:24 +08:00
Alex
06f3884627 [KYUUBI #4575] Fix the empty last line may causes the session to exit directly
### _Why are the changes needed?_
If the last line of the file is a blank line, there may be a cmd whose content is empty but the length is not 0 after split by ";". When running to this cmd, it will return false, causing the session to exit directly.

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
vim test.sql and write :
<img width="527" alt="image" src="https://user-images.githubusercontent.com/43542750/226647817-855d866e-836c-4c60-bc81-0bb192d3d6ff.png">
then start beeline and source this file successfully and session won't exit:
<img width="745" alt="image" src="https://user-images.githubusercontent.com/43542750/226652451-823891f3-2768-4165-8816-c052461d1f9c.png">
<img width="598" alt="image" src="https://user-images.githubusercontent.com/43542750/226650769-25a8becf-4655-41ec-bf91-7ae7e28b7d34.png">

- [x] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4575 from Kiss736921/fix_file_empty_last_line.

Closes #4575

ff3fcb189 [Alex] fix the empty last line may causes the session to exit directly

Authored-by: Alex <zoulimin@kanzhun.com>
Signed-off-by: fwang12 <fwang12@ebay.com>
2023-03-22 19:36:43 +08:00
xu.guo
465e23a0f6
[KYUUBI #4508][BEELINE] Beeline should reset stdin after consuming init SQL file
### _Why are the changes needed?_

To fix #4508 .

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4510 from thomasg19930417/master.

Closes #4508

f8405eb93 [Cheng Pan] Update kyuubi-hive-beeline/src/main/java/org/apache/hive/beeline/KyuubiCommands.java
3ad46c1d6 [Cheng Pan] Update kyuubi-hive-beeline/src/main/java/org/apache/hive/beeline/KyuubiCommands.java
31fa80771 [xu.guo] Merge branch 'master' of https://github.com/thomasg19930417/kyuubi
41e090d66 [thomasgx] Merge branch 'apache:master' into master
ba82229a8 [xu.guo] Move initializeConsoleReader to KyuubiCommands#connect
e06927b78 [xu.guo] Replace create new  consoleReader instance with  call initializeConsoleReader
86f2e5078 [xu.guo] Fix code style
b0c472229 [xu.guo] Fix Beeline with -i run sql faild
042987aa6 [xu.guo] Fix Beeline with -i run sql faild

Lead-authored-by: xu.guo <xu.guo@brgroup.com>
Co-authored-by: Cheng Pan <pan3793@gmail.com>
Co-authored-by: thomasgx <570736711@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2023-03-20 17:13:28 +08:00
Fu Chen
b39caed98f
[KYUUBI #4406] Revert "[KYUUBI #4305][Bug] Backport HIVE-15820: comment at the head …
…of beeline -e"

This reverts commit c489e29697.

### _Why are the changes needed?_

https://github.com/apache/kyuubi/pull/4333#issuecomment-1441438302

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4406 from cfmcgrady/revert-4333.

Closes #4406

Closes #4305

c7fff72a [Fu Chen] Revert "[KYUUBI #4305][Bug] Backport HIVE-15820: comment at the head of beeline -e"

Authored-by: Fu Chen <cfmcgrady@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2023-02-24 13:35:36 +08:00
senmiaoliu
c489e29697 [KYUUBI #4305][Bug] Backport HIVE-15820: comment at the head of beeline -e
### _Why are the changes needed?_

close [#4305](https://github.com/apache/kyuubi/issues/4305)

### manual tests

```sql
bin/beeline -u jdbc:hive2://X:10009 -e "
--asd
select 1 as a
"
```

![image](https://user-images.githubusercontent.com/18713676/218910222-b829d447-e5b7-4d80-842b-2ddd4f47a26d.png)

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [x] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4333 from lsm1/fix/beeline_comment_header.

Closes #4305

f932d4ead [senmiaoliu] reformat
9a071fb40 [senmiaoliu] added multi line ut
425c53631 [senmiaoliu] added ut
d4dc21a61 [senmiaoliu] comment at the head of beeline -e

Authored-by: senmiaoliu <senmiaoliu@trip.com>
Signed-off-by: Fu Chen <cfmcgrady@gmail.com>
2023-02-16 17:05:18 +08:00
liangbowen
bb3e06a035
[KYUUBI #4312] [DOCS] Include **/README.md in markdown style check
### _Why are the changes needed?_

- Include `**/README.md` markdown files in spotless style check

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4312 from bowenliang123/reformat-readme.

Closes #4312

1fda1bdeb [Bowen Liang] Merge branch 'master' into reformat-readme
2ca8b4c81 [liangbowen] merge master
876f52a4c [liangbowen] include `**/README.md` in spotless style check

Lead-authored-by: liangbowen <liangbowen@gf.com.cn>
Co-authored-by: Bowen Liang <bowenliang@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2023-02-14 02:23:32 +08:00
Cheng Pan
4e226ac3cc
Bump 1.8.0-SNAPSHOT 2023-02-10 15:25:49 +08:00
liangbowen
7b408efa33
[KYUUBI #4176] Eliminate unhelpful warning messages in Javac output of Java 11
### _Why are the changes needed?_

As #4166 cleanup unhelpful warning messages in Java 8, this PR does the same to Java 11 deprecation warnings.

- add SuppressWarnings("deprecation") to `HiveDecimal` and `FastHiveDecimalImpl`, eliminating warnings of using the deprecated field of `BigDecimal` since Java 9 in Java 11, e.g. (40 warnings in a build)

```
Warning:  /home/runner/work/kyuubi/kyuubi/kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/common/HiveDecimal.java:[133,50] [deprecation] ROUND_FLOOR in BigDecimal has been deprecated
Warning:  /home/runner/work/kyuubi/kyuubi/kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/common/HiveDecimal.java:[150,52] [deprecation] ROUND_CEILING in BigDecimal has been deprecated
Warning:  /home/runner/work/kyuubi/kyuubi/kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/common/HiveDecimal.java:[165,52] [deprecation] ROUND_HALF_UP in BigDecimal has been deprecated
Warning:  /home/runner/work/kyuubi/kyuubi/kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/common/HiveDecimal.java:[186,54] [deprecation] ROUND_HALF_EVEN in BigDecimal has been deprecated
Warning:  /home/runner/work/kyuubi/kyuubi/kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/common/FastHiveDecimalImpl.java:[2934,20] [deprecation] ROUND_HALF_UP in BigDecimal has been deprecated
Warning:  /home/runner/work/kyuubi/kyuubi/kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/common/FastHiveDecimalImpl.java:[4411,23] [deprecation] ROUND_DOWN in BigDecimal has been deprecated
Warning:  /home/runner/work/kyuubi/kyuubi/kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/common/FastHiveDecimalImpl.java:[4424,23] [deprecation] ROUND_UP in BigDecimal has been deprecated
```

- fix `rawtypes` usages and redundant boxing in `KyuubiCommands`
```
Warning:  /home/runner/work/kyuubi/kyuubi/kyuubi-hive-beeline/src/main/java/org/apache/hive/beeline/KyuubiCommands.java:408:9:  [rawtypes] found raw type: Iterator
Warning:  /home/runner/work/kyuubi/kyuubi/kyuubi-hive-beeline/src/main/java/org/apache/hive/beeline/KyuubiCommands.java:[475,71] [deprecation] Character(char) in Character has been deprecated
```

- add SuppressWarnings("deprecation") to `KyuubiBeeLine.java` for using deprecated `Class#newInstance` of Java 11

```
Warning:  /home/runner/work/kyuubi/kyuubi/kyuubi-hive-beeline/src/main/java/org/apache/hive/beeline/KyuubiBeeLine.java:[59,18] [deprecation] newInstance() in Class has been deprecated
  where T is a type-variable:
    T extends Object declared in class Class
Warning:  /home/runner/work/kyuubi/kyuubi/kyuubi-hive-beeline/src/main/java/org/apache/hive/beeline/KyuubiBeeLine.java:59:18:  [deprecation] newInstance() in Class has been deprecated
Warning:  /home/runner/work/kyuubi/kyuubi/kyuubi-hive-beeline/src/main/java/org/apache/hive/beeline/KyuubiCommands.java:475:71:  [deprecation] Character(char) in Character has been deprecated
```

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4176 from bowenliang123/java11-warnings.

Closes #4176

d0c0640f [liangbowen] style
b26679ea [liangbowen] add @SuppressWarnings("deprecation") to HiveDecimal and FastHiveDecimalImpl and constructor of KyuubiBeeLine, remove unnecessary boxing of char in KyuubiCommands#connect, fix raw types usage in KyuubiCommands#getProperty

Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
2023-01-17 14:46:59 +08:00
Tianlin Liao
8e8d8be8d2
[KYUUBI #4020] remove incubating from kyuubi source code
### _Why are the changes needed?_

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4069 from lightning-L/kyuubi-4020.

Closes #4020

97406ca0 [Tianlin Liao] [KYUUBI #4020] remove incubating from kyuubi source code

Authored-by: Tianlin Liao <tiliao@ebay.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
2023-01-04 09:43:20 +08:00
wForget
a494aa9b87 [KYUUBI #4006] Fix the incorrect execution of the source command
### _Why are the changes needed?_

close #4006

refer 7c343471aa/beeline/src/java/org/apache/hive/beeline/Commands.java (L856)

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

Successfully executed after fix:
![image](https://user-images.githubusercontent.com/17894939/208376659-fc5e5a2f-bec6-416e-a8e5-74082c70f24b.png)

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4007 from wForget/KYUUBI-4006.

Closes #4006

0a9710503 [wForget] Fix comment extra never reads next line
47b30247f [wForget] [KYUUBI-4006] Fix the incorrect execution of the source command

Authored-by: wForget <643348094@qq.com>
Signed-off-by: fwang12 <fwang12@ebay.com>
2022-12-20 20:49:37 +08:00
Tianlin Liao
70c0451f4f
[KYUUBI #3887] Provide kyuubiServerPrincipal as alias for principal in JDBC parameter
### _Why are the changes needed?_

Close #3887

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3933 from lightning-L/kyuubi-3887.

Closes #3887

65566232 [Tianlin Liao] [KYUUBI #3887] Provide kyuubiServerPrincipal as alias for principal in JDBC parameter

Authored-by: Tianlin Liao <tiliao@ebay.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-12-12 12:50:43 +08:00
liangbowen
2ac10f91d5
[KYUUBI #3842] [Improvement] Support maven pom.xml code style check with spotless plugin
### _Why are the changes needed?_

Introduce code style check support for Maven's pom.xml with sortPom in spotless maven plugin.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3843 from bowenliang123/spotless-pom.

Closes #3842

3c654597 [liangbowen] apply to pom.xml
fd1536f7 [liangbowen] set expandEmptyElements to true
e498423f [liangbowen] apply spotless:apply to all pom.xml
e46bcfec [liangbowen] add pom style check support in spotless

Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-11-23 22:08:00 +08:00
Fu Chen
70590f71ef
[KYUUBI #3782][PYSPARK] Initial support PySpark
### _Why are the changes needed?_

Close #3758 #3782

Limitations:
- only support kyuubi beeline

Examples:

![截屏2022-11-04 下午5 16 11](https://user-images.githubusercontent.com/8537877/199936938-f0fc9b7e-3886-461b-8197-bd39970f5a6f.png)

![截屏2022-11-04 下午5 16 32](https://user-images.githubusercontent.com/8537877/199936970-b3c14844-6864-4c67-8428-716d632a14db.png)

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3762 from cfmcgrady/python-support.

Closes #3782

83839a80 [Fu Chen] double check
3e4d6e3f [Fu Chen] multi-line
ec56b3c2 [Fu Chen] address comment
4d204b68 [Fu Chen] fix style
aa6aedfb [Fu Chen] address comment
db786fe3 [Fu Chen] resolve conflict
af0d1d9f [Fu Chen] revert kyuubi-hive-beeline/src/main/java/org/apache/hive/beeline/KyuubiCommands.java
8687a825 [Fu Chen] address comment
8954fed8 [Fu Chen] get conn_info_file from env
2952eb9f [Fu Chen] pythonExec
a919f1ad [Fu Chen] fix ga
47543bf0 [Fu Chen] remove findspark dependency
003bf343 [Fu Chen] [GA] setup python
594e3cdc [Fu Chen] add ut
427e1e96 [Fu Chen] pass SPARK_HOME environment variable.
69dd7dfb [Fu Chen] license
b8e44fd1 [Fu Chen] fix style
df33efcd [Fu Chen] PySpark support

Authored-by: Fu Chen <cfmcgrady@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-11-08 20:08:42 +08:00
Cheng Pan
714e591052
[KYUUBI #3775] Paving for further PySpark support in Beeline
### _Why are the changes needed?_

This is a simple method copy and reformat from the superclass.

https://github.com/apache/incubator-kyuubi/pull/3762/files#r1014863236

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #3775 from pan3793/paving-pyspark.

Closes #3775

1d1f9a86 [Cheng Pan] Paving for further PySpark support in Beeline

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
2022-11-07 21:30:19 +08:00