### _Why are the changes needed?_
[SPARK-41448](https://issues.apache.org/jira/browse/SPARK-41448) make consistent MR job IDs in FileBatchWriter and FileFormatWriter in Apache Spark 3.3.2, but it breaks a serializable issue, JobId is non-serializable.
And this pr aims to rewrite `FileWriterFactory` to circumvent the problem
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4359 from Yikf/FileWriterFactory.
Closes#4359
dd8c90fe [Cheng Pan] Update extensions/spark/kyuubi-spark-connector-hive/src/main/scala/org/apache/kyuubi/spark/connector/hive/write/FileWriterFactory.scala
1e5164ec [Yikf] Make a serializable jobTrackerId instead of a non-serializable JobID in FileWriterFactory
Lead-authored-by: Yikf <yikaifei@apache.org>
Co-authored-by: Cheng Pan <pan3793@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Add metrics for user opened connections with session type
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4364 from turboFei/user_session_type.
Closes#4364
d9505f256 [fwang12] get
Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: fwang12 <fwang12@ebay.com>
### _Why are the changes needed?_
Support to refresh the unlimited users for session limiter, so that we can unblock some customers without restart the kyuubi server.
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4360 from turboFei/limier_whitelist.
Closes#4360
c846148bd [fwang12] typo
839e71365 [fwang12] nit
904bc9fbc [fwang12] save
Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: fwang12 <fwang12@ebay.com>
### _Why are the changes needed?_
Fix broken link in `kerberos.rst`
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4361 from Yikf/kerberos-doc.
Closes#4361
26145902 [Yikf] fix break link
Authored-by: Yikf <yikaifei@apache.org>
Signed-off-by: Yikf <yikaifei@apache.org>
### _Why are the changes needed?_
- add config type `userDefaultsConf` (which is introduced in #3982 ) for config refreshing of admin tool
<img width="784" alt="image" src="https://user-images.githubusercontent.com/1935105/219381858-76b5d1dd-2a49-4fd7-bd3a-06178b6871e9.png">
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4346 from bowenliang123/doc-refresh-userconfigs.
Closes#4346
3ada9db4 [liangbowen] add config type `userDefaultsConf` for `kyuubi-admin refresh config`
cc3df715 [liangbowen] add config type `userDefaultsConf` for `kyuubi-admin refresh config`
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
This PR proposes to fix#4282, since log4j2 supports async mode, we need to make sure the `Log4j2DivertAppender#append` is thread-safe.
This PR also changes `OperationLog.getCurrentOperationLog` from `OperationLog` to `Option[OperationLog]`
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4300 from pan3793/log.
Closes#4282
010e34b0 [Cheng Pan] fix
068405b2 [Cheng Pan] fix compile
c79dedd5 [Cheng Pan] Use write lock instead
3daf8a4d [Cheng Pan] nit
94176a04 [Cheng Pan] [KYUUBI #4282] Fix ConcurrentModificationException when log4j2 async enabled
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- 1.6.8 release note: https://github.com/maven-download-plugin/maven-download-plugin/releases/tag/1.6.8
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4347 from bowenliang123/maven-download-plugin-1.6.8.
Closes#4347
947767d1 [liangbowen] bump maven download plugin version from 1.6.6 to 1.6.8
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
It can help to know the backend pressure if the exec pool is full.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4344 from turboFei/wait_queue.
Closes#4344
161e3808a [fwang12] nit
6d122e238 [fwang12] save
55a4b499d [fwang12] version
668ff8bfe [fwang12] save
9f56b98a8 [fwang12] save
a401771ec [fwang12] wait
Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: fwang12 <fwang12@ebay.com>
### _Why are the changes needed?_
according to `io.trino.jdbc.ColumnInfo`, there are some type requring signature parameter.
- varchar(n)
- char(n)
- decimal(precision, scale)
It failed with trino jdbc now
<img width="613" alt="image" src="https://user-images.githubusercontent.com/12025282/218707052-a2e9dc91-0333-483c-bc0a-96baec213578.png">
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4328 from ulysses-you/fix-signature.
Closes#4328
aede5cec [ulysses-you] nit
ae1a7968 [ulysses-you] fix test
8ecdb346 [ulysses-you] Make Trino jdbc driver work
Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: ulyssesyou <ulyssesyou@apache.org>
### _Why are the changes needed?_
Bump up markdown version from 3.3.7 to 3.4.1, the sphinx-markdown-tables has been upgraded to 0.0.17 in which the regression is resolved. It now conflicts with markdown 3.3.7
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4340 from yaooqinn/markdown.
Closes#4340
da950a74 [Kent Yao] Bump up markdown version form 3.3.7 to 3.4.1
Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Some DBMS tools like DBeaver and HUE will call thrift meta api for listing catalogs, databases, and tables. The current implementation of `CatalogShim_v3_0#getSchemas` will call `listAllNamespaces` first and do schema pruning on the Spark driver, which may cause "permission denied" exception when HMS has permission control, like the ranger plugin.
This PR proposes to call HMS API(through v1 session catalog) directly for `spark_catalog`, to suppress the above issue.
```
2023-02-15 20:02:13.048 ERROR org.apache.kyuubi.server.KyuubiTBinaryFrontendService: Error getting schemas:
org.apache.kyuubi.KyuubiSQLException: Error operating GetSchemas: org.apache.spark.sql.AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Permission denied: user [user1] does not have [SELECT] privilege on [userdb1])
at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:134)
at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:249)
at org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener.databaseExists(ExternalCatalogWithListener.scala:69)
at org.apache.spark.sql.catalyst.catalog.SessionCatalog.databaseExists(SessionCatalog.scala:294)
at org.apache.spark.sql.execution.datasources.v2.V2SessionCatalog.listNamespaces(V2SessionCatalog.scala:212)
at org.apache.kyuubi.engine.spark.shim.CatalogShim_v3_0.$anonfun$listAllNamespaces$1(CatalogShim_v3_0.scala:74)
at org.apache.kyuubi.engine.spark.shim.CatalogShim_v3_0.$anonfun$listAllNamespaces$1$adapted(CatalogShim_v3_0.scala:73)
at scala.collection.TraversableLike.$anonfun$flatMap$1(TraversableLike.scala:293)
at scala.collection.IndexedSeqOptimized.foreach(IndexedSeqOptimized.scala:36)
at scala.collection.IndexedSeqOptimized.foreach$(IndexedSeqOptimized.scala:33)
at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:198)
at scala.collection.TraversableLike.flatMap(TraversableLike.scala:293)
at scala.collection.TraversableLike.flatMap$(TraversableLike.scala:290)
at scala.collection.mutable.ArrayOps$ofRef.flatMap(ArrayOps.scala:198)
at org.apache.kyuubi.engine.spark.shim.CatalogShim_v3_0.listAllNamespaces(CatalogShim_v3_0.scala:73)
at org.apache.kyuubi.engine.spark.shim.CatalogShim_v3_0.listAllNamespaces(CatalogShim_v3_0.scala:90)
at org.apache.kyuubi.engine.spark.shim.CatalogShim_v3_0.getSchemasWithPattern(CatalogShim_v3_0.scala:118)
at org.apache.kyuubi.engine.spark.shim.CatalogShim_v3_0.getSchemas(CatalogShim_v3_0.scala:133)
at org.apache.kyuubi.engine.spark.operation.GetSchemas.runInternal(GetSchemas.scala:43)
at org.apache.kyuubi.operation.AbstractOperation.run(AbstractOperation.scala:164)
at org.apache.kyuubi.session.AbstractSession.runOperation(AbstractSession.scala:99)
at org.apache.kyuubi.engine.spark.session.SparkSessionImpl.runOperation(SparkSessionImpl.scala:78)
at org.apache.kyuubi.session.AbstractSession.getSchemas(AbstractSession.scala:150)
at org.apache.kyuubi.service.AbstractBackendService.getSchemas(AbstractBackendService.scala:83)
at org.apache.kyuubi.service.TFrontendService.GetSchemas(TFrontendService.scala:294)
at org.apache.kyuubi.shade.org.apache.hive.service.rpc.thrift.TCLIService$Processor$GetSchemas.getResult(TCLIService.java:1617)
at org.apache.kyuubi.shade.org.apache.hive.service.rpc.thrift.TCLIService$Processor$GetSchemas.getResult(TCLIService.java:1602)
at org.apache.kyuubi.shade.org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.kyuubi.shade.org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at org.apache.kyuubi.service.authentication.TSetIpAddressProcessor.process(TSetIpAddressProcessor.scala:36)
at org.apache.kyuubi.shade.org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4336 from pan3793/list-schemas.
Closes#4336
9ece864c [Cheng Pan] fix
f71587e9 [Cheng Pan] Avoid listing all schemas for Spark session catalog on schema prunning
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Supply a FTS connector doc in Hive
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4337 from a49a/hive-fts-doc.
Closes#3085
7ec5b73f [Luning Wang] [KYUUBI #3085][DOCS] Add Hive FTS Connector
Authored-by: Luning Wang <wang4luning@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
The wrong parsing of jvm parameters in jdbc url.
For example:
```shell
jdbc:kyuubi://127.0.0.1:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=kyuubi;user=tom?spark.driver.memory=8G;spark.driver.extraJavaOptions=-Xss256m -XX:+PrintGCDetails -XX:HeapDumpPath=/heap.hprof
```
Now because the regex `([^;]*)=([^;]*);?` only matches `=`, resulting in wrong parsing:
```shell
<spark.driver.memory, 8G>
<spark.driver.extraJavaOptions=-Xss256m -XX:+PrintGCDetails -XX:HeapDumpPath, /heap.hprof>
```
The correct parsing should be:
```shell
<spark.driver.memory, 8G>
<spark.driver.extraJavaOptions, -Xss256m -XX:+PrintGCDetails -XX:HeapDumpPath=/heap.hprof>
```
This PR change `org.apache.kyuubi.jdbc.hive.Utils` and added unit test in `org.apache.kyuubi.jdbc.hive.UtilsTest`.
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4311 from imperio-wxm/master.
Closes#4311
d9e124ca [liangbowen] fix percent encoding problem
31ae63e0 [liangbowen] change `com.sun.jndi.toolkit.url.UrlUtil` to `java.net.URLEncoder`
f8271ec2 [wxm] Fix the wrong parsing of jvm parameters in jdbc url.
2e93df57 [wxmimperio] Merge branch 'apache:master' into master
4c236447 [wxmimperio] Merge branch 'apache:master' into master
3e0b7764 [wxmimperio] Scala repl output log level adjusted to debug
Lead-authored-by: wxmimperio <wxmimperio@outlook.com>
Co-authored-by: liangbowen <liangbowen@gf.com.cn>
Co-authored-by: wxm <wxmimperio@outlook.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
[NoHttpResponseException](https://hc.apache.org/httpclient-legacy/exception-handling.html)
> In some circumstances, usually when under heavy load, the web server may be able to receive requests but unable to process them. A lack of sufficient resources like worker threads is a good example. This may cause the server to drop the connection to the client without giving any response. HttpClient throws NoHttpResponseException when it encounters such a condition. In most cases it is safe to retry a method that failed with NoHttpResponseException.
In case of Kyuubi Server is overloaded and does not have enough resources to allocate threads to handle the corresponding request, the request will be dropped and no response will be returned, Kyuubi Rest Client should catch this exception and retry.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4334 from zwangsheng/rest_client/retry_with_no_http_response.
Closes#4334
d526e4ab [zwangsheng] Remove Unrelated Code
708f2496 [zwangsheng] [REST] Retry should catch NoHttpResponse Exception and retry
Authored-by: zwangsheng <2213335496@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
This PR proposes to use `org.apache.spark.sql.execution#toHiveString` to replace `org.apache.kyuubi.engine.spark.schema#toHiveString` to get consistent result w/ `spark-sql` and `STS`.
Because of [SPARK-32006](https://issues.apache.org/jira/browse/SPARK-32006), it only works w/ Spark 3.1 and above.
The patch takes effects on both thrift and arrow result format.
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
```
➜ ~ beeline -u 'jdbc:hive2://0.0.0.0:10009/default'
Connecting to jdbc:hive2://0.0.0.0:10009/default
Connected to: Spark SQL (version 3.3.1)
Driver: Hive JDBC (version 2.3.9)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 2.3.9 by Apache Hive
0: jdbc:hive2://0.0.0.0:10009/default> select to_timestamp('2023-02-08 22:17:33.123456789');
+----------------------------------------------+
| to_timestamp(2023-02-08 22:17:33.123456789) |
+----------------------------------------------+
| 2023-02-08 22:17:33.123456 |
+----------------------------------------------+
1 row selected (0.415 seconds)
```
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4318 from pan3793/hive-string.
Closes#4316
ba9016f6 [Cheng Pan] nit
8be774b4 [Cheng Pan] nit
bd696fe3 [Cheng Pan] nit
b5cf051c [Cheng Pan] fix
dd6b7021 [Cheng Pan] test
63edd34d [Cheng Pan] nit
37cc70af [Cheng Pan] Fix python ut
c66ad22d [Cheng Pan] [KYUUBI #4316] Fix returned Timestamp values may lose precision
41d94445 [Cheng Pan] Revert "[KYUUBI #3958] Fix Spark session timezone format"
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
This pr improves the trino session context:
1. always reuse the kyuubi session if session id exists, so we can restore the session context for next query
2. transform trino client information to kyuubi session, e.g. trino request source (trino-cli)
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4323 from ulysses-you/trino-session.
Closes#4323
59804bad [ulysses-you] style
fcf540a6 [ulysses-you] Improve trino session context
Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: ulyssesyou <ulyssesyou@apache.org>
### _Why are the changes needed?_
Add Publish docker image step. The step details is maintained by docker repo.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4324 from ulysses-you/release.
Closes#4324
0f6a6646 [ulysses-you] Add Publish docker image step
Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: ulyssesyou <ulyssesyou@apache.org>
### _Why are the changes needed?_
Support GetPrimaryKeys for Trino Fe, close https://github.com/apache/kyuubi/issues/4320
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4321 from Yikf/primaryKeys.
Closes#4320
3690a2c7 [Yikf] Support GetPrimaryKeys for Trino Fe
Authored-by: Yikf <yikaifei@apache.org>
Signed-off-by: ulyssesyou <ulyssesyou@apache.org>
### _Why are the changes needed?_
The mysql url configuration joiner should use `&` instead of `;`
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4322 from Yikf/mysql-fe-test.
Closes#4322
8d289e793 [Yikf] The configuration joiner should use `&` instead of `;`
Authored-by: Yikf <yikaifei@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- fix the README files' over-matched pattern mentioned in https://github.com/apache/kyuubi/pull/4312/files#r1105223855
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4319 from bowenliang123/readme-match-pattern.
Closes#4312
3a8266e6 [liangbowen] update
189695be [liangbowen] fix overmatching pattern for README file
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
- Include `**/README.md` markdown files in spotless style check
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4312 from bowenliang123/reformat-readme.
Closes#4312
1fda1bdeb [Bowen Liang] Merge branch 'master' into reformat-readme
2ca8b4c81 [liangbowen] merge master
876f52a4c [liangbowen] include `**/README.md` in spotless style check
Lead-authored-by: liangbowen <liangbowen@gf.com.cn>
Co-authored-by: Bowen Liang <bowenliang@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- Check whether modules are available by running `mvnd dependency:resolve`. if true, skip the `build` step of `mvnd install`.
- Reducing the time cost of dependency workflow from ~ 10 min to ~4 min ([see log](https://github.com/apache/kyuubi/actions/runs/4162071562/jobs/7200786535)).
### _How was this patch tested?_
- [x] Pass CI workflows
Closes#4315 from bowenliang123/dep-check-first.
Closes#4315
7c20ea24 [liangbowen] nit
65397868 [liangbowen] add validate to bring back enforcer check
1c035136 [liangbowen] add validate to bring back enforcer check
d4e703de [liangbowen] update dep.yml
8f8d0457 [liangbowen] update dep.yml
e7adfacf [liangbowen] update
c88eff98 [liangbowen] check modules available before build
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
```
...
07:43:26.520 KyuubiSessionManager-exec-pool: Thread-779 INFO ProcBuilder: Logging to /home/runner/work/kyuubi/kyuubi/kyuubi-server/target/work/runner/kyuubi-spark-batch-submit.log.0
...
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4317 from pan3793/ci-log.
Closes#4317
dbde5306 [Cheng Pan] Upload kyuubi-spark-batch-submit.log on CI failure
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: ulyssesyou <ulyssesyou@apache.org>
### _Why are the changes needed?_
Close#3935
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4232 from iodone/kyuubi-3935.
Closes#3935
936ea1f8 [odone] address
e7bd01a1 [odone] support trino client connect kyuubi trino server
9ea8b6af [odone] [WIP] trion request/response implementation
Authored-by: odone <odone.zhang@gmail.com>
Signed-off-by: ulyssesyou <ulyssesyou@apache.org>
### _Why are the changes needed?_
The changes are needed to make doc page clearer and a bit nicer, and reflect latest changes in the chart.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
This patch had conflicts when merged, resolved by
Committer: Cheng Pan <chengpan@apache.org>
Closes#4314 from dnskr/update_helm_chart_doc_page.
Closes#4314
d7c18fd53 [dnskr] [DOCS][HELM] Refine helm chart docs
Authored-by: dnskr <dnskrv88@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
The changes are needed to hide license text printed after chart installed.
Before changes:
```
$ helm install kyuubi ${KYUUBI_HOME}/charts/kyuubi -n kyuubi --create-namespace
NAME: kyuubi
LAST DEPLOYED: Sat Feb 11 20:35:52 2023
NAMESPACE: kyuubi
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
The chart has been installed!
In order to check the release status, use:
helm status kyuubi -n kyuubi
or for more detailed info
helm get all kyuubi -n kyuubi
************************
******* Services *******
************************
THRIFT_BINARY:
- To access kyuubi-thrift-binary service within the cluster, use the following URL:
kyuubi-thrift-binary.kyuubi.svc.cluster.local
- To access kyuubi-thrift-binary service from outside the cluster for debugging, run the following command:
kubectl port-forward svc/kyuubi-thrift-binary 10009:10009 -n kyuubi
and use 127.0.0.1:10009
```
After changes:
```
$ helm install kyuubi ${KYUUBI_HOME}/charts/kyuubi -n kyuubi --create-namespace
NAME: kyuubi
LAST DEPLOYED: Sat Feb 11 20:37:45 2023
NAMESPACE: kyuubi
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
The chart has been installed!
In order to check the release status, use:
helm status kyuubi -n kyuubi
or for more detailed info
helm get all kyuubi -n kyuubi
************************
******* Services *******
************************
THRIFT_BINARY:
- To access kyuubi-thrift-binary service within the cluster, use the following URL:
kyuubi-thrift-binary.kyuubi.svc.cluster.local
- To access kyuubi-thrift-binary service from outside the cluster for debugging, run the following command:
kubectl port-forward svc/kyuubi-thrift-binary 10009:10009 -n kyuubi
and use 127.0.0.1:10009
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4313 from dnskr/comment_license_in_helm_chart_notes.
Closes#4313
7cc76639b [dnskr] [K8S][HELM] Comment license in helm chart NOTES
Authored-by: dnskr <dnskrv88@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
bump Mysql jdbc driver to 8.0.32
release: https://github.com/mysql/mysql-connector-j/releases/tag/8.0.32
Mysql jdbc driver used in test scope only
nothing new after running build/dependency.sh --replace
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4309 from runzhliu/patch-7.
Closes#4299
a9e43e91 [runzhliu] Merge branch 'apache:master' into patch-7
53ce3a37 [runzhliu] Merge branch 'apache:master' into patch-7
d3f02cd6 [runzhliu] Bump Mysql jdbc driver from 8.0.31 to 8.0.32
Authored-by: runzhliu <runzhliu@163.com>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
bump JUnit4 from 4.13.1 to 4.13.2
release note: https://github.com/junit-team/junit4/blob/HEAD/doc/ReleaseNotes4.13.2.md
nothing new after running `build/dependency.sh --replace`
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4310 from runzhliu/patch-6.
Closes#4299
8857d254 [runzhliu] Merge branch 'apache:master' into patch-6
c2893632 [runzhliu] Bump Junit4 from 4.13.1 to 4.13.2
Authored-by: runzhliu <runzhliu@163.com>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
The message is not noisy and is helpful when failing to create, e.g. no permission.
```
2023-02-10 14:30:32.731 INFO org.apache.kyuubi.operation.LaunchEngine: Processing simba's query[d51a4def-d3d7-454b-a63d-28e27a1e3c3f]: RUNNING_STATE -> ERROR_STATE, time taken: 0.1 seconds
Error: org.apache.kyuubi.KyuubiSQLException: Error operating LaunchEngine: java.io.IOException: 权限不够
at java.io.UnixFileSystem.createFileExclusively(Native Method)
at java.io.File.createNewFile(File.java:1012)
at org.apache.kyuubi.engine.ProcBuilder.$anonfun$engineLog$6(ProcBuilder.scala:187)
at scala.Option.getOrElse(Option.scala:189)
at org.apache.kyuubi.engine.ProcBuilder.engineLog(ProcBuilder.scala:184)
at org.apache.kyuubi.engine.ProcBuilder.engineLog$(ProcBuilder.scala:159)
at org.apache.kyuubi.engine.spark.SparkProcessBuilder.engineLog$lzycompute(SparkProcessBuilder.scala:36)
at org.apache.kyuubi.engine.spark.SparkProcessBuilder.engineLog(SparkProcessBuilder.scala:36)
at org.apache.kyuubi.engine.ProcBuilder.processBuilder(ProcBuilder.scala:141)
at org.apache.kyuubi.engine.ProcBuilder.processBuilder$(ProcBuilder.scala:135)
at org.apache.kyuubi.engine.spark.SparkProcessBuilder.processBuilder$lzycompute(SparkProcessBuilder.scala:36)
at org.apache.kyuubi.engine.spark.SparkProcessBuilder.processBuilder(SparkProcessBuilder.scala:36)
at org.apache.kyuubi.engine.ProcBuilder.start(ProcBuilder.scala:198)
at org.apache.kyuubi.engine.ProcBuilder.start$(ProcBuilder.scala:197)
at org.apache.kyuubi.engine.spark.SparkProcessBuilder.start(SparkProcessBuilder.scala:36)
at org.apache.kyuubi.engine.EngineRef.$anonfun$create$1(EngineRef.scala:194)
at org.apache.kyuubi.ha.client.zookeeper.ZookeeperDiscoveryClient.tryWithLock(ZookeeperDiscoveryClient.scala:174)
at org.apache.kyuubi.engine.EngineRef.tryWithLock(EngineRef.scala:160)
at org.apache.kyuubi.engine.EngineRef.create(EngineRef.scala:165)
at org.apache.kyuubi.engine.EngineRef.$anonfun$getOrCreate$1(EngineRef.scala:260)
at scala.Option.getOrElse(Option.scala:189)
at org.apache.kyuubi.engine.EngineRef.getOrCreate(EngineRef.scala:260)
at org.apache.kyuubi.session.KyuubiSessionImpl.$anonfun$openEngineSession$1(KyuubiSessionImpl.scala:120)
at org.apache.kyuubi.session.KyuubiSessionImpl.$anonfun$openEngineSession$1$adapted(KyuubiSessionImpl.scala:113)
at org.apache.kyuubi.ha.client.DiscoveryClientProvider$.withDiscoveryClient(DiscoveryClientProvider.scala:36)
at org.apache.kyuubi.session.KyuubiSessionImpl.openEngineSession(KyuubiSessionImpl.scala:113)
at org.apache.kyuubi.operation.LaunchEngine.$anonfun$runInternal$2(LaunchEngine.scala:49)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at org.apache.kyuubi.KyuubiSQLException$.apply(KyuubiSQLException.scala:69)
at org.apache.kyuubi.operation.KyuubiOperation$$anonfun$onError$1.applyOrElse(KyuubiOperation.scala:75)
at org.apache.kyuubi.operation.KyuubiOperation$$anonfun$onError$1.applyOrElse(KyuubiOperation.scala:56)
at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:38)
at org.apache.kyuubi.operation.LaunchEngine.$anonfun$runInternal$2(LaunchEngine.scala:51)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: 权限不够
at java.io.UnixFileSystem.createFileExclusively(Native Method)
at java.io.File.createNewFile(File.java:1012)
at org.apache.kyuubi.engine.ProcBuilder.$anonfun$engineLog$6(ProcBuilder.scala:187)
at scala.Option.getOrElse(Option.scala:189)
at org.apache.kyuubi.engine.ProcBuilder.engineLog(ProcBuilder.scala:184)
at org.apache.kyuubi.engine.ProcBuilder.engineLog$(ProcBuilder.scala:159)
at org.apache.kyuubi.engine.spark.SparkProcessBuilder.engineLog$lzycompute(SparkProcessBuilder.scala:36)
at org.apache.kyuubi.engine.spark.SparkProcessBuilder.engineLog(SparkProcessBuilder.scala:36)
at org.apache.kyuubi.engine.ProcBuilder.processBuilder(ProcBuilder.scala:141)
at org.apache.kyuubi.engine.ProcBuilder.processBuilder$(ProcBuilder.scala:135)
at org.apache.kyuubi.engine.spark.SparkProcessBuilder.processBuilder$lzycompute(SparkProcessBuilder.scala:36)
at org.apache.kyuubi.engine.spark.SparkProcessBuilder.processBuilder(SparkProcessBuilder.scala:36)
at org.apache.kyuubi.engine.ProcBuilder.start(ProcBuilder.scala:198)
at org.apache.kyuubi.engine.ProcBuilder.start$(ProcBuilder.scala:197)
at org.apache.kyuubi.engine.spark.SparkProcessBuilder.start(SparkProcessBuilder.scala:36)
at org.apache.kyuubi.engine.EngineRef.$anonfun$create$1(EngineRef.scala:194)
at org.apache.kyuubi.ha.client.zookeeper.ZookeeperDiscoveryClient.tryWithLock(ZookeeperDiscoveryClient.scala:174)
at org.apache.kyuubi.engine.EngineRef.tryWithLock(EngineRef.scala:160)
at org.apache.kyuubi.engine.EngineRef.create(EngineRef.scala:165)
at org.apache.kyuubi.engine.EngineRef.$anonfun$getOrCreate$1(EngineRef.scala:260)
at scala.Option.getOrElse(Option.scala:189)
at org.apache.kyuubi.engine.EngineRef.getOrCreate(EngineRef.scala:260)
at org.apache.kyuubi.session.KyuubiSessionImpl.$anonfun$openEngineSession$1(KyuubiSessionImpl.scala:120)
at org.apache.kyuubi.session.KyuubiSessionImpl.$anonfun$openEngineSession$1$adapted(KyuubiSessionImpl.scala:113)
at org.apache.kyuubi.ha.client.DiscoveryClientProvider$.withDiscoveryClient(DiscoveryClientProvider.scala:36)
at org.apache.kyuubi.session.KyuubiSessionImpl.openEngineSession(KyuubiSessionImpl.scala:113)
at org.apache.kyuubi.operation.LaunchEngine.$anonfun$runInternal$2(LaunchEngine.scala:49)
... 5 more (state=,code=0)
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4302 from pan3793/info.
Closes#4302
24c8ab39d [Cheng Pan] Change log level from debug to info for creating work dir
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
See more in #4184
This subtask remove build kyuubi docker image from dev mode(source code).
In follow subtask, will add feature build kyuubi docker image from source code.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4217 from zwangsheng/tool/docker-image-tool.
Closes#4184
34c866025 [zwangsheng] Add +x premission for docker-image-tool
97b404db7 [zwangsheng] Copy download spark binary
6f267e221 [zwangsheng] Refactor docker-image-tool.sh
Authored-by: zwangsheng <2213335496@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
https://github.com/apache/spark/pull/39733
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4296 from cfmcgrady/spark-3.5.0-SNAPSHOT.
Closes#4296
e16e3f1b0 [Fu Chen] bump spark from 3.4.0-SNAPSHOT to 3.5.0-SNAPSHOT
Authored-by: Fu Chen <cfmcgrady@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
This PR proposes to change the paths of distributed lock and seq_num(used for POLLING engine pool select policy) on the Service Discovery component. The reason is that namespace `${serverSpace}_${KYUUBI_VERSION}_${shareLevel}_${engineType}/` should be dedicated to engine registration, we'd better use the separated namespace for other functionalities.
- lock path
```
# before
${serverSpace}_${shareLevel}_${engineType}/lock/${user}/${subdomain}
# after
${serverSpace}_${KYUUBI_VERSION}_${shareLevel}_${engineType}_lock/${user}/${subdomain}
```
- seq_num
```
# before
${serverSpace}_${KYUUBI_VERSION}_${shareLevel}_${engineType}/seq_num/${user}/${poolName}
# after
${serverSpace}_${KYUUBI_VERSION}_${shareLevel}_${engineType}_seqNum/${user}/${poolName}
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4145 from pan3793/namespace.
Closes#4145
c912b3f66 [Cheng Pan] name
3326b9b95 [Cheng Pan] name
10083db0b [Cheng Pan] Change lock and polloing seq_num path on service discovery
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
eclipse-temurin is the successor for openjdk, see https://github.com/apache/spark/pull/37705
"The core change is: the OS of base image changes debian-bullseye to ubuntu-focal (based on debian bullseye)."
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4288 from pan3793/image.
Closes#4288
b7b619018 [Cheng Pan] Use eclipse-temurin:8-jdk-focal as default base image
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
This PR proposes to change the publish `master-snapshot` image from every commit of the master branch to the daily scheduled, it saves GA resources.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4297 from pan3793/docker-nightly.
Closes#4297
1424287cb [Cheng Pan] nit
32cffe8ab [Cheng Pan] Daily publish master snapshot docker image
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- Jackson 2.14.2 release note: https://github.com/FasterXML/jackson/wiki/Jackson-Release-2.14.2
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4286 from runzhliu/patch-4.
Closes#4286
6902e2ec [runzhliu] Update dependencyList
3cde2004 [runzhliu] Merge branch 'apache:master' into patch-4
8ac6eccd [runzhliu] bump the jackson maven dependecy version to 2.14.2
Authored-by: runzhliu <runzhliu@163.com>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
This PR updates `pom.xml` and CI workflow to change the default Flink version to 1.16, close#4253
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4289 from pan3793/flink-1.16.
Closes#4253
dc3885a38 [Cheng Pan] [KYUUBI #4253] Change default Flink version to 1.16
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Pin `org.threeten:threeten-extra:1.7.0` for spark master as a workaround for https://github.com/ThreeTen/threeten-extra/issues/226
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4292 from pan3793/threeten.
Closes#4292
8c1a24fd5 [Cheng Pan] Pin threeten 1.7.0
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
This PR proposes to fix the flaky test
```
- post session exception if failed to open engine session *** FAILED ***
```
which was introduced in #4185https://github.com/apache/kyuubi/actions/runs/4131523933/jobs/7139249142
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4283 from pan3793/flaky.
Closes#4283
f5c70077 [Cheng Pan] [TEST] Fix flaky test in SessionsResourceSuite
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Add a GA Job For Documentation Verification
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4269 from yaooqinn/docga.
Closes#4269
c9b39de6 [Cheng Pan] Update .github/workflows/docs.yml
db2d37f8 [Kent Yao] Update .github/workflows/docs.yml
c5bc8518 [Kent Yao] trigger ga
6340129b [Kent Yao] trigger ga
fafbf847 [Kent Yao] trigger ga
52f23240 [Kent Yao] trigger ga
1e0000a7 [Kent Yao] Add a GA Job For Documentaion Verification
Lead-authored-by: Kent Yao <yao@apache.org>
Co-authored-by: Cheng Pan <pan3793@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
bump the minikube version to 1.29.0 and the kubernetes version to 1.26.1.
After ran some tests for the used action manusa/actions-setup-minikubev2.7.2, it turns out that it would have some problems when I bump the minikube version to 1.29.0, please refer to [this link](https://github.com/runzhliu/yum-with-browser/actions/runs/4113082204/jobs/7098775474) for details.
In addition, I have tried another minikube action medyagh/setup-minikubemaster, it's good to bump the version for both minikube and Kubernetes, please refer to [this link](https://github.com/runzhliu/yum-with-browser/actions/runs/4112397435/jobs/7097254604#step:5:8).
However, I would prefer to [this way](https://github.com/apache/spark/blob/master/.github/workflows/build_and_test.yml#L945-L966) to setup a minikube cluster for the rest Kubernetes related tests, it's also good to set up the cluster as well, please refer to [this link](https://github.com/runzhliu/yum-with-browser/actions/runs/4113018527/jobs/7098637193#step:4:2).
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4248 from runzhliu/patch-1.
Closes#4248
7d12e0e8 [runzhliu] Update master.yml
52448ae0 [runzhliu] Update master.yml
5fb0b9ec [runzhliu] Update master.yml
3491b33a [runzhliu] Merge branch 'apache:master' into patch-1
64101d95 [runzhliu] bump the minikube version to 1.29.0 and the kubernetes version to 1.26.1
4b278697 [runzhliu] Merge branch 'apache:master' into patch-1
b19e0983 [runzhliu] Update master.yml
f6bc18ee [runzhliu] Update master.yml
bb133312 [runzhliu] Update master.yml
b44ad030 [runzhliu] Update master.yml
7f22810f [runzhliu] Update master.yml
09a6dadf [runzhliu] Update master.yml
60623715 [runzhliu] Update master.yml
0d2c6568 [runzhliu] Update master.yml
5c8b0cc6 [runzhliu] bump the minikube version to 1.29.0 and the kubernetes version to 1.26.1
Authored-by: runzhliu <runzhliu@163.com>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
`build/mvn` should download the maven when the local `mvn` version does not match, this corrects the change in #4261
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4291 from pan3793/mvn-version.
Closes#4247
42717a73c [Cheng Pan] [KYUUBI #4247][FOLLOWUP] Fix build/mvn version comparison
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
to close#4262 .
- change the required privilege of the source table from DROP to ALTER
- skip privilege checks of the new target table, as the same way in Ranger's Hive plugin
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4281 from bowenliang123/4262-renametable.
Closes#4262
e7117de9 [liangbowen] change required privilege of the source table from DROP to ALTER
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
close#4250
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4275 from df-Liu/fix_4250.
Closes#4250
723c4b745 [df_liu] [KYUUBI #4250] fix 1.16.1
0e151a408 [df_liu] [KYUUBI #4250] Bump Flink 1.16.1
Authored-by: df_liu <df_liu@trip.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
updated `.github/workflows/master.yml#L161`
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4284 from df-Liu/follow_4252.
Closes#4252
3b0069418 [df_liu] [KYUUBI #4252][FOLLOWUP] Bump Flink 1.14.6
Authored-by: df_liu <df_liu@trip.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
