### _Why are the changes needed?_
Update codecov token to fix stale code coverage action
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
no
Closes#5501 from yaooqinn/token.
Closes#5501
1f4053231 [Kent Yao] java-17
fcd8391e7 [Kent Yao] java-17
5ccead77d [Kent Yao] upgrade jacoco
5cb4a8374 [Kent Yao] Revert "rm"
45d7a2134 [Kent Yao] rm
0a3dd2f42 [Kent Yao] Update codecov token
Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
Resolve: #5405 Support the Flink 1.18
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No
Closes#5465 from YesOrNo828/flink-1.18.
Closes#5405
a0010ca14 [Xianxun Ye] [KYUUBI #5405] [FLINK] Remove flink1.18 rc repo
2a4ae365c [Xianxun Ye] Update .github/workflows/master.yml
d4d458dc7 [Xianxun Ye] [KYUUBI #5405] [FLINK] Update the flink1.18-rc3 repo
99172e3da [Xianxun Ye] [KYUUBI #5405] [FLINK] Using the staging repo during the RC stage
4c0cf887b [Xianxun Ye] [KYUUBI #5405] [FLINK] Using the staging repo during the RC stage
c74f5c31b [Xianxun Ye] [KYUUBI #5405] [FLINK] fixed Pan's comments.
1933ebadd [Xianxun Ye] [KYUUBI #5405] [FLINK] Support Flink 1.18
Authored-by: Xianxun Ye <yesorno828423@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Improve Kyuubi On Kubernetes IT
Done:
1. Copy spark submit engine log in kyuubi pod to local and upload when test failed.
2. pre install spark image into minikube to avoid image pull error
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No
Closes#5437 from zwangsheng/KYUUBI#5435.
Closes#5435
0cbbafce7 [zwangsheng] add comment
1f1336c59 [zwangsheng] ready
e1c10a6ea [zwangsheng] debug
32759015c [zwangsheng] debug
8e2f1eaf1 [zwangsheng] debug
80eaae30a [zwangsheng] [KYUUBI #5435][NOT_MERGE][TEST] Improve Kyuubi On Kubernetes IT
Authored-by: zwangsheng <binjieyang@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- Check style with profile of Spark 3.4 and 3.5 in the style workflow
- Isolated scalastyle check for Spark 3.1 profile, as Iceberg 1.3.1 does not support Spark 3.5
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No.
Closes#5444 from bowenliang123/spotless-spark34.
Closes#5444
ca98c52e6 [Bowen Liang] update
9550a4a0a [Bowen Liang] enable spark-3.1 profile for spotless style checks
dae35290b [Bowen Liang] revert order
3e488cf48 [Bowen Liang] Check with Spark 3.1 profile separately
e879f09f8 [Bowen Liang] revert the order of spark versions in profiles
16456a7fd [Bowen Liang] check style with profile of Spark 3.4 and 3.5
Authored-by: Bowen Liang <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
### _Why are the changes needed?_
Apache Paimon is an incubating Apache project of data lake platform for high-speed data ingestion, changelog tracking and efficient real-time analytics.
- Initial support for Paimon tables in Authz plugin
- Create Table Command: https://paimon.apache.org/docs/master/engines/spark3/#create-table
- Paimon `0.5.0-incubating` supports Spark 3.1/3.2/3.3/3.4.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
Closes#5462 from bowenliang123/authz-paimon-createtable.
Closes#5462
8393f6ecc [liangbowen] remove spark-authz-paimon-test profile, use 3.4 as spark binary version for Paimon
a4e01a2a8 [liangbowen] update workflow inclusions
f372882aa [liangbowen] update workflow exclusions
7785dec52 [liangbowen] update workflow include
64ee20106 [liangbowen] update
73edea38f [liangbowen] add spark-authz-paimon-test profile and excluded for Spark 3.5
de58605bb [liangbowen] nit
81a4a4370 [liangbowen] nit
eb2de8359 [Bowen Liang] exclude paimon tests for spark-3.5
4f58d7727 [Bowen Liang] fix typo
cf360e6da [Bowen Liang] drop database
af8ffda14 [Bowen Liang] drop database
dc3ee9462 [Bowen Liang] support Paimon create table
Lead-authored-by: Bowen Liang <liangbowen@gf.com.cn>
Co-authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
### _Why are the changes needed?_
To close#5284.
Hudi also is a common used data format, since kyuubi already support iceberg and delta, we should also support hudi.
In this pr we support hoodie sql about ALTER COMMAND in authz
In this PR we use default Hudi version 0.14.0.
We support from spark 3.1 to spark 3.4, since Hudi don't support spark 3.5 yet
- [x] spark 3.1
- [x] spark 3.2
- [x] spark 3.3
- [x] spark 3.4
- [ ] spark 3.5
Also since Hudi only supports Scala 2.12, I also made Hudi as a separate profile to avoid importing Hudi when enable Scala 2.13
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No
Closes#5287 from AngersZhuuuu/KYUUBI-5284.
Closes#5284
f171e11af [Angerszhuuuu] Update pom.xml
3f57a3dc5 [Angerszhuuuu] follow comment
f6c764028 [Angerszhuuuu] follow comment
51797e25c [Angerszhuuuu] trigger
b3c059af9 [Angerszhuuuu] Update HudiCatalogRangerSparkExtensionSuite.scala
3510e7601 [liangbowen] remove scope in dependencyManagement
14ea0d498 [liangbowen] change to use `spark.binary.version` for hudi dependency by default
354260eb0 [liangbowen] remove the abbreviation tite
658bddbab [liangbowen] remove clarification and use ALTERTABLE_PROPERTIES for opType
150edcd40 [Angerszhuuuu] update
30c417b19 [Angerszhuuuu] trigger
56e5cb17b [Angerszhuuuu] Update HudiCatalogRangerSparkExtensionSuite.scala
fe9b75270 [Angerszhuuuu] update
888943831 [Angerszhuuuu] Update HudiCatalogRangerSparkExtensionSuite.scala
db749a277 [Angerszhuuuu] update
9b09e78c2 [Angerszhuuuu] Update HudiCommands.scala
87de62e52 [Angerszhuuuu] follow comment
2d551d112 [Angerszhuuuu] Update master.yml
89082e06b [Angerszhuuuu] Update master.yml
7c7846378 [Angerszhuuuu] Merge branch 'KYUUBI-5284' of https://github.com/AngersZhuuuu/incubator-kyuubi into KYUUBI-5284
d32ca9839 [Angerszhuuuu] Update master.yml
ec43e2a7b [Angerszhuuuu] Merge branch 'master' into KYUUBI-5284
b3611fd3e [Angerszhuuuu] update
2a0dfa74f [Angerszhuuuu] Update AuthZUtils.scala
45ee9e251 [Angerszhuuuu] update
0560a5e14 [Angerszhuuuu] Update pom.xml
97c50f622 [Angerszhuuuu] update
f57ee0093 [Angerszhuuuu] Update table_command_spec.json
fb72197e6 [Angerszhuuuu] update
2154cf928 [Angerszhuuuu] trigger
44469359f [Angerszhuuuu] trigger
b0e768cb8 [Angerszhuuuu] Update HoodieCatalogRangerSparkExtensionSuite.scala
83795ed63 [Angerszhuuuu] Update pom.xml
eed190f92 [Angerszhuuuu] update
361660145 [Angerszhuuuu] update
1ed1f3ab6 [Angerszhuuuu] Update
7ee3c7dd5 [Angerszhuuuu] Merge branch 'KYUUBI-5284' of https://github.com/AngersZhuuuu/incubator-kyuubi into KYUUBI-5284
ee0916f63 [Angerszhuuuu] Update HoodieCatalogRangerSparkExtensionSuite.scala
010260fa4 [Angerszhuuuu] Merge branch 'master' into KYUUBI-5284
c11d02def [Angerszhuuuu] update
b84f91f65 [Angerszhuuuu] update
42fbb0ffa [Angerszhuuuu] Update HoodieCatalogRangerSparkExtensionSuite.scala
c1346adb1 [Angerszhuuuu] update
2ec63ae94 [Angerszhuuuu] Update pom.xml
39bce7468 [Angerszhuuuu] update
c70b0ea2f [Angerszhuuuu] Update pom.xml
e1d85ff77 [Angerszhuuuu] Update pom.xml
59012ac25 [Angerszhuuuu] Update pom.xml
a46de65b5 [Angerszhuuuu] Update HoodieTest.java
b8173b893 [Angerszhuuuu] update
055713329 [Angerszhuuuu] Update table_command_spec.json
d7b21e820 [Angerszhuuuu] Update HoodieCatalogRangerSparkExtensionSuite.scala
0a93ff794 [Angerszhuuuu] [KYUUBI #5284] Kyuubi authz support Hoodie Alter Table Command
Lead-authored-by: Angerszhuuuu <angers.zhu@gmail.com>
Co-authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Kent Yao <yao@apache.org>
### _Why are the changes needed?_
To close#5362 .
Considering the maintenance burden of the Kyuubi community and easy cross-support for data lake projects.
Drop support EOLs of Spark 3 for the coming Spark 4.x era in kyuubi v1.9.
We will still do bugfix release for these spark3.0.x users.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No
Closes#5363 from AngersZhuuuu/KYUUBI-5362.
Closes#5362
d34cd6e2b [Angerszhuuuu] Update build.md
99f414bd4 [Angerszhuuuu] Update build.md
a5129e4f6 [Angerszhuuuu] Update build.md
6ee008cc5 [Angerszhuuuu] Update README.md
af792cc42 [Angerszhuuuu] Update master.yml
69b333161 [Angerszhuuuu] Merge branch 'master' into KYUUBI-5362
528554e9c [Angerszhuuuu] Update IcebergCatalogPrivilegesBuilderSuite.scala
427ebd48d [Angerszhuuuu] Update DataMaskingForJDBCV2Suite.scala
64809a54f [Angerszhuuuu] update
f7d89fd9b [Angerszhuuuu] [KYUUBI-5362] Kyuubi remove Authz test for spark3.0.3
Authored-by: Angerszhuuuu <angers.zhu@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- enable CI test on Scala-2.13 for all modules except Flink SQL engine
- For testing, choose available Spark engine home in `download` module by `SCALA_COMPILE_VERSION` of Kyuubi server
- Choose the Scala version of Spark engine main resource Jar in the following order:
1. `SPARK_SCALA_VERSION` system env
2. Extract Scala version from Spark home's `spark-core` jar filename
- Fixed 1 assertion error of kyuubi-spark-lineage module, as Spark on Scala 2.12 and 2.13 show different order of column linage output in `MergeIntoTable` ut
```
SparkSQLLineageParserHelperSuite:
- columns lineage extract - MergeIntoTable *** FAILED ***
inputTables(List(v2_catalog.db.source_t))
outputTables(List(v2_catalog.db.target_t))
columnLineage(List(ColumnLineage(v2_catalog.db.target_t.name,Set(v2_catalog.db.source_t.name)), ColumnLineage(v2_catalog.db.target_t.price,Set(v2_catalog.db.source_t.price)), ColumnLineage(v2_catalog.db.target_t.id,Set(v2_catalog.db.source_t.id)))) did not equal inputTables(List(v2_catalog.db.source_t))
outputTables(List(v2_catalog.db.target_t))
columnLineage(List(ColumnLineage(v2_catalog.db.target_t.id,Set(v2_catalog.db.source_t.id)), ColumnLineage(v2_catalog.db.target_t.name,Set(v2_catalog.db.source_t.name)), ColumnLineage(v2_catalog.db.target_t.price,Set(v2_catalog.db.source_t.price)))) (SparkSQLLineageParserHelperSuite.scala:182)
```
- Fixed other tests relying on Scala scripting results
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
Closes#5196 from bowenliang123/scala213-test.
Closes#5196
97fafacd3 [liangbowen] prevent repeated compilation for regrex pattern
76b99d423 [Bowen Liang] test on scala-2.13
Lead-authored-by: Bowen Liang <liangbowen@gf.com.cn>
Co-authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
### _Why are the changes needed?_
It is basically copied from `kyuubi-extension-spark-3-4`.
### _How was this patch tested?_
Compiled successfully:
```
build/mvn clean install -DskipTests -Pflink-provided,spark-provided,hive-provided,spark-3.5
```
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No
Closes#5336 from wForget/dev_spark_3_5.
Closes#5336
7ba99804a [wforget] remove iceberg.version in spark-3.5 profile
a18ce166a [wforget] Regenerate KyuubiEnsureRequirements based on EnsureRequirements in spark 3.5
4725c4701 [wforget] fix iceberg version
f5a8ea934 [wforget] Bump iceberg 1.4.0
06558dcfa [wforget] make kyuubi-spark-authz plugin compatible with Spark3.5
90d0e4c70 [wforget] make kyuubi-spark-authz plugin compatible with Spark3.5
4bc8d24d6 [wforget] add ci
1b3f2d916 [wforget] Make kyuubi spark extension compatible with Spark3.5
Authored-by: wforget <643348094@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Apache Spark 3.5.0 released in few days ago, this PR aims to add cross-version verification to ensure that Kyuubi engine built against Spark 3.4.1 works on Spark 3.5.0 runtime.
https://spark.apache.org/releases/spark-release-3-5-0.html
For users who want to try Kyuubi with Spark 3.5.0, please compile master/branch-1.8 via
```
./build/dist --tgz --web-ui --spark-provided --flink-provided --hive-provided
```
And then follow the [Quick Start](https://kyuubi.readthedocs.io/en/master/quick_start/quick_start.html) to setup Kyuubi and Spark properly.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
```log
Connected to: Spark SQL (version 3.5.0)
Driver: Kyuubi Project Hive JDBC Client (version 1.8.0-SNAPSHOT)
Beeline version 1.8.0-SNAPSHOT by Apache Kyuubi
0: jdbc:hive2://0.0.0.0:10009/default> select version();
+-------------------------------------------------+
| version() |
+-------------------------------------------------+
| 3.5.0 ce5ddad990373636e94071e7cef2f31021add07b |
+-------------------------------------------------+
1 row selected (0.39 seconds)
0: jdbc:hive2://0.0.0.0:10009/default> select kyuubi_version();
+-------------------+
| kyuubi_version() |
+-------------------+
| 1.8.0-SNAPSHOT |
+-------------------+
1 row selected (0.16 seconds)
```
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No.
Closes#5291 from pan3793/cross-3.5.
Closes#5291
24107a414 [Cheng Pan] Update .github/workflows/master.yml
1554e7131 [Cheng Pan] Verify Spark engine on Spark 3.5
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
branch-1.8 has been cut, we should update the GitHub workflow to daily publish snapshot jars on all active branches.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No.
Closes#5245 from pan3793/daily-snapshot-1.8.
Closes#5245
df3979743 [Cheng Pan] [INFRA] Daily deploy snapshot version on branch-1.8
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
`branch-1.5` and `branch-1.6` don't active for a while, the currently active maintained branches are `master` and `branch-1.7`
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No
Closes#5235 from pan3793/snapshot.
Closes#5235
d63e1a570 [Cheng Pan] [INFRA] Daily deploy snapshot version on master and branch-1.7
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Kyuubi fully supports Spark 3.4 now, it's time to move forward.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
Closes#5202 from pan3793/default-3.4.
Closes#5202
a0efccdbf [Cheng Pan] nit
30456dbb9 [Cheng Pan] nit
1cc83c871 [Cheng Pan] enable lineage test
d8ca7c7d8 [Cheng Pan] Switch to Spark 3.4 by default
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- Make Spark TPC-H connector compilable on Scala 2.13
- adding spark extensions to scala 2.13 compilation CI test
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
Closes#5195 from bowenliang123/scala213-conn.
Closes#5195
ad2558a16 [liangbowen] update
c50d29111 [liangbowen] update
d38eb3b6c [liangbowen] adapt spark connector plugin compilable on Scala 2.13
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
- adding a basic compilation CI test on Scala 2.13 , skipping test runs
- separate versions of `KyuubiSparkILoop` for Scala 2.12 and 2.13, adapting the changes of Scala interpreter packages
- rename `export` variable
```
[Error] /Users/bw/dev/kyuubi/kyuubi-server/src/test/scala/org/apache/kyuubi/server/api/v1/AdminResourceSuite.scala:563: Wrap `export` in backticks to use it as an identifier, it will become a keyword in Scala 3.
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No.
Closes#5188 from bowenliang123/sparksql-213.
Closes#5188
04f192064 [liangbowen] update
9e764271b [liangbowen] add ci for compilation with server module
a465375bd [liangbowen] update
5c3f24fdf [liangbowen] update
4b6a6e339 [liangbowen] use Iterable for the row in MySQLTextResultSetRowPacket
f09d61d26 [liangbowen] use ListMap.newBuilder
6b5480872 [liangbowen] Use Iterable for collections
1abfa29d8 [liangbowen] 2.12's KyuubiSparkILoop
b1c9da591 [liangbowen] rename export variable
b6a6e077b [liangbowen] remove original KyuubiSparkILoop
15438b503 [liangbowen] move back scala 2.12's KyuubiSparkILoop
dd3244351 [liangbowen] adapt spark sql module to 2.13
62d3abbf0 [liangbowen] adapt server module to 2.13
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
In minor cases, Spark Stage hangs forever when spark.sql.finalWriteStage.eagerlyKillExecutors.enabled is true.
The bug occurs if two conditions are met in the same time:
1. All executors are either removed because of idle time out or killed by FinalStageResourceManager.
Target executor num in YarnAllocator will be set to 0 and no more executor will be launched.
2. Target executor num in ExecutorAllocationManager equals to the executor num needed by final stage.
Then ExecutorAllocationManager will not sync target executor num to YarnAllocator.
### _How was this patch tested?_
- [x] Add a new test suite `FinalStageResourceManagerSuite`
Closes#5141 from zhouyifan279/adjust-executors.
Closes#5136
c4403eefa [zhouyifan279] assert adjustedTargetExecutors == 1
ea8f24733 [zhouyifan279] Add comment
5f3ca1d9c [zhouyifan279] [KYUUBI #5136][Bug] Spark App may hang forever if FinalStageResourceManager killed all executors
12687eee7 [zhouyifan279] [KYUUBI #5136][Bug] Spark App may hang forever if FinalStageResourceManager killed all executors
9dcbc780d [zhouyifan279] [KYUUBI #5136][Bug] Spark App may hang forever if FinalStageResourceManager killed all executors
Authored-by: zhouyifan279 <zhouyifan279@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
It was planned but actually delayed, remove this dummy module to save CI and avoid confusing users and release managers.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
Closes#5113 from pan3793/remove-kudu.
Closes#5113
ff8fd2e6a [Cheng Pan] Remove Spark Kudu connector
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
The main change is copy code from `kyuubi-extension-spark-common_2.12` and `kyuubi-extension-spark-3-3`. The reason copy `kyuubi-extension-spark-common_2.12` is that, Spark3.4 v1 writes does not have ctas...
### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
Closes#5018 from ulysses-you/extension.
Closes#5018
1ca4951e1 [ulysses-you] fix
fb20bb1f0 [ulysses-you] fix
338a3120a [ulysses-you] rebase
241a436fd [ulysses-you] fix
70618f368 [ulysses-you] fix
5f5e2b7c8 [ulysses-you] fix
301ae5c04 [ulysses-you] fix
bb9416811 [ulysses-you] fix
a38595d0a [ulysses-you] copy
9dec27d6c [ulysses-you] copy
Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: ulyssesyou <ulyssesyou@apache.org>
### _Why are the changes needed?_
- fetch versions for NodeJS and Pnpm from Maven properties
- enabled npm dependency caching for global packages and web-ui module, as the step above takes extra time in builds.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4948 from bowenliang123/pnpm-version.
Closes#4948
80b0f6934 [liangbowen] Reuse versions of NodeJS and Pnpm from maven properties
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
Authz plugin's tests now pass with Spark3.4.
` mvn clean install -pl :kyuubi-spark-authz_2.12 -Pspark-3.4 -am -DskipTests
&& mvn test -pl :kyuubi-spark-authz_2.12 -Pspark-3.4`
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4909 from bowenliang123/authz-enabletest-3.4.
Closes#4909
b0bdc106e [liangbowen] enable authz plugin tests with Spark 3.4
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
After previous efforts, we are close to the goal. This is the latest step to to allow Kyuubi run on JDK 17 by adding `--add-open` in bootstrap scripts and test the plugin's parameter.
This PR also enables JDK 17 test in CI, but only for Kyuubi server and Spark(3.3+) engine moudles, because
- Spark supports JDK 17 since 3.3, see SPARK-33772
- Flink does not support JDK 17 until the latest 1.17
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
- Run Kyuubi server on JDK 17
- Run simple Spark (3.3.2) query on JDK 17
- Run `kyuubi-zk-cli` on JDK 17
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4849 from pan3793/jdk-17.
Closes#4849
165e3558c [Cheng Pan] Enable JDK 17 in CI
f5778e7bd [Cheng Pan] Open modules in bootstrap scripts
76b507ffd [Cheng Pan] Open modules in UT
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
This PR aims to migrate the vanilla Zookeeper and Curator to the Kyuubi Shaded Zookeeper. It's the first step to adapting JDK 17.
There is a known issue [ZOOKEEPER-3779](https://issues.apache.org/jira/browse/ZOOKEEPER-3779) that Zookeeper 3.4 client can not run on JDK 14 and above, in https://github.com/apache/kyuubi-shaded/pull/5, we fixed this issue by a surgical.
With the above fixing, zk-3.4 and zk-3.6 clients both work well on JDK 17, we just randomly pick some cases to make sure zk-3.6 is tested
zk-3.4 client supports zk-3.4+ server, but zk-3.6 client only supports zk-3.5+ server; in the meanwhile, zk-3.4 is adopted widely, (CDH 5/6, HDP, EMR created before 2023).
We are sticky to zk-3.4 to ensure that Kyuubi can be out-of-box in the most existing Hadoop cluster but also provide zk-3.6 as an alternative(simply replace the kyuubi-shaded-zk-3.4 jar w/ kyuubi-shaded-zk-3.6, or build w/ -Pzookeeper-3.6) for users who concerns that zk-3.4 is EOL.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4852 from pan3793/shaded-zk.
Closes#4852
d960cc945 [Cheng Pan] remove staging repo
1b3622080 [Cheng Pan] Switch to Kyuubi Shaded Zookeeper
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- nodejs v18 is the current active LTS version of nodejs since October 2022, while v16 is in maintenance mode and no longer in the active status
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
This patch had conflicts when merged, resolved by
Committer: Cheng Pan <chengpan@apache.org>
Closes#4784 from bowenliang123/nodejs18.
Closes#4784
483c3ac56 [liangbowen] use nodejs v18 in ci builds and maven builds
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- v8 is now the stable version of pnpm
- lockfile v6 is used by default since pnpm v8.0, which improves the readability of the lockfile by removing hashes from package IDs and more resistant to git merge conflicts (https://github.com/pnpm/pnpm/releases/tag/v8.0.0)
- pnpm 8 supports nodejs 16 and 18 (https://pnpm.io/installation#compatibility)
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4783 from bowenliang123/pnpm8.
Closes#4783
8b3db3820 [liangbowen] use pnpm v8 and update lock file to v6
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- Add CI for Spark 3.4
- Kyuubi Spark TPC-DS/H connectors support Spark 3.4
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4741 from pan3793/spark-3.4.
Closes#4741
84a2d6ad7 [Cheng Pan] log
b9b2ec1fb [Cheng Pan] Add spark-3.4 profile
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- introduce a custom action for installing and caching Maven, to prevent repeatedly downloading Maven binary from remote.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4747 from bowenliang123/cache-maven.
Closes#4747
bbd9e1f4c [liangbowen] fix cache path
dc342180e [liangbowen] add action for caching maven in ci builds
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
This PR aims to remove Hudi integration tests from the Kyuubi project.
Actually, there is no obvious benefit to running Hudi tests w/ Kyuubi, since the real work happens on the compute engine and Hudi integration. Besides, Hudi's horrible dependency management brings significant maintenance efforts to the Kyuubi community.
This change only affects tests, does not affect any functionality.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4744 from pan3793/remove-hudi.
Closes#4744
ea99f747e [Cheng Pan] Remove Hudi integration tests
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
Fixes#4712
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4718 from anurag-rajawat/upgrade-spark.
Closes#4712
79dcf1b79 [Anurag Rajawat] Bump Spark from 3.2.3 to 3.2.4
Authored-by: Anurag Rajawat <anuragsinghrajawat22@gmail.com>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
- super-liner action v5.0.0 release notes: https://github.com/github/super-linter/releases/tag/v5.0.0
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4695 from bowenliang123/superlinter-v5.
Closes#4695
7cd3095a9 [liangbowen] update
7826c8043 [liangbowen] update
89e7b3d7d [liangbowen] bump superlinter to v5
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
Fixes#4712
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4718 from anurag-rajawat/upgrade-spark.
Closes#4712
79dcf1b79 [Anurag Rajawat] Bump Spark from 3.2.3 to 3.2.4
Authored-by: Anurag Rajawat <anuragsinghrajawat22@gmail.com>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
- add ut to check generated Ranger policy file in #4585
- manually activated `genpolicy` profile in CI builds, as the property based activation not auto-triggered as expectedly with property `ranger.version=2.4.0` set in project parent pom
- Support regenerated policy file within the same test suite, by running
`KYUUBI_UPDATE=1 build/mvn clean test -pl :kyuubi-spark-authz_2.12 -Dtest=none -DwildcardSuites=org.apache.kyuubi.plugin.spark.authz.gen.PolicyJsonFileGenerator -Pgenpolicy`
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4677 from bowenliang123/authz-check-policy-gen.
Closes#4677
a372bdfd4 [liangbowen] remove unnecessary profile used in style workflow
7562c88f2 [liangbowen] include in spotless
37b674223 [liangbowen] update policy id
724ec5e28 [liangbowen] replace counter by using zipWithIndex
d322980e7 [liangbowen] extract KRangerPolicyResource object to simplify resource assembly
42c37606a [liangbowen] nit
18a8f4c51 [liangbowen] add usage comments
4ee254d6d [liangbowen] fix issue name in docs
d3cb08d21 [liangbowen] improve file reading
37e4c9c9f [Bowen Liang] Merge branch 'master' into authz-check-policy-gen
6366c50e4 [liangbowen] rename profile to `gen-policy` and remove activation rule by property setting
892faf5ef [liangbowen] update clue
266baa71a [liangbowen] update
cb94e8014 [liangbowen] update
de1f36531 [liangbowen] cleanup
e88c75d46 [liangbowen] check policy file gen
Lead-authored-by: liangbowen <liangbowen@gf.com.cn>
Co-authored-by: Bowen Liang <bowenliang@apache.org>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
After pnpm released 8.X version
We should specify pnpm version when npm install.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
- [x] CI with Web UI Build Check & Web UI Style Check
Closes#4626 from zwangsheng/fix/web_ui_ci.
Closes#4626
f7fdc0676 [Cheng Pan] Update .github/workflows/web-ui.yml
61bc10124 [Cheng Pan] Update .github/workflows/style.yml
7d9353218 [zwangsheng] Specify pnpm version
d678c6cb6 [zwangsheng] for feature version
6d83c2b01 [zwangsheng] revert
bbc4b4c2c [zwangsheng] Specify pnpm version
Lead-authored-by: zwangsheng <2213335496@qq.com>
Co-authored-by: Cheng Pan <pan3793@gmail.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- Flink 1.15.4 release note: https://flink.apache.org/2023/03/15/apache-flink-1.15.4-release-announcement/
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4598 from bowenliang123/flink-1.15.4.
Closes#4598
40ea2396f [liangbowen] bump flink-1.15 from 1.15.3 to 1.15.4
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
As discussed before, Kyuubi is going to support the latest 3 Flink versions, and to reduce the complexity of supporting Flink 1.17 https://github.com/apache/kyuubi/pull/4368, we are going to remove support Flink 1.14 first.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4588 from pan3793/rm-flink-1.14.
Closes#4387
97d263324 [Cheng Pan] Remove support for Flink 1.14
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- introduce `action/super-linter` with multiple linters, (https://github.com/marketplace/actions/super-linter)
- using the smaller size version of image `github/super-linter/slim` for faster image pulling
- enable linting for `bash_exec` for checking executable of bash scripts, with fixed `tools/spark-block-cleaner/kubernetes/docker/entrypoint.sh`
- enable integrity checks for JSON files in JSONC style
- enable integrity checks for XML files
- enable linting for ENV files
### _How was this patch tested?_
- [x] Pass CI jobs
Closes#4295 from bowenliang123/superlinter-action.
Closes#4295
321d24912 [liangbowen] update
497bf801c [liangbowen] use JSONC instead of JSON to support comments
43787e233 [liangbowen] use superlinter
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
- Setup and use cached `build/mvn` in CI jobs, to prevent repeatedly downloading mvn from remote
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4481 from bowenliang123/setup-maven.
Closes#4481
3906430da [liangbowen] ignore occasional mvnd failure
366806256 [liangbowen] setup cached build/maven
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
Add `Kyuubi Kubernetes IT` Module to CI style check
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
- [X] Run CI
Closes#4461 from zwangsheng/ci/add_kubernetes.
Closes#4461
7ebac75b0 [zwangsheng] fix style for kubernetes-it
c1754d452 [zwangsheng] Add Kyuubi Kubernetes IT Module to CI style check
Authored-by: zwangsheng <2213335496@qq.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- concurrent execution and smart builder is shipped with `mvnd`, but the CI log shows they are not activated
- increase maximum degree of concurrency and utilize `SmartBuilder` feature in `mvnd`. `-Dmvnd.minThreads` is set as fallback and it will be ignored if `--threads` or `-Dmvnd.threads` is used.
#### Before:
(https://github.com/apache/kyuubi/actions/runs/4276652363/jobs/7444896450#step:6:64)
```
[INFO] Build maximum degree of concurrency is 1
```
#### After:
(https://github.com/apache/kyuubi/actions/runs/4279322563/jobs/7449912132#step:8:64)
```
[INFO] Using the SmartBuilder implementation with a thread count of 4
[INFO] Build maximum degree of concurrency is 4
```
(https://github.com/apache/kyuubi/actions/runs/4279322563/jobs/7449912132#step:8:553)
```
[INFO] Segment walltime 13 s, segment projects service time 36 s, effective/maximum degree of concurrency 2.63/4
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4420 from bowenliang123/mvnd-smart.
Closes#4274
64f0cc53e [liangbowen] update
391da26d3 [liangbowen] quite in dep and style jobs
c776356a5 [liangbowen] use -Dmvnd.minThreads
d10bfb70c [liangbowen] use -Dmvnd.minThreads
f793d2b6b [liangbowen] warn
bbb929fe3 [liangbowen] quiet
a5e6d4914 [liangbowen] logging warn level
3cfbaad45 [liangbowen] enable quite option and cancel redirect stderr
09442ff69 [liangbowen] update
ca5f855fa [liangbowen] try to redirect stderr
0ebdd18d4 [liangbowen] set completion and print MAVEN_CLI_OPTS
d3b2c9656 [liangbowen] revert explicitly setting `mvnd.noBuffering`
e2eca0140 [liangbowen] revert explicitly setting `mvnd.noBuffering`
dba25e327 [liangbowen] explicitly set `mvnd.noBuffering` to false prevent displaying events continuously
b7583f936 [liangbowen] shell
d60ebd047 [liangbowen] change mvnd cache key
2cc576ef8 [liangbowen] dep
bbd6414c9 [liangbowen] move multithread setting to mvnd's MAVEN_CLI_OPTS
347b58c8c [liangbowen] increase concurrency for spotless check
15c5519e4 [liangbowen] increase concurrency
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
Correct the Spark image tag to recover "Kyuubi Server On Kubernetes Integration Test"
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4369 from pan3793/spark-3.3.2.
Closes#4338
3232bf9f [Cheng Pan] [KYUUBI #4338][FOLLOWUP] Fix K8s integration tests
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
- to avoid violation in Apache archives website's daily total download size quote, prevent repeatedly downloading engine archives by manual cache in actions
### _How was this patch tested?_
- [ ] Pass CI jobs and check cached engine archives
Closes#4348 from bowenliang123/cache-downloaded-archives.
Closes#4348
a9deeea4 [liangbowen] apply engine caching to all jobs in master
d253a497 [liangbowen] Merge commit '38eb74c26ebbdbb57aba51ad18c7e4a6bb9e1144' into cache-downloaded-archives
0cf2a759 [liangbowen] remove conditions for action
38eb74c2 [Bowen Liang] Merge branch 'master' into cache-downloaded-archives
105d507e [liangbowen] remove
c6542797 [liangbowen] extract `cache-engine-archives` action and cache engine archives
Lead-authored-by: liangbowen <liangbowen@gf.com.cn>
Co-authored-by: Bowen Liang <bowenliang@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
### _Why are the changes needed?_
according to `io.trino.jdbc.ColumnInfo`, there are some type requring signature parameter.
- varchar(n)
- char(n)
- decimal(precision, scale)
It failed with trino jdbc now
<img width="613" alt="image" src="https://user-images.githubusercontent.com/12025282/218707052-a2e9dc91-0333-483c-bc0a-96baec213578.png">
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4328 from ulysses-you/fix-signature.
Closes#4328
aede5cec [ulysses-you] nit
ae1a7968 [ulysses-you] fix test
8ecdb346 [ulysses-you] Make Trino jdbc driver work
Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: ulyssesyou <ulyssesyou@apache.org>
### _Why are the changes needed?_
- Check whether modules are available by running `mvnd dependency:resolve`. if true, skip the `build` step of `mvnd install`.
- Reducing the time cost of dependency workflow from ~ 10 min to ~4 min ([see log](https://github.com/apache/kyuubi/actions/runs/4162071562/jobs/7200786535)).
### _How was this patch tested?_
- [x] Pass CI workflows
Closes#4315 from bowenliang123/dep-check-first.
Closes#4315
7c20ea24 [liangbowen] nit
65397868 [liangbowen] add validate to bring back enforcer check
1c035136 [liangbowen] add validate to bring back enforcer check
d4e703de [liangbowen] update dep.yml
8f8d0457 [liangbowen] update dep.yml
e7adfacf [liangbowen] update
c88eff98 [liangbowen] check modules available before build
Authored-by: liangbowen <liangbowen@gf.com.cn>
Signed-off-by: liangbowen <liangbowen@gf.com.cn>
### _Why are the changes needed?_
```
...
07:43:26.520 KyuubiSessionManager-exec-pool: Thread-779 INFO ProcBuilder: Logging to /home/runner/work/kyuubi/kyuubi/kyuubi-server/target/work/runner/kyuubi-spark-batch-submit.log.0
...
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request
Closes#4317 from pan3793/ci-log.
Closes#4317
dbde5306 [Cheng Pan] Upload kyuubi-spark-batch-submit.log on CI failure
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: ulyssesyou <ulyssesyou@apache.org>
