a07d2347cf
### _Why are the changes needed?_ to close #3942 . Adapt Spark's JDBC data type to Hive data type definitions. 1. adapt `DoubleType` to "DOUBLE" for compatibility with Hive 2.1.x and below - "DOUBLE PRECISION" mapped in [`JdbcUtils.getCommonJDBCType` ](https://github.com/apache/spark/blob/v3.3.1/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcUtils.scala#L145) is a pure alias for DOUBLE, only available starting with Hive 2.2.0. 2. adapt `IntegerType` to "INT" instead of "INTEGER" for compatibility with Hive 2.1.x and below - "INTEGER" is synonym for INT since Hive 2.2.0 3. fix the unsupported Spark data type mapping to the Hive data type definition with correct mappings: - FloatType to "FLOAT" instead of "REAL" - StringType to "STRING" instead of "TEXT" - BooleanType to "BOOLEAN" instead of "BIT(1)" - BinaryType to "BINARY" instead of "BLOB" - ByteType to "TINYINT" instead of "BYTE" Hive Data Type docs refers to https://cwiki.apache.org/confluence/display/hive/languagemanual+types . ### _How was this patch tested?_ - [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #3943 from bowenliang123/3942-hivetype. Closes #3942 10c3ddea [liangbowen] Add ut for non adapted data types 5f6ce015 [liangbowen] Add links to doc and issues for Hive data type definition 1ae0fd1d [liangbowen] mapping IntegerType to "INT" instead of "INTEGER" for compatibility with Hive 2.1.x and below 13486bb2 [liangbowen] update ut cb0a0537 [liangbowen] mapping ByteType to "TINYINT" c8a02828 [liangbowen] ut 25862584 [liangbowen] mapping BinaryType to "BINARY" e8b65a00 [liangbowen] adapt to Hive data type definitions 3f5cc75a [liangbowen] add ut 3a818da9 [liangbowen] comments a471466a [liangbowen] mapping FloatType to "FLOAT", BooleanType to "BOOLEAN" b9506dc9 [liangbowen] mapping DoubleType to "DOUBLE" and StringType to "STRING" Authored-by: liangbowen <liangbowen@gf.com.cn> Signed-off-by: Cheng Pan <chengpan@apache.org> |
||
|---|---|---|
| .. | ||
| server/kyuubi-server-plugin | ||
| spark | ||
| README.md | ||
For developers
This folder contains plugins/extension for kyuubi server and different engine types.
- ext
- kyuubi-server
- spark
- flink
- trino
- hive
- others
- ...