6bd0016fe2
### _Why are the changes needed?_ 1. this PR introduces a new configuration called `kyuubi.operation.result.arrow.timestampAsString`, when true, arrow-based rowsets will convert timestamp-type columns to strings for transmission. 2. `kyuubi.operation.result.arrow.timestampAsString` default setting to false for better transmission performance 3. the PR fixes timezone issue in arrow based result format described in #3958 ### _How was this patch tested?_ - [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request Closes #4326 from cfmcgrady/arrow-string-ts. Closes #4326 38c7fc9b [Fu Chen] fix style d864db00 [Fu Chen] address comment b714b3ee [Fu Chen] revert externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/schema/RowSet.scala 6c4eb507 [Fu Chen] minor 289b6007 [Fu Chen] timstampAsString = false by default 78b7caba [Fu Chen] fix f5601356 [Fu Chen] debug info b8e4b288 [Fu Chen] fix ut 87c6f9ef [Fu Chen] update docs 86f6cb73 [Fu Chen] arrow based rowset timestamp as string Authored-by: Fu Chen <cfmcgrady@gmail.com> Signed-off-by: Cheng Pan <chengpan@apache.org> |
||
|---|---|---|
| .. | ||
| spark | ||
| engine_lifecycle.md | ||
| engine_on_kubernetes.md | ||
| engine_on_yarn.md | ||
| engine_share_level.md | ||
| high_availability_guide.md | ||
| hive_metastore.md | ||
| index.rst | ||
| kyuubi_on_kubernetes.md | ||
| migration-guide.md | ||
| settings.md | ||