* [KYUUBI-156][KYUUBI-185] Implementing all meta operations support by spark
* rm used metastore client fix#156fix#185
* add more uts
* add more uts
* fix bugs and add show functions
* meta operation method abstraction
* readme
---
---
fix#181fix#120
---
Squashed commit of the following:
commit 3e7f81f3b8e5c4663fa54ed3e676312139693197
Author: hustfeiwang <wangfei3@corp.netease.com>
Date: Thu May 23 18:32:08 2019 +0800
fix unit test
commit 73763d7ca8416156132cb9e8df3e86966bf8c6e6
Author: hustfeiwang <wangfei3@corp.netease.com>
Date: Thu May 23 09:48:21 2019 +0800
set statementId to a val
commit 128effa6a9a89c5bb8f9c91f219e4e257ff96372
Author: hustfeiwang <wangfei3@corp.netease.com>
Date: Wed May 22 16:06:45 2019 +0800
set isClosedOrCanceled to a protected method
commit 267b55f1d2497872e348fc494a3ff7132a6012f0
Author: hustfeiwang <wangfei3@corp.netease.com>
Date: Wed May 22 10:10:48 2019 +0800
KYUUBI-120: define interface class IKyuubiOperation to allow different implementation of kyuubiOperation
---
---
fix#186fix#187
---
Squashed commit of the following:
commit 4b8297f8d76b40f22e5419c72f0ddf1eadb3c7d9
Author: Kent Yao <yaooqinn@hotmail.com>
Date: Mon May 20 19:36:34 2019 +0800
add log
commit 7b811f8a0ccb65fb0d0832ba1313e3c099f74dad
Author: Kent Yao <yaooqinn@hotmail.com>
Date: Thu May 16 16:46:04 2019 +0800
add ut
commit 19f67fa8909c4a19f49fbe3b30a75d09c0736350
Author: Kent Yao <yaooqinn@hotmail.com>
Date: Thu May 16 10:57:24 2019 +0800
fix ut
commit 7ad7c20f6536d3bbc9d789c71af238331a3e03c5
Author: Kent Yao <yaooqinn@hotmail.com>
Date: Thu May 16 10:40:42 2019 +0800
fix ut
commit 9a114abb8be3083160ea01dab188333f675677e4
Author: Kent Yao <yaooqinn@hotmail.com>
Date: Thu May 16 00:10:57 2019 +0800
mv cache validating in its own class
commit 0afba5a5ab7a005948acde545b4a68c8d26bfce9
Author: Kent Yao <yaooqinn@hotmail.com>
Date: Wed May 15 11:47:42 2019 +0800
fix ut
commit eff3f41f91b4bfc2bfc84efc6af29fa4365d9dae
Author: Kent Yao <yaooqinn@hotmail.com>
Date: Wed May 15 11:12:51 2019 +0800
add ut
commit 9bbbea78bcb8f5d409819628974628f0bf59f69c
Author: Kent Yao <yaooqinn@hotmail.com>
Date: Tue May 14 23:17:00 2019 +0800
add ut
commit 0e0e59eedc055ddb6788a9dd6f07d4308772145d
Author: Kent Yao <yaooqinn@hotmail.com>
Date: Tue May 14 19:39:09 2019 +0800
mv init time to spark session cache
commit a4a1c69e29ec7d5d51a0e5026cf8f3c2aaf6b2ef
Author: Kent Yao <yaooqinn@hotmail.com>
Date: Tue May 14 18:36:06 2019 +0800
Add a max cache time to clean SparkSessions that may have token expiry issue fix#186
commit 065238d830e1bf237cf5b5d74f987827cbe3ba61
Author: Guillem Nieto <gnieto@scopely.com>
Date: Thu May 9 17:01:37 2019 +0200
Cast to int wc jar count
While using Kyuubi on my local environment (OSX) I'm having issues with
`wc` returning the result with a preffixed tab character.
Due to this issue, it fails to compare " 1" with "1" and script
is aborted due to duplicated JARs found.
To solve that, I'm casting `wc` result to integer and perform integer
comparisions, which seems to solve the issue.
* fix#170 persist and coalesce before to local iterator while using incremental result collect
* add log
* use df.rdd.getPartitionNum as upper limit
* log & ut
* bypass coalesce if df.rdd.partition.size is small
* add ut
* code cov
* code cov
* reset class loader
* do not add jar twice in one create function cmd
* the classloader should be closeable
* the classloader should be closeable revert
* Revert "the classloader should be closeable revert"
This reverts commit 35602a0b689da0224a4fe3dcd778afabc3f7a803.
* add ut
* add ut
* add ut
* fix ut
fix#150
* [KYUUBI-167]Handling ApplicationMaster token expiry to support long caching sparkcontext with dynamic executor allocation - fix#167
* fix ut
* add ut
* fix ut
* add doc
* fix https://github.com/apache/spark/pull/24120
* logger
* typo
* comment/dist jar
* fix ut
* typo
* fixes @116 Support long caching SparkSession/SparkContext for secured hadoop cluster
* handle sub classes of coarse grained scheduler backend
* fix ut for spark 2.2
* updating doc
