Commit Graph

77 Commits

Author SHA1 Message Date
Kent Yao
b11a9d4172
Support _HOST 2020-12-14 20:47:11 +08:00
Kent Yao
d357203b0e Add authentication guide 2020-11-30 17:45:20 +08:00
Kent Yao
87e243c596 support query log for async exec 2020-11-27 16:40:07 +08:00
Kent Yao
575e9c9599 Add support for user defaults 2020-11-25 21:37:25 +08:00
Kent Yao
dd4fb44d40 Kinit Auxiliary Service should be scheduled w/o delay during starting 2020-11-25 14:38:21 +08:00
Kent Yao
f8ef7c8997 Support ansynchronous exec for Kyuubi Server 2020-11-24 10:55:29 +08:00
Kent Yao
9ca23bcf20 Support ansynchronous exec for Spark SQL engine 2020-11-20 19:52:57 +08:00
Kent Yao
f5900b052f reorder doc 2020-11-19 11:32:29 +08:00
Kent Yao
91d0ce302e add HADOOP_CONF_DIR to classpath 2020-11-19 00:30:25 +08:00
Kent Yao
ea1431b162 [DOC] Integration w/ Hive Metastore 2020-11-18 16:37:21 +08:00
Kent Yao
64b83a48c5 Show possible error even timeout 2020-11-16 22:56:25 +08:00
Kent Yao
15820a2b09 fix 2020-11-16 19:42:06 +08:00
Kent Yao
b3d996917d fix static and core conf via hiveconf or hivevar 2020-11-13 16:08:47 +08:00
Kent Yao
a69fb2553d nit 2020-11-13 14:48:05 +08:00
Kent Yao
d58484c4ec pics 2020-11-13 14:18:45 +08:00
Kent Yao
4396e59abe Add doc for overview 2020-11-13 14:17:42 +08:00
Kent Yao
fb4bace6a5 Ignore static and other unmodifiable settting in session 2020-11-11 11:15:09 +08:00
Kent Yao
574c119712 JDBCTests added and some related bugs fixed 2020-11-10 21:24:02 +08:00
Kent Yao
a67d485951 Missing version in readMessageBegin, old client
2020-11-10 10:15:42,001 [61208289] - ERROR [FrontendServiceHandler-Pool: Thread-449:TThreadPoolServer$WorkerProcess@315] - Thrift error occurred during processing of message.
org.apache.thrift.protocol.TProtocolException: Missing version in readMessageBegin, old client?
	at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:234)
	at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:27)
	at org.apache.kyuubi.service.authentication.TSetIpAddressProcessor.process(TSetIpAddressProcessor.scala:36)
	at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:310)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
2020-11-10 17:21:17 +08:00
Kent Yao
f8dc5a9ca3 Using ha.zookeeper.acl.enabled to control ZK acl 2020-11-09 17:34:55 +08:00
Kent Yao
c8bdb5649d Fix NullPointerException in org.apache.kyuubi.session.KyuubiSessionImpl.close 2020-11-09 14:57:17 +08:00
Kent Yao
56ead65496 Add Trouble Shooting with a HADOOP_CONF_DIR case 2020-11-06 19:40:23 +08:00
Kent Yao
b0bb38fa9d Improve OpenSession error Msg 2020-11-06 19:00:00 +08:00
Kent Yao
92e3d6d443 FIX org.apache.kyuubi.config.KyuubiConf#toSparkPrefixedConf
/**
   * This method is used to convert kyuubi configs to configs that Spark could identify.
   * - If the key is start with `spark.`, keep it AS IS as it is a Spark Conf
   * - If the key is start with `hadoop.`, it will be prefixed with `spark.hadoop.`
   * - Otherwise, the key will be added a `spark.` prefix
   * @return a map with spark specified configs
   */
2020-11-06 14:37:40 +08:00
Kent Yao
7e1f6ef7ec Add Trouble Shooting with a UnsupportedClassVersionError case 2020-11-06 11:40:25 +08:00
Kent Yao
39e5da584d add gitter 2020-11-05 20:49:22 +08:00
Kent Yao
7157370ef6 Add Building Doc 2020-11-05 19:05:26 +08:00
Kent Yao
bd319f4216 remove most tests in kyuubi-server module 2020-11-05 14:14:11 +08:00
Kent Yao
cdb46e90db Add Configurations Guide 2020-11-04 22:40:02 +08:00
Kent Yao
c888a5b6e0 add logo 2 2020-11-03 21:07:10 +08:00
Kent Yao
2331d9c490 add logo 2 2020-11-03 21:07:04 +08:00
Kent Yao
43bfbed473 add logo 2020-11-03 20:46:58 +08:00
Kent Yao
9aea755b5e add readthedocs badge 2020-11-03 15:44:06 +08:00
Kent Yao
a8281fc34c readthedocs 2020-11-03 14:59:32 +08:00
Kent Yao
67858e8c15 [DOCS] Add qucik start documentation 2020-11-02 21:07:11 +08:00
Kent Yao
ba9bff1487 https://stackoverflow.com/questions/56336234/build-fail-sphinx-error-contents-rst-not-found
FIX SphinxError: master file /home/docs/checkouts/readthedocs.org/user_builds/kyuubi/checkouts/latest/docs/contents.rst not found
2020-10-28 16:01:56 +08:00
Kent Yao
7fcad51d55 doc init 2 2020-10-28 15:55:21 +08:00
Kent Yao
0abf85dcb7 doc init 2020-10-28 15:28:24 +08:00
Kent Yao
470d7ca90d Make man service work 2020-09-21 14:05:34 +08:00
Kent Yao
6621d361af
add user info and statement id in executing log (#212)
* add user info and statement id in executing log

* add ut

* update conf

* long max

* modify log

* modify log

* unpersit

* typo

* add ut

* fix ut

* typo

* code cov

* add log

* ex order

* rm some ex

* none get

* Revert "none get"

This reverts commit 6792e33db11619ec4fa7c075706da1500fb229e3.

* rm debug
2019-08-16 16:51:12 +08:00
Kent Yao
5edf961a44
[KYUUBI-204]Apache Kudu Integration Guide fix #204 (#205)
* [KYUUBI-204]Apache Kudu Integration Guide fix #204

* references
2019-07-09 16:47:20 +08:00
Kent Yao
63245dd691
[KYUUBI-196]Refactoring SparkSessionWithUGI‘s get and create SparkContext logic (#197)
* fxit 194 Refactoring SparkSessionWithUGI‘s get and create sc logic

* upgrade scala-maven-plugin to 3.4.6

* fix travis

* fix travis again

* fix travis open jdk8

* fix travis open jdk8

* fix travis open jdk8

* fix travis open jdk8

* fix travis open jdk8

* add state to get yarn applications

* promise complete

* fix #69 capture spark exceptions

* typo

* fix #196 add ut and log

* typo

* typo

* add ut
2019-06-17 11:47:13 +08:00
Kent Yao
e295995959
[KYUUBI-194]Add a server side metrics system (#195)
* fix #194 add server side metrics system

* add spark session cache size

* add ut

* add ut

* add doc
2019-06-12 11:12:18 +08:00
Kent Yao
505e01d7e2 [KYUUBI-186]Add a max cache time to clean SparkSessions that may have token expiry issue
---

---

fix #186 fix #187

---
Squashed commit of the following:

commit 4b8297f8d76b40f22e5419c72f0ddf1eadb3c7d9
Author: Kent Yao <yaooqinn@hotmail.com>
Date:   Mon May 20 19:36:34 2019 +0800

    add log

commit 7b811f8a0ccb65fb0d0832ba1313e3c099f74dad
Author: Kent Yao <yaooqinn@hotmail.com>
Date:   Thu May 16 16:46:04 2019 +0800

    add ut

commit 19f67fa8909c4a19f49fbe3b30a75d09c0736350
Author: Kent Yao <yaooqinn@hotmail.com>
Date:   Thu May 16 10:57:24 2019 +0800

    fix ut

commit 7ad7c20f6536d3bbc9d789c71af238331a3e03c5
Author: Kent Yao <yaooqinn@hotmail.com>
Date:   Thu May 16 10:40:42 2019 +0800

    fix ut

commit 9a114abb8be3083160ea01dab188333f675677e4
Author: Kent Yao <yaooqinn@hotmail.com>
Date:   Thu May 16 00:10:57 2019 +0800

    mv cache validating in its own class

commit 0afba5a5ab7a005948acde545b4a68c8d26bfce9
Author: Kent Yao <yaooqinn@hotmail.com>
Date:   Wed May 15 11:47:42 2019 +0800

    fix ut

commit eff3f41f91b4bfc2bfc84efc6af29fa4365d9dae
Author: Kent Yao <yaooqinn@hotmail.com>
Date:   Wed May 15 11:12:51 2019 +0800

    add ut

commit 9bbbea78bcb8f5d409819628974628f0bf59f69c
Author: Kent Yao <yaooqinn@hotmail.com>
Date:   Tue May 14 23:17:00 2019 +0800

    add ut

commit 0e0e59eedc055ddb6788a9dd6f07d4308772145d
Author: Kent Yao <yaooqinn@hotmail.com>
Date:   Tue May 14 19:39:09 2019 +0800

    mv init time to spark session cache

commit a4a1c69e29ec7d5d51a0e5026cf8f3c2aaf6b2ef
Author: Kent Yao <yaooqinn@hotmail.com>
Date:   Tue May 14 18:36:06 2019 +0800

    Add a max cache time to clean SparkSessions that may have token expiry issue fix #186
2019-05-21 22:07:18 +08:00
Kent Yao
1f0bc742e4
[KYUUBI-167][FOLLOWUP]populate tokens via spark session cache mgr (#183)
* Prepare releasing v0.6.0

* fix #KYUUBI-167 populate token via spark session cache mgr

* typo

* fix ut

* code cov
2019-05-07 10:32:04 +08:00
Kent Yao
13c68b5550
[KYUUBI-167]Handling ApplicationMaster token expiry to support long caching sparkcontext with dynamic executor allocation (#168)
* [KYUUBI-167]Handling ApplicationMaster token expiry to support long caching sparkcontext with dynamic executor allocation - fix #167

* fix ut

* add ut

* fix ut

* add doc

* fix https://github.com/apache/spark/pull/24120

* logger

* typo

* comment/dist jar

* fix ut

* typo
2019-03-19 13:51:08 +08:00
Kent Yao
51e7a5d480
Prepare for developing v0.6.0 (#154) 2019-02-27 13:40:29 +08:00
Kent Yao
ae85c245d2
Prepare releasing v0.5.0 (#152)
* Prepare releasing 0.5.0

* update documentations
2019-02-25 12:33:48 +08:00
Kent Yao
72e664fced
[KYUUBI-116][Experimental] Support long cache spark session in kerberized cluster (#117)
* fixes @116 Support long caching SparkSession/SparkContext for secured hadoop cluster

* handle sub classes of coarse grained scheduler backend

* fix ut for spark 2.2

* updating doc
2019-02-23 01:10:05 +08:00
Kent Yao
7171f77d7a fix #140 Add a server side configuration to limit the query result size 2019-01-11 15:37:17 +08:00