apache/kyuubi
1
0
Fork 0
Code Issues Pull Requests Actions 5 Packages Projects Releases Wiki Activity
627 Commits 23 Branches 0 Tags 65 MiB
512fbc50cf
Commit Graph

627 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Zen
512fbc50cf
Support Spark YARN cluster deployment mode fix #264 
Squashed commit of the following:

commit 3c875db5dd245b4f978afec92e1d02ac831c94bf
Author: zen <xinjingziranchan@gmail.com>
Date:   Wed Dec 23 19:51:22 2020 +0800

    remove unnecessary code

commit c2487ee7ca4bb4271c7ae6a6263e32e040c4c159
Merge: 67bf250 d8e5c5a
Author: zen <xinjingziranchan@gmail.com>
Date:   Wed Dec 23 10:58:16 2020 +0800

    Merge branch 'master' into pr4-deploy-mode-with-cluster

    # Conflicts:
    #	externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/SparkSQLEngine.scala

commit 67bf25062d3cb38ccf3831b2e73561e16c4f8576
Author: zen <xinjingziranchan@gmail.com>
Date:   Wed Dec 23 10:51:46 2020 +0800

    delete unit test

commit 053834808f37c18a777a923d989a9b4d62d8419c
Author: zen <xinjingziranchan@gmail.com>
Date:   Wed Dec 23 10:34:05 2020 +0800

    Support Spark YARN cluster deployment mode

commit 61b24d0e3246f319b8d2cf70932a5b71cbf4a28a
Author: zen <xinjingziranchan@gmail.com>
Date:   Tue Dec 15 15:10:31 2020 +0800

    fix null exception and add unit test

commit 363cf93a77294485e7093716b6cb1cd58b4dba45
Author: zen <xinjingziranchan@gmail.com>
Date:   Tue Dec 15 14:21:19 2020 +0800

    Support Spark YARN cluster deployment mode
 2020-12-23 20:45:09 +08:00
Kent Yao
d8e5c5aa4a
get the lastOption of serverList 2020-12-21 16:31:54 +08:00
Zen
e8bc44c2ad
[#258] delete the engine's timeoutChecker scheduler and merge it to sessionManager's scheduler 
fix #258
Squashed commit of the following:

commit 5b0a4d1f77b32cef00f55b80b8562752c1f462d7
Author: zen <xinjingziranchan@gmail.com>
Date:   Wed Dec 16 10:15:16 2020 +0800

    fix closing session log

commit c0fe24d06638f9be18d91a4ad05943a023967c62
Author: zen <xinjingziranchan@gmail.com>
Date:   Sat Dec 12 22:08:25 2020 +0800

    move this check task to SparkSQLSessionManager.

commit 1968913702872f0ba433679596596f114f2e16d1
Author: zen <xinjingziranchan@gmail.com>
Date:   Thu Dec 10 15:55:32 2020 +0800

    add log when close session

commit 26a944787c63666813a0d64e67a36b671640d493
Author: zen <xinjingziranchan@gmail.com>
Date:   Thu Dec 10 11:32:45 2020 +0800

    delete the engine's timeoutChecker scheduler and merge it to  sessionManager's scheduler.
 2020-12-21 11:01:31 +08:00
Kent Yao
b11a9d4172
Support _HOST 2020-12-14 20:47:11 +08:00
Kent Yao
f5eab9bf32 README Update 2020-12-13 02:57:16 +08:00
Kent Yao
671b8c668f README Update 2020-12-13 02:42:44 +08:00
Kent Yao
a8b70d0841
Add a one-click tpcds generating tool (#262) 
* Add a one click tpcds generating tool

* nit
 2020-12-12 03:04:42 +08:00
Zen
7fd6f71271 ZkClient should close when the session is opened 
fixes #260

Squashed commit of the following:

commit ed4fa0cad9f8ad0053035b44a3ca703dd49ba362
Author: zen <xinjingziranchan@gmail.com>
Date:   Fri Dec 11 11:13:32 2020 +0800

    release the zkClient after session established.

commit add46a6acaa19942fcea3175ae253fdae2ef525b
Author: zen <xinjingziranchan@gmail.com>
Date:   Thu Dec 10 14:49:26 2020 +0800

    ZkClient should close when the session is opened.
 2020-12-11 15:49:34 +08:00
Kent Yao
d357203b0e Add authentication guide 2020-11-30 17:45:20 +08:00
Kent Yao
5374398050 Stringlify exception for better client debugging 2020-11-27 22:20:51 +08:00
Kent Yao
2d5d58cb9f Fix Kerberos Auth Error 2020-11-27 21:22:09 +08:00
Kent Yao
87e243c596 support query log for async exec 2020-11-27 16:40:07 +08:00
Kent Yao
575e9c9599 Add support for user defaults 2020-11-25 21:37:25 +08:00
Kent Yao
dd4fb44d40 Kinit Auxiliary Service should be scheduled w/o delay during starting 2020-11-25 14:38:21 +08:00
Kent Yao
6a62ba165f Simplify log direcotries 2020-11-24 14:33:16 +08:00
Kent Yao
f8ef7c8997 Support ansynchronous exec for Kyuubi Server 2020-11-24 10:55:29 +08:00
Kent Yao
9ca23bcf20 Support ansynchronous exec for Spark SQL engine 2020-11-20 19:52:57 +08:00
Kent Yao
f5900b052f reorder doc 2020-11-19 11:32:29 +08:00
Kent Yao
91d0ce302e add HADOOP_CONF_DIR to classpath 2020-11-19 00:30:25 +08:00
Kent Yao
8843513496 Add full stacktrace if the spark proc crash 2020-11-18 17:57:05 +08:00
Zen
6637c58273 followup [add SparkSQLEngine idle check] fix #251 
Squashed commit of the following:

commit 808ccb738f8774e465d12dfb04a6d37256545e78
Author: Zen <xinjingziranchan@gmail.com>
Date:   Wed Nov 18 14:18:46 2020 +0800

    Fixed duplicate issues with the Jars directory when packaging Kyuubi

commit 91ca0a0f8a5a260b1ae29a31c323f69d37a14bfe
Author: zen <xinjingziranchan@gmail.com>
Date:   Wed Nov 18 14:01:56 2020 +0800

    fix log print and SparkSQLEngine timeoutChecker concurrent issue.

commit 15343d9cd07b48d3909c1085836687a4cc7f05c3
Author: zen <xinjingziranchan@gmail.com>
Date:   Tue Nov 17 21:21:03 2020 +0800

    消除打印dehug日志时，重复判断是否开启debug模式

commit eecc0bf01144b3f2601e6a821bedf786e5aafb22
Author: zen <xinjingziranchan@gmail.com>
Date:   Tue Nov 17 15:08:13 2020 +0800

    Revert "修复打包时kyuubi依赖jars目录重复，导致启动时找不到org.apache.kyuubi.server.KyuubiServer启动类"

    This reverts commit 29e9dd45

commit 29e9dd45e90b1549b68e96024b1ee866429bf40d
Author: zen <xinjingziranchan@gmail.com>
Date:   Tue Nov 17 14:36:23 2020 +0800

    修复打包时kyuubi依赖jars目录重复，导致启动时找不到org.apache.kyuubi.server.KyuubiServer启动类

commit 44364ecd289ca40181bd150998cce76762b145be
Author: zen <xinjingziranchan@gmail.com>
Date:   Tue Nov 17 13:48:21 2020 +0800

    修复SparkSQLEngine停止时没有关闭timeoutChecker线程池，以及增加timeoutChecker打印日志

commit 63a67e2c487a5418a056ff4f797b46185ed4c17b
Merge: 6028f55 64b83a4
Author: zen <xinjingziranchan@gmail.com>
Date:   Tue Nov 17 13:37:41 2020 +0800

    Merge branch 'yaooqinn-kyuubi-master'

commit 6028f55504cbc3d522bf061fde71608019243d3e
Author: 金晶(000538) <jinjing@fcbox.com>
Date:   Mon Nov 16 16:26:09 2020 +0800

    测试代码SparkOperationSuite格式调整

commit 0b14c69183691e3601aef3a9d56c545edef44499
Author: 金晶(000538) <jinjing@fcbox.com>
Date:   Mon Nov 16 14:49:42 2020 +0800

    1.新增SparkSQLEngine空闲定时检测任务
    2.修复SparkSQLEngine关闭session时未移除SparkSQLOperationManager中缓存的sparkSession

commit 8fdb6b7cec70f32dc05bfeeedcc233d0f6fb4017
Merge: fb4bace 7bfc470
Author: zen <xinjingziranchan@gmail.com>
Date:   Mon Nov 16 11:36:35 2020 +0800

    Merge pull request #1 from yaooqinn/master

    同步更新
 2020-11-18 16:41:43 +08:00
Kent Yao
ea1431b162 [DOC] Integration w/ Hive Metastore 2020-11-18 16:37:21 +08:00
Kent Yao
64b83a48c5 Show possible error even timeout 2020-11-16 22:56:25 +08:00
Kent Yao
9bd4139453 Remove Kyuubi useless deps 2020-11-16 20:52:45 +08:00
Kent Yao
91f622f94f Remove Kyuubi 0.8 and before 2020-11-16 20:12:08 +08:00
Kent Yao
15820a2b09 fix 2020-11-16 19:42:06 +08:00
zen
7fd225b098 add SparkSQLEngine idle check 
fix #250
Squashed commit of the following:

commit 6028f55504cbc3d522bf061fde71608019243d3e
Author: 金晶(000538) <jinjing@fcbox.com>
Date:   Mon Nov 16 16:26:09 2020 +0800

    测试代码SparkOperationSuite格式调整

commit 0b14c69183691e3601aef3a9d56c545edef44499
Author: 金晶(000538) <jinjing@fcbox.com>
Date:   Mon Nov 16 14:49:42 2020 +0800

    1.新增SparkSQLEngine空闲定时检测任务
    2.修复SparkSQLEngine关闭session时未移除SparkSQLOperationManager中缓存的sparkSession

commit 8fdb6b7cec70f32dc05bfeeedcc233d0f6fb4017
Merge: fb4bace 7bfc470
Author: zen <xinjingziranchan@gmail.com>
Date:   Mon Nov 16 11:36:35 2020 +0800

    Merge pull request #1 from yaooqinn/master

    同步更新
 2020-11-16 19:28:33 +08:00
Kent Yao
7bfc4703f3 add github action for release last 2020-11-13 18:51:17 +08:00
Kent Yao
4b0e0ee1fe add github action for release 2020-11-13 17:39:55 +08:00
Kent Yao
99abbe4389 add github action for release 2020-11-13 17:37:46 +08:00
Kent Yao
218dd045db add github action for release 2020-11-13 17:32:36 +08:00
Kent Yao
b3d996917d fix static and core conf via hiveconf or hivevar 2020-11-13 16:08:47 +08:00
Kent Yao
a69fb2553d nit 2020-11-13 14:48:05 +08:00
Kent Yao
d58484c4ec pics 2020-11-13 14:18:45 +08:00
Kent Yao
4396e59abe Add doc for overview 2020-11-13 14:17:42 +08:00
Kent Yao
fb4bace6a5 Ignore static and other unmodifiable settting in session 2020-11-11 11:15:09 +08:00
Kent Yao
574c119712 JDBCTests added and some related bugs fixed 2020-11-10 21:24:02 +08:00
Kent Yao
a67d485951 Missing version in readMessageBegin, old client 
2020-11-10 10:15:42,001 [61208289] - ERROR [FrontendServiceHandler-Pool: Thread-449:TThreadPoolServer$WorkerProcess@315] - Thrift error occurred during processing of message.
org.apache.thrift.protocol.TProtocolException: Missing version in readMessageBegin, old client?
	at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:234)
	at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:27)
	at org.apache.kyuubi.service.authentication.TSetIpAddressProcessor.process(TSetIpAddressProcessor.scala:36)
	at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:310)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
 2020-11-10 17:21:17 +08:00
Kent Yao
f8dc5a9ca3 Using ha.zookeeper.acl.enabled to control ZK acl 2020-11-09 17:34:55 +08:00
Kent Yao
c8bdb5649d Fix NullPointerException in org.apache.kyuubi.session.KyuubiSessionImpl.close 2020-11-09 14:57:17 +08:00
Kent Yao
56ead65496 Add Trouble Shooting with a HADOOP_CONF_DIR case 2020-11-06 19:40:23 +08:00
Kent Yao
b0bb38fa9d Improve OpenSession error Msg 2020-11-06 19:00:00 +08:00
Kent Yao
92e3d6d443 FIX org.apache.kyuubi.config.KyuubiConf#toSparkPrefixedConf 
/**
   * This method is used to convert kyuubi configs to configs that Spark could identify.
   * - If the key is start with `spark.`, keep it AS IS as it is a Spark Conf
   * - If the key is start with `hadoop.`, it will be prefixed with `spark.hadoop.`
   * - Otherwise, the key will be added a `spark.` prefix
   * @return a map with spark specified configs
   */
 2020-11-06 14:37:40 +08:00
Kent Yao
7e1f6ef7ec Add Trouble Shooting with a UnsupportedClassVersionError case 2020-11-06 11:40:25 +08:00
Kent Yao
39e5da584d add gitter 2020-11-05 20:49:22 +08:00
Kent Yao
7157370ef6 Add Building Doc 2020-11-05 19:05:26 +08:00
Kent Yao
bd319f4216 remove most tests in kyuubi-server module 2020-11-05 14:14:11 +08:00
Kent Yao
cdb46e90db Add Configurations Guide 2020-11-04 22:40:02 +08:00
Kent Yao
c888a5b6e0 add logo 2 2020-11-03 21:07:10 +08:00
Kent Yao
2331d9c490 add logo 2 2020-11-03 21:07:04 +08:00