[KYUUBI #1002] Add classification of SQL_Type: this PR is just for DDL

Add classification of SQL_Type: this PR is just for DDL Now we use the simple name to judge this operation belong to which classification of SQL_Type. The matching rule store in #{sql-classification-default.json}. The key is the simpleClassName of this analyzed logical plan, the value is the classification of this sql. It will store in the #{sparksession.conf}, the configuration item is #{spark.sql.classification}. Why we use the file to store the whole rules? If we need to change the matching rule, we only need to revise this file. When the kyuubi started, it will load this json file only once. When user execute a sql, it will generate a logicalPlan and get its simple name. By the rules, the function will return the classification that belong to this sql, then it will set this value into #{sparksession.conf}. For this function, I use SparkSessionExtension to develop it. ### _Why are the changes needed?_ By these SQL_Type, we can do something below: 1. Reduce some log print: For Auxiliary Statements(the sql type is others), we don't need to print operation log. 2. Optimizing some configuration item. For example, in final stage, the conf is different between DML and DQL. Through this configuration item, we can use different conf for them. ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1002 from zhang1002/branch-1.3_get-sql-type-for-ddl. Closes #1002 55ef6af7 [张宇翔] some modification f1f8b355 [张宇翔] change other to undefined 1052ae45 [张宇翔] Change some code standards 5e21dc62 [张宇翔] Change some code standards 3017b96c [张宇翔] Merge branch 'master' into branch-1.3_get-sql-type-for-ddl c55652fd [张宇翔] Merge remote-tracking branch 'upstream/master' c2572b98 [张宇翔] 1. Use RuleBuilder to develop this function 2. Use analyzed logical plan to judge this sql's classification 3. Change the matching rule: use map, the key is simpleClassName, the value is classification of this sql 93b5624a [张宇翔] Exclude license check for json d8187ced [张宇翔] Exclude license check for json e46bc86e [张宇翔] Add exception 3b358bf0 [张宇翔] Add licence 1125b600 [张宇翔] Merge branch 'master' into branch-1.3_get-sql-type-for-ddl ef5e8c55 [张宇翔] Merge remote-tracking branch 'upstream/master' ba8f99eb [张宇翔] Use extension to get simpleName c0bdea7b [张宇翔] Merge branch 'master' into branch-1.3_get-sql-type-for-ddl 5a75384c [张宇翔] Merge remote-tracking branch 'upstream/master' 55b85849 [张宇翔] Update settings.md ecbd8000 [张宇翔] Repair the scalastyle violations. 76edd20d [张宇翔] Repair the scalastyle violations. d8e820ee [张宇翔] Merge branch 'master' into branch-1.3_get-sql-type-for-ddl 8da4f7ed [张宇翔] Merge remote-tracking branch 'upstream/master' 65a90958 [张宇翔] Classification for sqlType: DDL a7ba1bfc [张宇翔] Merge remote-tracking branch 'upstream/master' b662a989 [张宇翔] Merge remote-tracking branch 'upstream/master' 4c8f3b87 [张宇翔] Merge remote-tracking branch 'upstream/master' 8b686767 [张宇翔] Merge remote-tracking branch 'upstream/master' cf99e309 [张宇翔] Merge remote-tracking branch 'upstream/master' 0afaa578 [张宇翔] Merge remote-tracking branch 'upstream/master' b24fea07 [张宇翔] Merge remote-tracking branch 'upstream/master' e517cfc5 [张宇翔] Merge remote-tracking branch 'upstream/master' 18aebe76 [张宇翔] Merge remote-tracking branch 'upstream/master' f248bef7 [张宇翔] Merge remote-tracking branch 'upstream/master' 5ffb54f3 [张宇翔] Add kyuubi-spark-monitor module for nightly.yml Authored-by: 张宇翔 <zhang1002@126.com> Signed-off-by: Kent Yao <yao@apache.org>
2021-09-02 15:31:50 +08:00 · 2021-09-02 15:31:50 +08:00 · 7b289ae6ff
commit 7b289ae6ff
parent d375eb0cfc
8 changed files with 419 additions and 0 deletions
--- a/dev/kyuubi-extension-spark-3-1/src/main/resources/sql-classification-default.json
+++ b/dev/kyuubi-extension-spark-3-1/src/main/resources/sql-classification-default.json
@ -0,0 +1,30 @@
+{
+  "AlterDatabasePropertiesCommand": "ddl",
+  "AlterTableRenameCommand": "ddl",
+  "AlterTableRenamePartitionCommand": "ddl",
+  "AlterTableAddColumnsCommand": "ddl",
+  "AlterTableChangeColumnCommand": "ddl",
+  "AlterTableAddPartitionCommand": "ddl",
+  "AlterTableDropPartitionCommand": "ddl",
+  "AlterTableSetPropertiesCommand": "ddl",
+  "AlterTableSetLocationCommand": "ddl",
+  "AlterTableSerDePropertiesCommand": "ddl",
+  "AlterTableUnsetPropertiesCommand": "ddl",
+  "AlterTableRenameCommand": "ddl",
+  "AlterTableSetPropertiesCommand": "ddl",
+  "AlterTableUnsetPropertiesCommand": "ddl",
+  "AlterViewAsCommand": "ddl",
+  "CreateDatabaseCommand": "ddl",
+  "CreateFunctionCommand": "ddl",
+  "CreateDataSourceTableCommand": "ddl",
+  "CreateDataSourceTableAsSelectCommand": "ddl",
+  "CreateTableLikeCommand": "ddl",
+  "CreateViewCommand": "ddl",
+  "DropDatabaseCommand": "ddl",
+  "DropFunctionCommand": "ddl",
+  "NoopDropTable": "ddl",
+  "DropTableCommand": "ddl",
+  "TruncateTableCommand": "ddl",
+  "AlterTableRecoverPartitionsCommand": "ddl",
+  "SetCatalogAndNamespace": "ddl"
+}
--- a/dev/kyuubi-extension-spark-3-1/src/main/scala/org/apache/kyuubi/sql/KyuubiSQLConf.scala
+++ b/dev/kyuubi-extension-spark-3-1/src/main/scala/org/apache/kyuubi/sql/KyuubiSQLConf.scala
@ -67,4 +67,13 @@ object KyuubiSQLConf {
      .version("1.2.0")
      .booleanConf
      .createWithDefault(false)
+
+  val SQL_CLASSIFICATION_ENABLED =
+    buildConf("kyuubi.spark.sql.classification.enabled")
+      .doc("When true, allows Kyuubi engine to judge this SQL's classification " +
+        "and set it into sessionConf. " +
+        "Through this configuration item, Spark can optimizing configuration dynamic")
+      .version("1.4.0")
+      .booleanConf
+      .createWithDefault(true)
 }
--- a/dev/kyuubi-extension-spark-3-1/src/main/scala/org/apache/kyuubi/sql/KyuubiSparkSQLExtension.scala
+++ b/dev/kyuubi-extension-spark-3-1/src/main/scala/org/apache/kyuubi/sql/KyuubiSparkSQLExtension.scala
@ -19,6 +19,8 @@ package org.apache.kyuubi.sql

 import org.apache.spark.sql.SparkSessionExtensions

+import org.apache.kyuubi.sql.sqlclassification.KyuubiSqlClassification
+
 // scalastyle:off line.size.limit
 /**
 * Depend on Spark SQL Extension framework, we can use this extension follow steps
@ -28,6 +30,7 @@ import org.apache.spark.sql.SparkSessionExtensions
 // scalastyle:on line.size.limit
 class KyuubiSparkSQLExtension extends (SparkSessionExtensions => Unit) {
  override def apply(extensions: SparkSessionExtensions): Unit = {
+    extensions.injectPostHocResolutionRule(KyuubiSqlClassification)
    extensions.injectPostHocResolutionRule(RepartitionBeforeWrite)
    extensions.injectPostHocResolutionRule(RepartitionBeforeWriteHive)
    extensions.injectPostHocResolutionRule(FinalStageConfigIsolationCleanRule)
--- a/dev/kyuubi-extension-spark-3-1/src/main/scala/org/apache/kyuubi/sql/sqlclassification/KyuubiGetSqlClassification.scala
+++ b/dev/kyuubi-extension-spark-3-1/src/main/scala/org/apache/kyuubi/sql/sqlclassification/KyuubiGetSqlClassification.scala
@ -0,0 +1,72 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kyuubi.sql.sqlclassification
+
+import java.io.File
+
+import com.fasterxml.jackson.databind.{JsonNode, ObjectMapper}
+import org.apache.spark.sql.internal.SQLConf
+
+import org.apache.kyuubi.sql.KyuubiSQLConf._
+
+/**
+ * This object is used for getting sql_classification by the logicalPlan's simpleName.
+ * When the configuration item: SQL_CLASSIFICATION_ENABLED is on,
+ * we will load the rule from sql-classification-default.json.
+ */
+object KyuubiGetSqlClassification {
+
+  private val jsonNode: Option[JsonNode] = {
+    SQLConf.get.getConf(SQL_CLASSIFICATION_ENABLED) match {
+      case true =>
+        try {
+          val defaultSqlClassificationFile =
+            Thread.currentThread().getContextClassLoader
+              .getResource("sql-classification-default.json").getPath
+          val objectMapper = new ObjectMapper
+          Some(objectMapper.readTree(new File(defaultSqlClassificationFile)))
+        } catch {
+          case e: Exception =>
+            throw new IllegalArgumentException("sql-classification-default.json is not exist.", e)
+        }
+      case false =>
+        None
+    }
+  }
+
+  /**
+   * Notice:
+   *    You need to make sure that the configuration item: kyuubi.spark.sql.classification.enabled
+    *   is true
+   * @param simpleName: the analyzied_logical_plan's getSimpleName
+   * @return: This sql's classification
+   */
+  def getSqlClassification(simpleName: String): String = {
+    jsonNode.map { json =>
+      val sqlClassififation = json.get(simpleName)
+      if (sqlClassififation == null) {
+        "undefined"
+      } else {
+        sqlClassififation.asText()
+      }
+    }.getOrElse(
+      throw new IllegalArgumentException(
+        "The configuration item: kyuubi.spark.sql.classification.enabled is false")
+    )
+  }
+}
--- a/dev/kyuubi-extension-spark-3-1/src/main/scala/org/apache/kyuubi/sql/sqlclassification/KyuubiSqlClassification.scala
+++ b/dev/kyuubi-extension-spark-3-1/src/main/scala/org/apache/kyuubi/sql/sqlclassification/KyuubiSqlClassification.scala
@ -0,0 +1,36 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kyuubi.sql.sqlclassification
+
+import org.apache.spark.sql.SparkSession
+import org.apache.spark.sql.catalyst.plans.logical.LogicalPlan
+import org.apache.spark.sql.catalyst.rules.Rule
+
+import org.apache.kyuubi.sql.KyuubiSQLConf._
+
+case class KyuubiSqlClassification(session: SparkSession) extends Rule[LogicalPlan] {
+
+  override def apply(plan: LogicalPlan): LogicalPlan = {
+    if (conf.getConf(SQL_CLASSIFICATION_ENABLED) && plan.resolved) {
+      val simpleName = plan.getClass.getSimpleName
+      val sqlClassification = KyuubiGetSqlClassification.getSqlClassification(simpleName)
+      session.conf.set("kyuubi.spark.sql.classification", sqlClassification)
+    }
+    plan
+  }
+}
--- a/dev/kyuubi-extension-spark-3-1/src/test/resources/sql-classification-default.json
+++ b/dev/kyuubi-extension-spark-3-1/src/test/resources/sql-classification-default.json
@ -0,0 +1,30 @@
+{
+  "AlterDatabasePropertiesCommand": "ddl",
+  "AlterTableRenameCommand": "ddl",
+  "AlterTableRenamePartitionCommand": "ddl",
+  "AlterTableAddColumnsCommand": "ddl",
+  "AlterTableChangeColumnCommand": "ddl",
+  "AlterTableAddPartitionCommand": "ddl",
+  "AlterTableDropPartitionCommand": "ddl",
+  "AlterTableSetPropertiesCommand": "ddl",
+  "AlterTableSetLocationCommand": "ddl",
+  "AlterTableSerDePropertiesCommand": "ddl",
+  "AlterTableUnsetPropertiesCommand": "ddl",
+  "AlterTableRenameCommand": "ddl",
+  "AlterTableSetPropertiesCommand": "ddl",
+  "AlterTableUnsetPropertiesCommand": "ddl",
+  "AlterViewAsCommand": "ddl",
+  "CreateDatabaseCommand": "ddl",
+  "CreateFunctionCommand": "ddl",
+  "CreateDataSourceTableCommand": "ddl",
+  "CreateDataSourceTableAsSelectCommand": "ddl",
+  "CreateTableLikeCommand": "ddl",
+  "CreateViewCommand": "ddl",
+  "DropDatabaseCommand": "ddl",
+  "DropFunctionCommand": "ddl",
+  "NoopDropTable": "ddl",
+  "DropTableCommand": "ddl",
+  "TruncateTableCommand": "ddl",
+  "AlterTableRecoverPartitionsCommand": "ddl",
+  "SetCatalogAndNamespace": "ddl"
+}
--- a/dev/kyuubi-extension-spark-3-1/src/test/scala/org/apache/spark/sql/KyuubiExtensionSuite.scala
+++ b/dev/kyuubi-extension-spark-3-1/src/test/scala/org/apache/spark/sql/KyuubiExtensionSuite.scala
@ -418,4 +418,242 @@ class KyuubiExtensionSuite extends QueryTest with SQLTestUtils with AdaptiveSpar
      }
    }
  }
+
+  test("get simple name for DDL") {
+
+    import scala.collection.mutable.Set
+
+    val ddlSimpleName: Set[String] = Set()
+
+    // Notice: When we get Analyzed Logical Plan, the field of LogicalPlan.analyzed.analyzed is true
+
+    // ALTER DATABASE
+    val sql = "CREATE DATABASE inventory;"
+    ddlSimpleName.add(
+      spark.sessionState.analyzer.execute(
+        spark.sessionState.sqlParser.parsePlan(sql)
+      ).getClass.getSimpleName
+    )
+    val sql02 = "ALTER DATABASE inventory SET DBPROPERTIES " +
+      "('Edited-by' = 'John', 'Edit-date' = '01/01/2001');"
+    ddlSimpleName.add(
+      spark.sessionState.analyzer.execute(
+        spark.sessionState.sqlParser.parsePlan(sql02)
+      ).getClass.getSimpleName
+    )
+
+    // ALTER TABLE RENAME
+    val sql03 = "CREATE TABLE student (name VARCHAR(64), rollno INT, age INT) " +
+      "USING PARQUET PARTITIONED BY (age);"
+    ddlSimpleName.add(
+      spark.sessionState.analyzer.execute(
+        spark.sessionState.sqlParser.parsePlan(sql03)
+      ).getClass.getSimpleName
+    )
+    val sql04 = "INSERT INTO student VALUES " +
+      "('zhang', 1, 10),('yu', 2, 11),('xiang', 3, 12),('zhangyuxiang', 4, 17);"
+    ddlSimpleName.add(
+      spark.sessionState.analyzer.execute(
+        spark.sessionState.sqlParser.parsePlan(sql04)
+      ).getClass.getSimpleName
+    )
+    val sql05 = "ALTER TABLE Student RENAME TO StudentInfo;"
+    ddlSimpleName.add(
+      spark.sessionState.analyzer.execute(
+        spark.sessionState.sqlParser.parsePlan(sql05)
+      ).getClass.getSimpleName
+    )
+
+    // ALTER TABLE RENAME PARTITION
+    val sql06 = "ALTER TABLE default.StudentInfo PARTITION (age='10') " +
+      "RENAME TO PARTITION (age='15');"
+    ddlSimpleName.add(
+      spark.sessionState.analyzer.execute(
+        spark.sessionState.sqlParser.parsePlan(sql06)
+      ).getClass.getSimpleName
+    )
+
+    var pre_sql = "CREATE TABLE IF NOT EXISTS StudentInfo " +
+      "(name VARCHAR(64), rollno INT, age INT) USING PARQUET PARTITIONED BY (age);"
+    spark.sql(pre_sql)
+    // ALTER TABLE ADD COLUMNS
+    val sql07 = "ALTER TABLE StudentInfo ADD columns (LastName string, DOB timestamp);"
+    ddlSimpleName.add(
+      spark.sessionState.analyzer.execute(
+        spark.sessionState.sqlParser.parsePlan(sql07)
+      ).getClass.getSimpleName
+    )
+
+    // ALTER TABLE ALTER COLUMN
+    val sql08 = "ALTER TABLE StudentInfo ALTER COLUMN age COMMENT \"new comment\";"
+    ddlSimpleName.add(
+      spark.sessionState.analyzer.execute(
+        spark.sessionState.sqlParser.parsePlan(sql08)
+      ).getClass.getSimpleName
+    )
+
+    // ALTER TABLE CHANGE COLUMN
+    val sql09 = "ALTER TABLE StudentInfo CHANGE COLUMN age COMMENT \"new comment123\";"
+    ddlSimpleName.add(
+      spark.sessionState.analyzer.execute(
+        spark.sessionState.sqlParser.parsePlan(sql09)
+      ).getClass.getSimpleName
+    )
+
+    // ALTER TABLE ADD PARTITION
+    val sql10 = "ALTER TABLE StudentInfo ADD IF NOT EXISTS PARTITION (age=18);"
+    ddlSimpleName.add(
+      spark.sessionState.analyzer.execute(
+        spark.sessionState.sqlParser.parsePlan(sql10)
+      ).getClass.getSimpleName
+    )
+
+    // ALTER TABLE DROP PARTITION
+    val sql11 = "ALTER TABLE StudentInfo DROP IF EXISTS PARTITION (age=18);"
+    ddlSimpleName.add(
+      spark.sessionState.analyzer.execute(
+        spark.sessionState.sqlParser.parsePlan(sql11)
+      ).getClass.getSimpleName
+    )
+
+    // CREAT VIEW
+    val sql12 = "CREATE OR REPLACE VIEW studentinfo_view " +
+      "AS SELECT name, rollno FROM studentinfo;"
+    ddlSimpleName.add(
+      spark.sessionState.analyzer.execute(
+        spark.sessionState.sqlParser.parsePlan(sql12)
+      ).getClass.getSimpleName
+    )
+
+    // ALTER VIEW RENAME TO
+    val sql13 = "ALTER VIEW studentinfo_view RENAME TO studentinfo_view2;"
+    ddlSimpleName.add(
+      spark.sessionState.analyzer.execute(
+        spark.sessionState.sqlParser.parsePlan(sql13)
+      ).getClass.getSimpleName
+    )
+
+    // ALTER VIEW SET TBLPROPERTIES
+    val sql14 = "ALTER VIEW studentinfo_view2 SET TBLPROPERTIES " +
+      "('created.by.user' = \"zhangyuxiang\", 'created.date' = '08-20-2021' );"
+    ddlSimpleName.add(
+      spark.sessionState.analyzer.execute(
+        spark.sessionState.sqlParser.parsePlan(sql14)
+      ).getClass.getSimpleName
+    )
+
+    // ALTER VIEW UNSET TBLPROPERTIES
+    val sql15 = "ALTER VIEW studentinfo_view2 UNSET TBLPROPERTIES " +
+      "('created.by.user', 'created.date');"
+    ddlSimpleName.add(
+      spark.sessionState.analyzer.execute(
+        spark.sessionState.sqlParser.parsePlan(sql15)
+      ).getClass.getSimpleName
+    )
+
+    // ALTER VIEW AS SELECT
+    val sql16 = "ALTER VIEW studentinfo_view2 AS SELECT * FROM studentinfo;"
+    ddlSimpleName.add(
+      spark.sessionState.analyzer.execute(
+        spark.sessionState.sqlParser.parsePlan(sql16)
+      ).getClass.getSimpleName
+    )
+
+    // CREATE DATASOURCE TABLE AS SELECT
+    val sql17 = "CREATE TABLE student_copy USING CSV AS SELECT * FROM studentinfo;"
+    ddlSimpleName.add(
+      spark.sessionState.analyzer.execute(
+        spark.sessionState.sqlParser.parsePlan(sql17)
+      ).getClass.getSimpleName
+    )
+
+    // CREATE DATASOURCE TABLE AS LIKE
+    val sql18 = "CREATE TABLE Student_Dupli like studentinfo;"
+    ddlSimpleName.add(
+      spark.sessionState.analyzer.execute(
+        spark.sessionState.sqlParser.parsePlan(sql18)
+      ).getClass.getSimpleName
+    )
+
+    // USE DATABASE
+    val sql26 = "USE default;"
+    ddlSimpleName.add(
+      spark.sessionState.analyzer.execute(
+        spark.sessionState.sqlParser.parsePlan(sql26)
+      ).getClass.getSimpleName
+    )
+
+    // DROP DATABASE
+    val sql19 = "DROP DATABASE IF EXISTS inventory_db CASCADE;"
+    ddlSimpleName.add(
+      spark.sessionState.analyzer.execute(
+        spark.sessionState.sqlParser.parsePlan(sql19)
+      ).getClass.getSimpleName
+    )
+
+    // CREATE FUNCTION
+    val sql20 = "CREATE FUNCTION test_avg AS " +
+      "'org.apache.hadoop.hive.ql.udf.generic.GenericUDAFAverage';"
+    ddlSimpleName.add(
+      spark.sessionState.analyzer.execute(
+        spark.sessionState.sqlParser.parsePlan(sql20)
+      ).getClass.getSimpleName
+    )
+
+    // DROP FUNCTION
+    val sql21 = "DROP FUNCTION test_avg;"
+    ddlSimpleName.add(
+      spark.sessionState.analyzer.execute(
+        spark.sessionState.sqlParser.parsePlan(sql21)
+      ).getClass.getSimpleName
+    )
+
+    spark.sql("CREATE TABLE IF NOT EXISTS studentabc (name VARCHAR(64), rollno INT, age INT) " +
+      "USING PARQUET PARTITIONED BY (age);")
+    // DROP TABLE
+    val sql22 = "DROP TABLE IF EXISTS studentabc;"
+    ddlSimpleName.add(
+      spark.sessionState.analyzer.execute(
+        spark.sessionState.sqlParser.parsePlan(sql22)
+      ).getClass.getSimpleName
+    )
+
+    // DROP VIEW
+    val sql23 = "DROP VIEW studentinfo_view2;"
+    ddlSimpleName.add(
+      spark.sessionState.analyzer.execute(
+        spark.sessionState.sqlParser.parsePlan(sql23)
+      ).getClass.getSimpleName
+    )
+
+    // TRUNCATE TABLE
+    val sql24 = "TRUNCATE TABLE StudentInfo;"
+    ddlSimpleName.add(
+      spark.sessionState.analyzer.execute(
+        spark.sessionState.sqlParser.parsePlan(sql24)
+      ).getClass.getSimpleName
+    )
+
+    // REPAIR TABLE
+    val sql25 = "MSCK REPAIR TABLE StudentInfo;"
+    ddlSimpleName.add(
+      spark.sessionState.analyzer.execute(
+        spark.sessionState.sqlParser.parsePlan(sql25)
+      ).getClass.getSimpleName
+    )
+    // scalastyle:off println
+    println("ddl simple name is :" + ddlSimpleName)
+    // scalastyle:on println
+  }
+
+  test("Sql classification for ddl") {
+    withSQLConf(KyuubiSQLConf.SQL_CLASSIFICATION_ENABLED.key -> "true") {
+      withDatabase("inventory") {
+        val df = sql("CREATE DATABASE inventory;")
+        assert(df.sparkSession.conf.get("kyuubi.spark.sql.classification") === "ddl")
+      }
+      val df = sql("select timestamp'2021-06-01'")
+      assert(df.sparkSession.conf.get("kyuubi.spark.sql.classification") !== "ddl")
+    }
+  }
 }
--- a/pom.xml
+++ b/pom.xml
@ -1501,6 +1501,7 @@
                        </licenseFamilies>
                        <excludes>
                            <exclude>**/.*/**</exclude>
+                            <exclude>**/*.json</exclude>
                            <exclude>**/*.prefs</exclude>
                            <exclude>**/*.log</exclude>
                            <exclude>**/*.md</exclude>