[KYUUBI #5660] [TEST] Ensure SPI provider file content sorted in alphabet order
### _Why are the changes needed?_ Check the content of the SPI provider files sorted in alphabet order. ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request ### _Was this patch authored or co-authored using generative AI tooling?_ No. Closes #5660 from bowenliang123/spi-sort. Closes #5660 30c7028f6 [Bowen Liang] ensure SPI provider file sorted Authored-by: Bowen Liang <liangbowen@gf.com.cn> Signed-off-by: Bowen Liang <liangbowen@gf.com.cn>
This commit is contained in:
parent
e8e981d4ab
commit
bad0ccb5ed
@ -6,7 +6,7 @@
|
|||||||
# (the "License"); you may not use this file except in compliance with
|
# (the "License"); you may not use this file except in compliance with
|
||||||
# the License. You may obtain a copy of the License at
|
# the License. You may obtain a copy of the License at
|
||||||
#
|
#
|
||||||
# http://www.apache.org/licenses/LICENSE-2.0
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
#
|
#
|
||||||
# Unless required by applicable law or agreed to in writing, software
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
|||||||
@ -6,7 +6,7 @@
|
|||||||
# (the "License"); you may not use this file except in compliance with
|
# (the "License"); you may not use this file except in compliance with
|
||||||
# the License. You may obtain a copy of the License at
|
# the License. You may obtain a copy of the License at
|
||||||
#
|
#
|
||||||
# http://www.apache.org/licenses/LICENSE-2.0
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
#
|
#
|
||||||
# Unless required by applicable law or agreed to in writing, software
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
|||||||
@ -6,7 +6,7 @@
|
|||||||
# (the "License"); you may not use this file except in compliance with
|
# (the "License"); you may not use this file except in compliance with
|
||||||
# the License. You may obtain a copy of the License at
|
# the License. You may obtain a copy of the License at
|
||||||
#
|
#
|
||||||
# http://www.apache.org/licenses/LICENSE-2.0
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
#
|
#
|
||||||
# Unless required by applicable law or agreed to in writing, software
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
|||||||
@ -6,7 +6,7 @@
|
|||||||
# (the "License"); you may not use this file except in compliance with
|
# (the "License"); you may not use this file except in compliance with
|
||||||
# the License. You may obtain a copy of the License at
|
# the License. You may obtain a copy of the License at
|
||||||
#
|
#
|
||||||
# http://www.apache.org/licenses/LICENSE-2.0
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
#
|
#
|
||||||
# Unless required by applicable law or agreed to in writing, software
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
|||||||
@ -6,7 +6,7 @@
|
|||||||
# (the "License"); you may not use this file except in compliance with
|
# (the "License"); you may not use this file except in compliance with
|
||||||
# the License. You may obtain a copy of the License at
|
# the License. You may obtain a copy of the License at
|
||||||
#
|
#
|
||||||
# http://www.apache.org/licenses/LICENSE-2.0
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
#
|
#
|
||||||
# Unless required by applicable law or agreed to in writing, software
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
|||||||
@ -6,7 +6,7 @@
|
|||||||
# (the "License"); you may not use this file except in compliance with
|
# (the "License"); you may not use this file except in compliance with
|
||||||
# the License. You may obtain a copy of the License at
|
# the License. You may obtain a copy of the License at
|
||||||
#
|
#
|
||||||
# http://www.apache.org/licenses/LICENSE-2.0
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
#
|
#
|
||||||
# Unless required by applicable law or agreed to in writing, software
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
|||||||
@ -6,7 +6,7 @@
|
|||||||
# (the "License"); you may not use this file except in compliance with
|
# (the "License"); you may not use this file except in compliance with
|
||||||
# the License. You may obtain a copy of the License at
|
# the License. You may obtain a copy of the License at
|
||||||
#
|
#
|
||||||
# http://www.apache.org/licenses/LICENSE-2.0
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
#
|
#
|
||||||
# Unless required by applicable law or agreed to in writing, software
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
|||||||
@ -6,7 +6,7 @@
|
|||||||
# (the "License"); you may not use this file except in compliance with
|
# (the "License"); you may not use this file except in compliance with
|
||||||
# the License. You may obtain a copy of the License at
|
# the License. You may obtain a copy of the License at
|
||||||
#
|
#
|
||||||
# http://www.apache.org/licenses/LICENSE-2.0
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
#
|
#
|
||||||
# Unless required by applicable law or agreed to in writing, software
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
@ -19,10 +19,10 @@ org.apache.kyuubi.plugin.spark.authz.serde.CatalogTableOptionTableExtractor
|
|||||||
org.apache.kyuubi.plugin.spark.authz.serde.CatalogTableTableExtractor
|
org.apache.kyuubi.plugin.spark.authz.serde.CatalogTableTableExtractor
|
||||||
org.apache.kyuubi.plugin.spark.authz.serde.DataSourceV2RelationTableExtractor
|
org.apache.kyuubi.plugin.spark.authz.serde.DataSourceV2RelationTableExtractor
|
||||||
org.apache.kyuubi.plugin.spark.authz.serde.ExpressionSeqTableExtractor
|
org.apache.kyuubi.plugin.spark.authz.serde.ExpressionSeqTableExtractor
|
||||||
org.apache.kyuubi.plugin.spark.authz.serde.HudiDataSourceV2RelationTableExtractor
|
|
||||||
org.apache.kyuubi.plugin.spark.authz.serde.HudiMergeIntoTargetTableExtractor
|
|
||||||
org.apache.kyuubi.plugin.spark.authz.serde.HudiCallProcedureInputTableExtractor
|
org.apache.kyuubi.plugin.spark.authz.serde.HudiCallProcedureInputTableExtractor
|
||||||
org.apache.kyuubi.plugin.spark.authz.serde.HudiCallProcedureOutputTableExtractor
|
org.apache.kyuubi.plugin.spark.authz.serde.HudiCallProcedureOutputTableExtractor
|
||||||
|
org.apache.kyuubi.plugin.spark.authz.serde.HudiDataSourceV2RelationTableExtractor
|
||||||
|
org.apache.kyuubi.plugin.spark.authz.serde.HudiMergeIntoTargetTableExtractor
|
||||||
org.apache.kyuubi.plugin.spark.authz.serde.IdentifierTableExtractor
|
org.apache.kyuubi.plugin.spark.authz.serde.IdentifierTableExtractor
|
||||||
org.apache.kyuubi.plugin.spark.authz.serde.LogicalRelationTableExtractor
|
org.apache.kyuubi.plugin.spark.authz.serde.LogicalRelationTableExtractor
|
||||||
org.apache.kyuubi.plugin.spark.authz.serde.ResolvedDbObjectNameTableExtractor
|
org.apache.kyuubi.plugin.spark.authz.serde.ResolvedDbObjectNameTableExtractor
|
||||||
@ -30,6 +30,6 @@ org.apache.kyuubi.plugin.spark.authz.serde.ResolvedIdentifierTableExtractor
|
|||||||
org.apache.kyuubi.plugin.spark.authz.serde.ResolvedTableTableExtractor
|
org.apache.kyuubi.plugin.spark.authz.serde.ResolvedTableTableExtractor
|
||||||
org.apache.kyuubi.plugin.spark.authz.serde.StringTableExtractor
|
org.apache.kyuubi.plugin.spark.authz.serde.StringTableExtractor
|
||||||
org.apache.kyuubi.plugin.spark.authz.serde.SubqueryAliasTableExtractor
|
org.apache.kyuubi.plugin.spark.authz.serde.SubqueryAliasTableExtractor
|
||||||
org.apache.kyuubi.plugin.spark.authz.serde.TableIdentifierTableExtractor
|
|
||||||
org.apache.kyuubi.plugin.spark.authz.serde.TableIdentifierOptionTableExtractor
|
org.apache.kyuubi.plugin.spark.authz.serde.TableIdentifierOptionTableExtractor
|
||||||
|
org.apache.kyuubi.plugin.spark.authz.serde.TableIdentifierTableExtractor
|
||||||
org.apache.kyuubi.plugin.spark.authz.serde.TableTableExtractor
|
org.apache.kyuubi.plugin.spark.authz.serde.TableTableExtractor
|
||||||
|
|||||||
@ -6,7 +6,7 @@
|
|||||||
# (the "License"); you may not use this file except in compliance with
|
# (the "License"); you may not use this file except in compliance with
|
||||||
# the License. You may obtain a copy of the License at
|
# the License. You may obtain a copy of the License at
|
||||||
#
|
#
|
||||||
# http://www.apache.org/licenses/LICENSE-2.0
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
#
|
#
|
||||||
# Unless required by applicable law or agreed to in writing, software
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
|||||||
@ -6,7 +6,7 @@
|
|||||||
# (the "License"); you may not use this file except in compliance with
|
# (the "License"); you may not use this file except in compliance with
|
||||||
# the License. You may obtain a copy of the License at
|
# the License. You may obtain a copy of the License at
|
||||||
#
|
#
|
||||||
# http://www.apache.org/licenses/LICENSE-2.0
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
#
|
#
|
||||||
# Unless required by applicable law or agreed to in writing, software
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
@ -15,10 +15,10 @@
|
|||||||
# limitations under the License.
|
# limitations under the License.
|
||||||
#
|
#
|
||||||
|
|
||||||
org.apache.kyuubi.plugin.spark.authz.serde.CatalogStorageFormatURIExtractor
|
|
||||||
org.apache.kyuubi.plugin.spark.authz.serde.BaseRelationFileIndexURIExtractor
|
org.apache.kyuubi.plugin.spark.authz.serde.BaseRelationFileIndexURIExtractor
|
||||||
|
org.apache.kyuubi.plugin.spark.authz.serde.CatalogStorageFormatURIExtractor
|
||||||
org.apache.kyuubi.plugin.spark.authz.serde.PartitionLocsSeqURIExtractor
|
org.apache.kyuubi.plugin.spark.authz.serde.PartitionLocsSeqURIExtractor
|
||||||
org.apache.kyuubi.plugin.spark.authz.serde.PropertiesPathUriExtractor
|
|
||||||
org.apache.kyuubi.plugin.spark.authz.serde.PropertiesLocationUriExtractor
|
org.apache.kyuubi.plugin.spark.authz.serde.PropertiesLocationUriExtractor
|
||||||
org.apache.kyuubi.plugin.spark.authz.serde.StringURIExtractor
|
org.apache.kyuubi.plugin.spark.authz.serde.PropertiesPathUriExtractor
|
||||||
org.apache.kyuubi.plugin.spark.authz.serde.StringSeqURIExtractor
|
org.apache.kyuubi.plugin.spark.authz.serde.StringSeqURIExtractor
|
||||||
|
org.apache.kyuubi.plugin.spark.authz.serde.StringURIExtractor
|
||||||
|
|||||||
@ -27,9 +27,9 @@ import com.fasterxml.jackson.databind.json.JsonMapper
|
|||||||
import com.fasterxml.jackson.databind.node.ObjectNode
|
import com.fasterxml.jackson.databind.node.ObjectNode
|
||||||
import com.fasterxml.jackson.module.scala.DefaultScalaModule
|
import com.fasterxml.jackson.module.scala.DefaultScalaModule
|
||||||
import org.apache.ranger.plugin.model.RangerPolicy
|
import org.apache.ranger.plugin.model.RangerPolicy
|
||||||
|
// scalastyle:off
|
||||||
import org.scalatest.funsuite.AnyFunSuite
|
import org.scalatest.funsuite.AnyFunSuite
|
||||||
|
|
||||||
// scalastyle:off
|
|
||||||
import org.apache.kyuubi.plugin.spark.authz.RangerTestNamespace._
|
import org.apache.kyuubi.plugin.spark.authz.RangerTestNamespace._
|
||||||
import org.apache.kyuubi.plugin.spark.authz.RangerTestUsers._
|
import org.apache.kyuubi.plugin.spark.authz.RangerTestUsers._
|
||||||
import org.apache.kyuubi.plugin.spark.authz.gen.KRangerPolicyItemAccess.allowTypes
|
import org.apache.kyuubi.plugin.spark.authz.gen.KRangerPolicyItemAccess.allowTypes
|
||||||
@ -37,6 +37,7 @@ import org.apache.kyuubi.plugin.spark.authz.gen.KRangerPolicyResource._
|
|||||||
import org.apache.kyuubi.plugin.spark.authz.gen.RangerAccessType._
|
import org.apache.kyuubi.plugin.spark.authz.gen.RangerAccessType._
|
||||||
import org.apache.kyuubi.plugin.spark.authz.gen.RangerClassConversions._
|
import org.apache.kyuubi.plugin.spark.authz.gen.RangerClassConversions._
|
||||||
import org.apache.kyuubi.util.AssertionUtils._
|
import org.apache.kyuubi.util.AssertionUtils._
|
||||||
|
import org.apache.kyuubi.util.GoldenFileUtils._
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Generates the policy file to test/main/resources dir.
|
* Generates the policy file to test/main/resources dir.
|
||||||
@ -59,11 +60,9 @@ class PolicyJsonFileGenerator extends AnyFunSuite {
|
|||||||
.build()
|
.build()
|
||||||
|
|
||||||
test("check ranger policy file") {
|
test("check ranger policy file") {
|
||||||
val pluginHome = getClass.getProtectionDomain.getCodeSource.getLocation.getPath
|
|
||||||
.split("target").head
|
|
||||||
val policyFileName = "sparkSql_hive_jenkins.json"
|
val policyFileName = "sparkSql_hive_jenkins.json"
|
||||||
val policyFilePath =
|
val policyFilePath = Paths.get(
|
||||||
Paths.get(pluginHome, "src", "test", "resources", policyFileName)
|
s"${getCurrentModuleHome(this)}/src/test/resources/$policyFileName")
|
||||||
val generatedStr = mapper.writerWithDefaultPrettyPrinter()
|
val generatedStr = mapper.writerWithDefaultPrettyPrinter()
|
||||||
.writeValueAsString(servicePolicies)
|
.writeValueAsString(servicePolicies)
|
||||||
|
|
||||||
|
|||||||
@ -0,0 +1,49 @@
|
|||||||
|
/*
|
||||||
|
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||||
|
* contributor license agreements. See the NOTICE file distributed with
|
||||||
|
* this work for additional information regarding copyright ownership.
|
||||||
|
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||||
|
* (the "License"); you may not use this file except in compliance with
|
||||||
|
* the License. You may obtain a copy of the License at
|
||||||
|
*
|
||||||
|
* http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
*
|
||||||
|
* Unless required by applicable law or agreed to in writing, software
|
||||||
|
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
* See the License for the specific language governing permissions and
|
||||||
|
* limitations under the License.
|
||||||
|
*/
|
||||||
|
|
||||||
|
package org.apache.kyuubi.plugin.spark.authz.gen
|
||||||
|
|
||||||
|
import java.nio.file.Paths
|
||||||
|
|
||||||
|
// scalastyle:off
|
||||||
|
import org.scalatest.funsuite.AnyFunSuite
|
||||||
|
|
||||||
|
import org.apache.kyuubi.util.AssertionUtils._
|
||||||
|
import org.apache.kyuubi.util.GoldenFileUtils._
|
||||||
|
|
||||||
|
class CheckAuthzExtractorSPISuite extends AnyFunSuite {
|
||||||
|
// scalastyle:on
|
||||||
|
|
||||||
|
test("check authz extractor SPI service file sorted") {
|
||||||
|
Seq(
|
||||||
|
"org.apache.kyuubi.plugin.spark.authz.serde.ActionTypeExtractor",
|
||||||
|
"org.apache.kyuubi.plugin.spark.authz.serde.CatalogExtractor",
|
||||||
|
"org.apache.kyuubi.plugin.spark.authz.serde.ColumnExtractor",
|
||||||
|
"org.apache.kyuubi.plugin.spark.authz.serde.DatabaseExtractor",
|
||||||
|
"org.apache.kyuubi.plugin.spark.authz.serde.FunctionExtractor",
|
||||||
|
"org.apache.kyuubi.plugin.spark.authz.serde.FunctionTypeExtractor",
|
||||||
|
"org.apache.kyuubi.plugin.spark.authz.serde.QueryExtractor",
|
||||||
|
"org.apache.kyuubi.plugin.spark.authz.serde.TableExtractor",
|
||||||
|
"org.apache.kyuubi.plugin.spark.authz.serde.TableTypeExtractor",
|
||||||
|
"org.apache.kyuubi.plugin.spark.authz.serde.URIExtractor")
|
||||||
|
.foreach { fileName =>
|
||||||
|
val filePath = Paths.get(
|
||||||
|
s"${getCurrentModuleHome(this)}/src/main/resources/META-INF/services/$fileName")
|
||||||
|
assertFileContentSorted(filePath)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@ -26,6 +26,7 @@ import org.scalatest.funsuite.AnyFunSuite
|
|||||||
import org.apache.kyuubi.plugin.spark.authz.serde.{mapper, CommandSpec}
|
import org.apache.kyuubi.plugin.spark.authz.serde.{mapper, CommandSpec}
|
||||||
import org.apache.kyuubi.plugin.spark.authz.serde.CommandSpecs
|
import org.apache.kyuubi.plugin.spark.authz.serde.CommandSpecs
|
||||||
import org.apache.kyuubi.util.AssertionUtils._
|
import org.apache.kyuubi.util.AssertionUtils._
|
||||||
|
import org.apache.kyuubi.util.GoldenFileUtils._
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Generates the default command specs to src/main/resources dir.
|
* Generates the default command specs to src/main/resources dir.
|
||||||
@ -52,10 +53,9 @@ class JsonSpecFileGenerator extends AnyFunSuite {
|
|||||||
def writeCommandSpecJson[T <: CommandSpec](
|
def writeCommandSpecJson[T <: CommandSpec](
|
||||||
commandType: String,
|
commandType: String,
|
||||||
specsArr: Seq[CommandSpecs[T]]): Unit = {
|
specsArr: Seq[CommandSpecs[T]]): Unit = {
|
||||||
val pluginHome = getClass.getProtectionDomain.getCodeSource.getLocation.getPath
|
|
||||||
.split("target").head
|
|
||||||
val filename = s"${commandType}_command_spec.json"
|
val filename = s"${commandType}_command_spec.json"
|
||||||
val filePath = Paths.get(pluginHome, "src", "main", "resources", filename)
|
val filePath = Paths.get(
|
||||||
|
s"${getCurrentModuleHome(this)}/src/main/resources/$filename")
|
||||||
|
|
||||||
val allSpecs = specsArr.flatMap(_.specs.sortBy(_.classname))
|
val allSpecs = specsArr.flatMap(_.specs.sortBy(_.classname))
|
||||||
val duplicatedClassnames = allSpecs.groupBy(_.classname).values
|
val duplicatedClassnames = allSpecs.groupBy(_.classname).values
|
||||||
|
|||||||
@ -6,7 +6,7 @@
|
|||||||
# (the "License"); you may not use this file except in compliance with
|
# (the "License"); you may not use this file except in compliance with
|
||||||
# the License. You may obtain a copy of the License at
|
# the License. You may obtain a copy of the License at
|
||||||
#
|
#
|
||||||
# http://www.apache.org/licenses/LICENSE-2.0
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
#
|
#
|
||||||
# Unless required by applicable law or agreed to in writing, software
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
|||||||
@ -6,7 +6,7 @@
|
|||||||
# (the "License"); you may not use this file except in compliance with
|
# (the "License"); you may not use this file except in compliance with
|
||||||
# the License. You may obtain a copy of the License at
|
# the License. You may obtain a copy of the License at
|
||||||
#
|
#
|
||||||
# http://www.apache.org/licenses/LICENSE-2.0
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
#
|
#
|
||||||
# Unless required by applicable law or agreed to in writing, software
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
|||||||
@ -0,0 +1,41 @@
|
|||||||
|
/*
|
||||||
|
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||||
|
* contributor license agreements. See the NOTICE file distributed with
|
||||||
|
* this work for additional information regarding copyright ownership.
|
||||||
|
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||||
|
* (the "License"); you may not use this file except in compliance with
|
||||||
|
* the License. You may obtain a copy of the License at
|
||||||
|
*
|
||||||
|
* http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
*
|
||||||
|
* Unless required by applicable law or agreed to in writing, software
|
||||||
|
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
* See the License for the specific language governing permissions and
|
||||||
|
* limitations under the License.
|
||||||
|
*/
|
||||||
|
|
||||||
|
package org.apache.kyuubi.engine.jdbc
|
||||||
|
|
||||||
|
import java.nio.file.Paths
|
||||||
|
|
||||||
|
// scalastyle:off
|
||||||
|
import org.scalatest.funsuite.AnyFunSuite
|
||||||
|
|
||||||
|
import org.apache.kyuubi.util.AssertionUtils._
|
||||||
|
import org.apache.kyuubi.util.GoldenFileUtils._
|
||||||
|
|
||||||
|
class CheckJdbcDialectSPISuite extends AnyFunSuite {
|
||||||
|
// scalastyle:on
|
||||||
|
|
||||||
|
test("check JDBC dialect SPI service file sorted") {
|
||||||
|
Seq(
|
||||||
|
"org.apache.kyuubi.engine.jdbc.connection.JdbcConnectionProvider",
|
||||||
|
"org.apache.kyuubi.engine.jdbc.dialect.JdbcDialect")
|
||||||
|
.foreach { fileName =>
|
||||||
|
val filePath = Paths.get(
|
||||||
|
s"${getCurrentModuleHome(this)}/src/main/resources/META-INF/services/$fileName")
|
||||||
|
assertFileContentSorted(filePath)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@ -25,6 +25,8 @@ import com.vladsch.flexmark.profile.pegdown.PegdownOptionsAdapter
|
|||||||
import com.vladsch.flexmark.util.data.{MutableDataHolder, MutableDataSet}
|
import com.vladsch.flexmark.util.data.{MutableDataHolder, MutableDataSet}
|
||||||
import com.vladsch.flexmark.util.sequence.SequenceUtils.EOL
|
import com.vladsch.flexmark.util.sequence.SequenceUtils.EOL
|
||||||
|
|
||||||
|
import org.apache.kyuubi.util.GoldenFileUtils.getLicenceContent
|
||||||
|
|
||||||
class MarkdownBuilder {
|
class MarkdownBuilder {
|
||||||
private val buffer = new ListBuffer[String]
|
private val buffer = new ListBuffer[String]
|
||||||
|
|
||||||
@ -58,24 +60,8 @@ class MarkdownBuilder {
|
|||||||
* @return
|
* @return
|
||||||
*/
|
*/
|
||||||
def licence(): MarkdownBuilder = {
|
def licence(): MarkdownBuilder = {
|
||||||
this ++= """
|
buffer.appendAll(getLicenceContent(header = "<!--", linePrefix = "-", footer = "-->"))
|
||||||
|<!--
|
this
|
||||||
|- Licensed to the Apache Software Foundation (ASF) under one or more
|
|
||||||
|- contributor license agreements. See the NOTICE file distributed with
|
|
||||||
|- this work for additional information regarding copyright ownership.
|
|
||||||
|- The ASF licenses this file to You under the Apache License, Version 2.0
|
|
||||||
|- (the "License"); you may not use this file except in compliance with
|
|
||||||
|- the License. You may obtain a copy of the License at
|
|
||||||
|-
|
|
||||||
|- http://www.apache.org/licenses/LICENSE-2.0
|
|
||||||
|-
|
|
||||||
|- Unless required by applicable law or agreed to in writing, software
|
|
||||||
|- distributed under the License is distributed on an "AS IS" BASIS,
|
|
||||||
|- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
||||||
|- See the License for the specific language governing permissions and
|
|
||||||
|- limitations under the License.
|
|
||||||
|-->
|
|
||||||
|"""
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
|
|||||||
@ -6,7 +6,7 @@
|
|||||||
# (the "License"); you may not use this file except in compliance with
|
# (the "License"); you may not use this file except in compliance with
|
||||||
# the License. You may obtain a copy of the License at
|
# the License. You may obtain a copy of the License at
|
||||||
#
|
#
|
||||||
# http://www.apache.org/licenses/LICENSE-2.0
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
#
|
#
|
||||||
# Unless required by applicable law or agreed to in writing, software
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
|||||||
@ -6,7 +6,7 @@
|
|||||||
# (the "License"); you may not use this file except in compliance with
|
# (the "License"); you may not use this file except in compliance with
|
||||||
# the License. You may obtain a copy of the License at
|
# the License. You may obtain a copy of the License at
|
||||||
#
|
#
|
||||||
# http://www.apache.org/licenses/LICENSE-2.0
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
#
|
#
|
||||||
# Unless required by applicable law or agreed to in writing, software
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
|||||||
@ -6,7 +6,7 @@
|
|||||||
# (the "License"); you may not use this file except in compliance with
|
# (the "License"); you may not use this file except in compliance with
|
||||||
# the License. You may obtain a copy of the License at
|
# the License. You may obtain a copy of the License at
|
||||||
#
|
#
|
||||||
# http://www.apache.org/licenses/LICENSE-2.0
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
#
|
#
|
||||||
# Unless required by applicable law or agreed to in writing, software
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
@ -15,6 +15,6 @@
|
|||||||
# limitations under the License.
|
# limitations under the License.
|
||||||
#
|
#
|
||||||
|
|
||||||
org.apache.kyuubi.engine.YarnApplicationOperation
|
|
||||||
org.apache.kyuubi.engine.JpsApplicationOperation
|
org.apache.kyuubi.engine.JpsApplicationOperation
|
||||||
org.apache.kyuubi.engine.KubernetesApplicationOperation
|
org.apache.kyuubi.engine.KubernetesApplicationOperation
|
||||||
|
org.apache.kyuubi.engine.YarnApplicationOperation
|
||||||
|
|||||||
@ -0,0 +1,42 @@
|
|||||||
|
/*
|
||||||
|
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||||
|
* contributor license agreements. See the NOTICE file distributed with
|
||||||
|
* this work for additional information regarding copyright ownership.
|
||||||
|
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||||
|
* (the "License"); you may not use this file except in compliance with
|
||||||
|
* the License. You may obtain a copy of the License at
|
||||||
|
*
|
||||||
|
* http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
*
|
||||||
|
* Unless required by applicable law or agreed to in writing, software
|
||||||
|
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
* See the License for the specific language governing permissions and
|
||||||
|
* limitations under the License.
|
||||||
|
*/
|
||||||
|
|
||||||
|
package org.apache.kyuubi.service
|
||||||
|
|
||||||
|
import java.nio.file.Paths
|
||||||
|
|
||||||
|
// scalastyle:off
|
||||||
|
import org.scalatest.funsuite.AnyFunSuite
|
||||||
|
|
||||||
|
import org.apache.kyuubi.util.AssertionUtils._
|
||||||
|
import org.apache.kyuubi.util.GoldenFileUtils._
|
||||||
|
|
||||||
|
class CheckServerSPISuite extends AnyFunSuite {
|
||||||
|
// scalastyle:on
|
||||||
|
|
||||||
|
test("check server SPI service file sorted") {
|
||||||
|
Seq(
|
||||||
|
"org.apache.hadoop.security.token.TokenIdentifier",
|
||||||
|
"org.apache.kyuubi.credentials.HadoopDelegationTokenProvider",
|
||||||
|
"org.apache.kyuubi.engine.ApplicationOperation")
|
||||||
|
.foreach { fileName =>
|
||||||
|
val filePath = Paths.get(
|
||||||
|
s"${getCurrentModuleHome(this)}/src/main/resources/META-INF/services/$fileName")
|
||||||
|
assertFileContentSorted(filePath)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@ -17,9 +17,10 @@
|
|||||||
package org.apache.kyuubi.util
|
package org.apache.kyuubi.util
|
||||||
|
|
||||||
import java.nio.charset.StandardCharsets
|
import java.nio.charset.StandardCharsets
|
||||||
import java.nio.file.Path
|
import java.nio.file.{Files, Path}
|
||||||
import java.util.Locale
|
import java.util.Locale
|
||||||
|
|
||||||
|
import scala.collection.JavaConverters._
|
||||||
import scala.collection.Traversable
|
import scala.collection.Traversable
|
||||||
import scala.io.Source
|
import scala.io.Source
|
||||||
import scala.reflect.ClassTag
|
import scala.reflect.ClassTag
|
||||||
@ -29,6 +30,8 @@ import org.scalactic.Prettifier
|
|||||||
import org.scalactic.source.Position
|
import org.scalactic.source.Position
|
||||||
import org.scalatest.Assertions._
|
import org.scalatest.Assertions._
|
||||||
|
|
||||||
|
import org.apache.kyuubi.util.GoldenFileUtils.getLicenceContent
|
||||||
|
|
||||||
object AssertionUtils {
|
object AssertionUtils {
|
||||||
|
|
||||||
def assertEqualsIgnoreCase(expected: AnyRef)(actual: AnyRef)(
|
def assertEqualsIgnoreCase(expected: AnyRef)(actual: AnyRef)(
|
||||||
@ -106,6 +109,24 @@ object AssertionUtils {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
def assertFileContentSorted(
|
||||||
|
filePath: Path,
|
||||||
|
headerSkipPrefix: String = "#",
|
||||||
|
licenceHeader: Iterable[String] = getLicenceContent(),
|
||||||
|
distinct: Boolean = true): Unit = {
|
||||||
|
val sortedLines = Files.readAllLines(filePath).asScala
|
||||||
|
.dropWhile(line => line.trim == "" || line.trim.startsWith(headerSkipPrefix))
|
||||||
|
.map(_.trim).filter(_.nonEmpty)
|
||||||
|
.sorted
|
||||||
|
val expectedSortedLines = if (distinct) {
|
||||||
|
sortedLines.distinct
|
||||||
|
} else {
|
||||||
|
sortedLines
|
||||||
|
}
|
||||||
|
val expectedLines = licenceHeader ++ Seq("") ++ expectedSortedLines
|
||||||
|
assertFileContent(filePath, expectedLines, s"Check SPI provider file sorted $filePath")
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Assert the iterable contains all the expected elements
|
* Assert the iterable contains all the expected elements
|
||||||
*/
|
*/
|
||||||
|
|||||||
@ -48,4 +48,34 @@ object GoldenFileUtils {
|
|||||||
assertFileContent(path, lines, regenScript)
|
assertFileContent(path, lines, regenScript)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
def getCurrentModuleHome(obj: Any): String = {
|
||||||
|
obj.getClass.getProtectionDomain.getCodeSource.getLocation.getPath
|
||||||
|
.split("target").head
|
||||||
|
}
|
||||||
|
|
||||||
|
val apacheLicenceContent: String =
|
||||||
|
""" Licensed to the Apache Software Foundation (ASF) under one or more
|
||||||
|
| contributor license agreements. See the NOTICE file distributed with
|
||||||
|
| this work for additional information regarding copyright ownership.
|
||||||
|
| The ASF licenses this file to You under the Apache License, Version 2.0
|
||||||
|
| (the "License"); you may not use this file except in compliance with
|
||||||
|
| the License. You may obtain a copy of the License at
|
||||||
|
|
|
||||||
|
| http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
|
|
||||||
|
| Unless required by applicable law or agreed to in writing, software
|
||||||
|
| distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
| See the License for the specific language governing permissions and
|
||||||
|
| limitations under the License.
|
||||||
|
|""".stripMargin
|
||||||
|
|
||||||
|
def getLicenceContent(
|
||||||
|
header: String = "#",
|
||||||
|
linePrefix: String = "#",
|
||||||
|
footer: String = "#"): Iterable[String] = {
|
||||||
|
val content = apacheLicenceContent.split("\n").map(line => linePrefix + line)
|
||||||
|
Seq(header) ++ content ++ Seq(footer)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
Loading…
Reference in New Issue
Block a user