2540f44a87
### _Why are the changes needed?_ This PR redesigned the authorization part of the spark authz module with a New Authz Plan Serde Layer. #### Motivation - add a general layer to describe a command, so that we can add a new command or users can add a third-party command easily according to the specification. - get rid of the spark version check. The built-in spark commands frequently vary from version to version, which makes us hard to maintain at compile& runtime phase, and the third-party commands are hard to check by spark versions. #### Data structure  Overall, we introduce 2 general basic data structures: - CommandSpec: used to describe a command - `classname` as key for the read-side to get the spec by a particular command - pre-defined operation type - descriptors - Descriptor: used to describe an object, such as table, db, query, - `fieldName`: the object to get - `fieldExtractor`: the method to get the object; use SPI to load - sub-descriptors: such as columns in a table - etc. #### SPI - Extractor: implementations for `fieldExtractor` - key: the name of the extractor for the read-side to get itself - func: converting the field value to specific and general objects #### Code Path - Write code path - automatically generated default json configuration files - custom json configuration files for thrid-party commands - Read code path - Load json as maps - RuleAuthorization -> PrivilegeBuilder.build -> get command desc from maps -> build privileges with the retrieved desc. #### TODOs - Add back the ArcticCommand - Add delta command - Add ways for loading custom json configuration files - Add hudi commands - etc ### _How was this patch tested?_ - [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #3904 from yaooqinn/na. Closes #3904 efafcbaa2 [Kent Yao] sort 7d2b3e41c [Kent Yao] ci 379e93343 [Kent Yao] style b52ab418a [Kent Yao] comments 9a24be64d [Kent Yao] [WIP][Extension][Spark] New Authz Plan Serde Layer 8926f0446 [Kent Yao] [WIP][Extension][Spark] New Authz Plan Serde Layer b45453a3e [Kent Yao] [WIP][Extension][Spark] New Authz Plan Serde Layer f56148ea2 [Kent Yao] [WIP][Extension][Spark] New Authz Plan Serde Layer e47749d96 [Kent Yao] [WIP][Extension][Spark] New Authz Plan Serde Layer c32feef63 [Kent Yao] [WIP][Extension][Spark] New Authz Plan Serde Layer 49dbb6842 [Kent Yao] [WIP][Extension][Spark] New Authz Plan Serde Layer 2bced47b6 [Kent Yao] Merge branch 'master' into na 453541bd1 [Kent Yao] [WIP][Extension][Spark] New Authz Plan Serde Layer Authored-by: Kent Yao <yao@apache.org> Signed-off-by: Kent Yao <yao@apache.org>
322 lines
12 KiB
XML
322 lines
12 KiB
XML
<?xml version="1.0" encoding="UTF-8"?>
|
|
<!--
|
|
~ Licensed to the Apache Software Foundation (ASF) under one or more
|
|
~ contributor license agreements. See the NOTICE file distributed with
|
|
~ this work for additional information regarding copyright ownership.
|
|
~ The ASF licenses this file to You under the Apache License, Version 2.0
|
|
~ (the "License"); you may not use this file except in compliance with
|
|
~ the License. You may obtain a copy of the License at
|
|
~
|
|
~ http://www.apache.org/licenses/LICENSE-2.0
|
|
~
|
|
~ Unless required by applicable law or agreed to in writing, software
|
|
~ distributed under the License is distributed on an "AS IS" BASIS,
|
|
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
~ See the License for the specific language governing permissions and
|
|
~ limitations under the License.
|
|
-->
|
|
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
|
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
|
|
<modelVersion>4.0.0</modelVersion>
|
|
<parent>
|
|
<groupId>org.apache.kyuubi</groupId>
|
|
<artifactId>kyuubi-parent</artifactId>
|
|
<version>1.7.0-SNAPSHOT</version>
|
|
<relativePath>../../../pom.xml</relativePath>
|
|
|
|
</parent>
|
|
|
|
<artifactId>kyuubi-spark-authz_2.12</artifactId>
|
|
<packaging>jar</packaging>
|
|
<name>Kyuubi Dev Spark Authorization Extension</name>
|
|
<url>https://kyuubi.apache.org/</url>
|
|
|
|
<properties>
|
|
<gethostname4j.version>1.0.0</gethostname4j.version>
|
|
<jna.version>5.7.0</jna.version>
|
|
</properties>
|
|
|
|
<dependencies>
|
|
<dependency>
|
|
<groupId>org.apache.ranger</groupId>
|
|
<artifactId>ranger-plugins-common</artifactId>
|
|
<version>${ranger.version}</version>
|
|
<exclusions>
|
|
<exclusion>
|
|
<groupId>org.apache.ranger</groupId>
|
|
<artifactId>ranger-plugin-classloader</artifactId>
|
|
</exclusion>
|
|
<exclusion>
|
|
<groupId>org.apache.ranger</groupId>
|
|
<artifactId>ranger-plugins-audit</artifactId>
|
|
</exclusion>
|
|
<exclusion>
|
|
<groupId>log4j</groupId>
|
|
<artifactId>log4j</artifactId>
|
|
</exclusion>
|
|
<exclusion>
|
|
<groupId>ch.qos.logback</groupId>
|
|
<artifactId>logback-classic</artifactId>
|
|
</exclusion>
|
|
<exclusion>
|
|
<groupId>commons-lang</groupId>
|
|
<artifactId>commons-lang</artifactId>
|
|
</exclusion>
|
|
<exclusion>
|
|
<groupId>commons-logging</groupId>
|
|
<artifactId>commons-logging</artifactId>
|
|
</exclusion>
|
|
<exclusion>
|
|
<groupId>org.apache.httpcomponents</groupId>
|
|
<artifactId>*</artifactId>
|
|
</exclusion>
|
|
<exclusion>
|
|
<groupId>org.apache.hadoop</groupId>
|
|
<artifactId>hadoop-common</artifactId>
|
|
</exclusion>
|
|
<exclusion>
|
|
<groupId>javax.ws.rs</groupId>
|
|
<artifactId>jsr311-api</artifactId>
|
|
</exclusion>
|
|
<exclusion>
|
|
<groupId>org.codehaus.jackson</groupId>
|
|
<artifactId>jackson-core-asl</artifactId>
|
|
</exclusion>
|
|
<exclusion>
|
|
<groupId>org.codehaus.jackson</groupId>
|
|
<artifactId>jackson-mapper-asl</artifactId>
|
|
</exclusion>
|
|
<exclusion>
|
|
<groupId>com.kstruct</groupId>
|
|
<artifactId>gethostname4j</artifactId>
|
|
</exclusion>
|
|
<exclusion>
|
|
<groupId>net.java.dev.jna</groupId>
|
|
<artifactId>jna</artifactId>
|
|
</exclusion>
|
|
<exclusion>
|
|
<groupId>net.java.dev.jna</groupId>
|
|
<artifactId>jna-platform</artifactId>
|
|
</exclusion>
|
|
</exclusions>
|
|
</dependency>
|
|
|
|
<dependency>
|
|
<groupId>com.kstruct</groupId>
|
|
<artifactId>gethostname4j</artifactId>
|
|
<version>${gethostname4j.version}</version>
|
|
</dependency>
|
|
|
|
<dependency>
|
|
<groupId>net.java.dev.jna</groupId>
|
|
<artifactId>jna</artifactId>
|
|
<version>${jna.version}</version>
|
|
</dependency>
|
|
|
|
<dependency>
|
|
<groupId>net.java.dev.jna</groupId>
|
|
<artifactId>jna-platform</artifactId>
|
|
<version>${jna.version}</version>
|
|
</dependency>
|
|
|
|
<dependency>
|
|
<groupId>org.apache.ranger</groupId>
|
|
<artifactId>ranger-plugins-audit</artifactId>
|
|
<version>${ranger.version}</version>
|
|
<exclusions>
|
|
<exclusion>
|
|
<groupId>org.apache.ranger</groupId>
|
|
<artifactId>ranger-plugins-cred</artifactId>
|
|
</exclusion>
|
|
<exclusion>
|
|
<groupId>org.apache.kafka</groupId>
|
|
<artifactId>*</artifactId>
|
|
</exclusion>
|
|
<exclusion>
|
|
<groupId>org.apache.solr</groupId>
|
|
<artifactId>solr-solrj</artifactId>
|
|
</exclusion>
|
|
<exclusion>
|
|
<groupId>org.elasticsearch</groupId>
|
|
<artifactId>*</artifactId>
|
|
</exclusion>
|
|
<exclusion>
|
|
<groupId>org.elasticsearch.client</groupId>
|
|
<artifactId>*</artifactId>
|
|
</exclusion>
|
|
<exclusion>
|
|
<groupId>org.elasticsearch.plugin</groupId>
|
|
<artifactId>*</artifactId>
|
|
</exclusion>
|
|
<exclusion>
|
|
<groupId>org.apache.lucene</groupId>
|
|
<artifactId>*</artifactId>
|
|
</exclusion>
|
|
<exclusion>
|
|
<groupId>log4j</groupId>
|
|
<artifactId>log4j</artifactId>
|
|
</exclusion>
|
|
<exclusion>
|
|
<groupId>commons-lang</groupId>
|
|
<artifactId>commons-lang</artifactId>
|
|
</exclusion>
|
|
<exclusion>
|
|
<groupId>commons-logging</groupId>
|
|
<artifactId>commons-logging</artifactId>
|
|
</exclusion>
|
|
<exclusion>
|
|
<groupId>com.carrotsearch</groupId>
|
|
<artifactId>hppc</artifactId>
|
|
</exclusion>
|
|
<exclusion>
|
|
<groupId>org.apache.httpcomponents</groupId>
|
|
<artifactId>*</artifactId>
|
|
</exclusion>
|
|
<exclusion>
|
|
<groupId>org.apache.hive</groupId>
|
|
<artifactId>hive-storage-api</artifactId>
|
|
</exclusion>
|
|
<exclusion>
|
|
<groupId>org.apache.orc</groupId>
|
|
<artifactId>orc-core</artifactId>
|
|
</exclusion>
|
|
<exclusion>
|
|
<groupId>org.apache.hadoop</groupId>
|
|
<artifactId>hadoop-common</artifactId>
|
|
</exclusion>
|
|
<exclusion>
|
|
<groupId>com.google.guava</groupId>
|
|
<artifactId>guava</artifactId>
|
|
</exclusion>
|
|
<exclusion>
|
|
<groupId>joda-time</groupId>
|
|
<artifactId>joda-time</artifactId>
|
|
</exclusion>
|
|
<exclusion>
|
|
<groupId>org.apache.logging.log4j</groupId>
|
|
<artifactId>*</artifactId>
|
|
</exclusion>
|
|
<exclusion>
|
|
<groupId>com.amazonaws</groupId>
|
|
<artifactId>aws-java-sdk-bundle</artifactId>
|
|
</exclusion>
|
|
</exclusions>
|
|
</dependency>
|
|
|
|
<dependency>
|
|
<groupId>org.scala-lang</groupId>
|
|
<artifactId>scala-library</artifactId>
|
|
<scope>provided</scope>
|
|
</dependency>
|
|
|
|
<dependency>
|
|
<groupId>org.apache.spark</groupId>
|
|
<artifactId>spark-sql_${scala.binary.version}</artifactId>
|
|
<scope>provided</scope>
|
|
</dependency>
|
|
|
|
<dependency>
|
|
<groupId>org.apache.hadoop</groupId>
|
|
<artifactId>hadoop-client-api</artifactId>
|
|
<scope>provided</scope>
|
|
</dependency>
|
|
|
|
<dependency>
|
|
<groupId>org.apache.hadoop</groupId>
|
|
<artifactId>hadoop-client-runtime</artifactId>
|
|
<scope>provided</scope>
|
|
</dependency>
|
|
|
|
<dependency>
|
|
<groupId>org.slf4j</groupId>
|
|
<artifactId>slf4j-api</artifactId>
|
|
<scope>provided</scope>
|
|
</dependency>
|
|
|
|
<dependency>
|
|
<groupId>org.apache.logging.log4j</groupId>
|
|
<artifactId>log4j-api</artifactId>
|
|
<scope>provided</scope>
|
|
</dependency>
|
|
|
|
<dependency>
|
|
<groupId>org.apache.logging.log4j</groupId>
|
|
<artifactId>log4j-core</artifactId>
|
|
<scope>provided</scope>
|
|
</dependency>
|
|
|
|
<dependency>
|
|
<groupId>commons-collections</groupId>
|
|
<artifactId>commons-collections</artifactId>
|
|
<scope>provided</scope>
|
|
</dependency>
|
|
|
|
<dependency>
|
|
<groupId>com.fasterxml.jackson.module</groupId>
|
|
<artifactId>jackson-module-scala_${scala.binary.version}</artifactId>
|
|
</dependency>
|
|
|
|
<dependency>
|
|
<groupId>org.apache.kyuubi</groupId>
|
|
<artifactId>kyuubi-common_${scala.binary.version}</artifactId>
|
|
<version>${project.version}</version>
|
|
<scope>test</scope>
|
|
</dependency>
|
|
|
|
<dependency>
|
|
<groupId>org.apache.spark</groupId>
|
|
<artifactId>spark-hive_${scala.binary.version}</artifactId>
|
|
<scope>test</scope>
|
|
<!-- for hive related test only -->
|
|
</dependency>
|
|
|
|
<dependency>
|
|
<groupId>org.apache.logging.log4j</groupId>
|
|
<artifactId>log4j-slf4j-impl</artifactId>
|
|
<scope>test</scope>
|
|
</dependency>
|
|
|
|
<dependency>
|
|
<groupId>org.apache.logging.log4j</groupId>
|
|
<artifactId>log4j-1.2-api</artifactId>
|
|
<scope>test</scope>
|
|
</dependency>
|
|
|
|
<dependency>
|
|
<groupId>com.google.code.gson</groupId>
|
|
<artifactId>gson</artifactId>
|
|
<scope>test</scope>
|
|
</dependency>
|
|
|
|
<dependency>
|
|
<groupId>com.google.guava</groupId>
|
|
<artifactId>guava</artifactId>
|
|
<scope>test</scope>
|
|
</dependency>
|
|
|
|
<dependency>
|
|
<groupId>org.apache.iceberg</groupId>
|
|
<artifactId>iceberg-spark-runtime-${spark.binary.version}_${scala.binary.version}</artifactId>
|
|
<scope>test</scope>
|
|
</dependency>
|
|
|
|
<dependency>
|
|
<groupId>org.scala-lang.modules</groupId>
|
|
<artifactId>scala-collection-compat_${scala.binary.version}</artifactId>
|
|
<scope>test</scope>
|
|
</dependency>
|
|
</dependencies>
|
|
|
|
<build>
|
|
|
|
<testResources>
|
|
<testResource>
|
|
<directory>${project.basedir}/src/test/resources</directory>
|
|
</testResource>
|
|
</testResources>
|
|
<outputDirectory>target/scala-${scala.binary.version}/classes</outputDirectory>
|
|
<testOutputDirectory>target/scala-${scala.binary.version}/test-classes</testOutputDirectory>
|
|
</build>
|
|
|
|
</project>
|