3abc7ce371
### _Why are the changes needed?_ to close #4167 which parent issue is #3632 Introduce HiveFunctionPrivilegeBuilder refactored with Serde layers. The core logic to this is scanning logical plan by transformAllExpressions and build function privileges for UDFs. As a SQL gateway, Kyuubi also needs to support control user UDF usage behavior. Therefore, the Spark SQL Authz module needs to add suport for extracting Privilege Objects for UDF usage in queries from a spark logical plans. In Spark SQL, the hive permanent UDF is wrapped into the following types based on different types: Org.apache.spark.sql.live.HiveSimpleUDF Org.apache.spark.sql.live.HiveGenericUDF Org. apache. park. SQL. live. HiveUDAFFunction Org.apache.spark.sql.live.HiveGenericUDTF Based on the existing serde module, we can define a ScanSpec to extract hive permanent udf expression's info, the most important info is the udf type. And the udf type decide whether to skip the construction of the privilege object based on whether the function type is a system function or a temporary function, as we only consider the permanent udf uasge. So we should define QualifiedNameStringFunctionExtractor and FunctionNameFunctionTypeExtractor to achieve the goal. And in `org.apache.kyuubi.plugin.spark.authz.PrivilegesBuilder#buildFunctions`, we define a method `buildFunctions`, use `transformAllExpressions` to gather all hive udf expression, and building input privilege objects for thoese permanent hive udf. ### _How was this patch tested?_ - [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #4168 from packyan/feature_introduce_hive_function_privileges_builder. Closes #4167 1276ac7f6 [Deng An] [KYUUBI #4167] [Authz] Introduce function support in PrivilegeBuilder with Serde layers Authored-by: Deng An <packyande@gmail.com> Signed-off-by: liangbowen <liangbowen@gf.com.cn> |
||
|---|---|---|
| .. | ||
| server/kyuubi-server-plugin | ||
| spark | ||
| README.md | ||
For developers
This folder contains plugins/extension for kyuubi server and different engine types.
- ext
- kyuubi-server
- spark
- flink
- trino
- hive
- others
- ...