kyuubi/dev
Fu Chen 5602c29b36
[KYUUBI #1105] Zorder codegen support
<!--
Thanks for sending a pull request!

Here are some tips for you:
  1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
  2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
  3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->

### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
  1. If you add a feature, you can talk about the use case of it.
  2. If you fix a bug, you can clarify why it is a bug.
-->

1. zorder codegen support
2. move all of the zorder tests to a separate file.
3. add more zorder unit tests.

generate.java
```java
/* 001 */ public java.lang.Object generate(Object[] references) {
/* 002 */   return new SpecificUnsafeProjection(references);
/* 003 */ }
/* 004 */
/* 005 */ class SpecificUnsafeProjection extends org.apache.spark.sql.catalyst.expressions.UnsafeProjection {
/* 006 */
/* 007 */   private Object[] references;
/* 008 */   private org.apache.spark.sql.catalyst.expressions.codegen.UnsafeRowWriter[] mutableStateArray_0 = new org.apache.spark.sql.catalyst.expressions.codegen.UnsafeRowWriter[1];
/* 009 */
/* 010 */   public SpecificUnsafeProjection(Object[] references) {
/* 011 */     this.references = references;
/* 012 */     mutableStateArray_0[0] = new org.apache.spark.sql.catalyst.expressions.codegen.UnsafeRowWriter(1, 32);
/* 013 */
/* 014 */   }
/* 015 */
/* 016 */   public void initialize(int partitionIndex) {
/* 017 */
/* 018 */   }
/* 019 */
/* 020 */   // Scala.Function1 need this
/* 021 */   public java.lang.Object apply(java.lang.Object row) {
/* 022 */     return apply((InternalRow) row);
/* 023 */   }
/* 024 */
/* 025 */   public UnsafeRow apply(InternalRow i) {
/* 026 */     mutableStateArray_0[0].reset();
/* 027 */
/* 028 */
/* 029 */
/* 030 */
/* 031 */     byte[] value_0 = null;
/* 032 */     byte[][] binaryArray_0 = new byte[2][];
/* 033 */
/* 034 */     boolean isNull_1 = i.isNullAt(0);
/* 035 */     long value_1 = isNull_1 ?
/* 036 */     -1L : (i.getLong(0));
/* 037 */     if (isNull_1) {
/* 038 */       binaryArray_0[0] = (byte[]) ((byte[][]) references[0] /* defaultValues */)[0];
/* 039 */     } else {
/* 040 */       binaryArray_0[0] = org.apache.kyuubi.sql.zorder.ZorderBytesUtils.toByte(value_1);
/* 041 */     }
/* 042 */
/* 043 */
/* 044 */     boolean isNull_2 = i.isNullAt(1);
/* 045 */     long value_2 = isNull_2 ?
/* 046 */     -1L : (i.getLong(1));
/* 047 */     if (isNull_2) {
/* 048 */       binaryArray_0[1] = (byte[]) ((byte[][]) references[0] /* defaultValues */)[1];
/* 049 */     } else {
/* 050 */       binaryArray_0[1] = org.apache.kyuubi.sql.zorder.ZorderBytesUtils.toByte(value_2);
/* 051 */     }
/* 052 */
/* 053 */     value_0 = org.apache.kyuubi.sql.zorder.ZorderBytesUtils.interleaveMultiByteArray(binaryArray_0);
/* 054 */     mutableStateArray_0[0].write(0, value_0);
/* 055 */     return (mutableStateArray_0[0].getRow());
/* 056 */   }
/* 057 */
/* 058 */
/* 059 */ }
```

### _How was this patch tested?_
- [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1109 from cfmcgrady/zorder-codegen.

Closes #1105

f06da25a [Fu Chen] update dependency
8e04dc4e [Fu Chen] JMap -> Array
da00f86c [Fu Chen] remove Logging trait
266b65b3 [Fu Chen] fix style and revert Expression
e15165f3 [Fu Chen] ev.isNull = FalseLiteral
0d77d9de [Fu Chen] review
0d8467f4 [Fu Chen] fix style
5ede5df9 [Fu Chen] refactor test
56e12172 [Fu Chen] clean up
88573102 [Fu Chen] update
5f3d1c02 [Fu Chen] zorder codegen support

Authored-by: Fu Chen <cfmcgrady@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
2021-09-16 22:42:52 +08:00
..
kyuubi-codecov [BUILD] Bump 1.4.0-SNAPSHOT 2021-08-17 01:39:06 +08:00
kyuubi-extension-spark-3-1 [KYUUBI #1105] Zorder codegen support 2021-09-16 22:42:52 +08:00
kyuubi-tpcds [BUILD] Bump 1.4.0-SNAPSHOT 2021-08-17 01:39:06 +08:00
dependencyList [KYUUBI #1009] Implement Hive metastore server Delegation Token Provider 2021-09-09 19:19:58 +08:00
merge_kyuubi_pr.py [KYUUBI #809] [INFRA] Support reopened PR in pr merge tool 2021-07-15 18:05:07 +08:00