3ecdd42286
### _Why are the changes needed?_ This PR proposes change the Kyuubi TPC-DS generator to pure Java implementation instead of the original C binary. The new pure Java TPC-DS generator is under Apache License, and in fact, I don't know the original C binary License, so we exclude them from release in the past. Since the change removes the License issue of Kyuubi TPC-DS module, we can bundle the TPC-DS tool in the future release. And after migration, I haven't see "error=26, Text file busy" described in #2439 any more. ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [x] Add screenshots for manual tests if appropriate 1. Use old C binary based TPC-DS generator generate 1GB data under database `tpcds_s1` 2. Use new pure Java base TPC-DS generator generate 1GB data under database `new_tpcds_sf1` 3. Compare results of `select count(*)`, and `select sum(hash(*))` ``` spark-sql> select count(*) from tpcds_s1.inventory; 11745000 Time taken: 0.161 seconds, Fetched 1 row(s) spark-sql> select count(*) from new_tpcds_sf1.inventory; 11745000 Time taken: 0.141 seconds, Fetched 1 row(s) spark-sql> select sum(hash(*)) from tpcds_s1.inventory; -556768665838 Time taken: 0.252 seconds, Fetched 1 row(s) spark-sql> select sum(hash(*)) from new_tpcds_sf1.inventory; -556768665838 Time taken: 0.232 seconds, Fetched 1 row(s) ``` - [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #2449 from pan3793/tpcds. Closes #2439 a270bcba [Cheng Pan] Remove the exclusion in source release 7c8d3271 [Cheng Pan] [KYUUBI #2439] Using Pure Java TPC-DS generator Authored-by: Cheng Pan <chengpan@apache.org> Signed-off-by: Kent Yao <yao@apache.org>
33 lines
1.1 KiB
Plaintext
33 lines
1.1 KiB
Plaintext
#
|
|
# Licensed to the Apache Software Foundation (ASF) under one or more
|
|
# contributor license agreements. See the NOTICE file distributed with
|
|
# this work for additional information regarding copyright ownership.
|
|
# The ASF licenses this file to You under the Apache License, Version 2.0
|
|
# (the "License"); you may not use this file except in compliance with
|
|
# the License. You may obtain a copy of the License at
|
|
#
|
|
# http://www.apache.org/licenses/LICENSE-2.0
|
|
#
|
|
# Unless required by applicable law or agreed to in writing, software
|
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
# See the License for the specific language governing permissions and
|
|
# limitations under the License.
|
|
#
|
|
|
|
.github/ export-ignore
|
|
.idea/ export-ignore
|
|
.readthedocs.yml export-ignore
|
|
.travis.yml export-ignore
|
|
_config.yml export-ignore
|
|
codecov.yml export-ignore
|
|
licenses-binary/ export-ignore
|
|
LICENSE-binary export-ignore
|
|
NOTICE-binary export-ignore
|
|
*.bat text eol=crlf
|
|
*.cmd text eol=crlf
|
|
*.java text eol=lf
|
|
*.scala text eol=lf
|
|
*.xml text eol=lf
|
|
*.py text eol=lf
|