[CELEBORN-2022] Spark4 Client should package commons-io
### What changes were proposed in this pull request? ### Why are the changes needed? ```java Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/celeborn/shaded/org/apache/commons/io/output/ByteArrayOutputStream at org.apache.spark.shuffle.celeborn.CelebornShuffleDataIO.<init>(CelebornShuffleDataIO.java:41) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:75) at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:53) at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:502) at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:486) at org.apache.spark.util.Utils$.$anonfun$loadExtensions$1(Utils.scala:2834) at scala.collection.immutable.List.flatMap(List.scala:294) at scala.collection.immutable.List.flatMap(List.scala:79) at org.apache.spark.util.Utils$.loadExtensions(Utils.scala:2826) at org.apache.spark.shuffle.ShuffleDataIOUtils$.loadShuffleDataIO(ShuffleDataIOUtils.scala:35) at org.apache.spark.SparkContext.<init>(SparkContext.scala:633) at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:3055) ``` ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? local test Closes #3304 from cxzl25/CELEBORN-2022. Authored-by: sychen <sychen@ctrip.com> Signed-off-by: Shuang <lvshuang.xjs@alibaba-inc.com>
This commit is contained in:
parent
7bde738e5e
commit
c0512c67fd
@ -77,6 +77,7 @@
|
||||
<include>io.netty:*</include>
|
||||
<include>org.apache.commons:commons-lang3</include>
|
||||
<include>org.roaringbitmap:RoaringBitmap</include>
|
||||
<include>commons-io:commons-io</include>
|
||||
</includes>
|
||||
</artifactSet>
|
||||
<filters>
|
||||
|
||||
Loading…
Reference in New Issue
Block a user