[CELEBORN-1380][FOLLOWUP] leveldbjni uses org.openlabtesting.leveldbjni to support linux aarch64 platform for leveldb via aarch64 profile

### What changes were proposed in this pull request?

Dependency leveldbjni uses `org.openlabtesting.leveldbjni` to support linux aarch64 platform for leveldb via `aarch64` profile.

Follow up #2476.

### Why are the changes needed?

Celeborn worker could not start on arm arch devices if db backend is `LevelDB`, which should support leveldbjni on the aarch64 platform.

aarch64 uses `org.openlabtesting.leveldbjni:leveldbjni-all.1.8`, and other platforms use `org.fusesource.leveldbjni:leveldbjni-all.1.8`. Meanwhile, because some hadoop dependencies packages are also depend on `org.fusesource.leveldbjni:leveldbjni-all`, but hadoop merge the similar change on trunk, details see
[HADOOP-16614](https://issues.apache.org/jira/browse/HADOOP-16614), therefore it should exclude the dependency of `org.fusesource.leveldbjni` for these hadoop packages related.

In addtion, `org.openlabtesting.leveldbjni` requires glibc version 3.4.21. Otherwise, there will be the following potential runtime risks:

```
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGBUS (0x7) at pc=0x00007fad3630b12a, pid=62, tid=0x00007f93394ef700
#
# JRE version: Java(TM) SE Runtime Environment (8.0_162-b12) (build 1.8.0_162-b12)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.162-b12 mixed mode linux-amd64 )
# Problematic frame:
# C  [libc.so.6+0x8412a]
#
# Core dump written. Default location: /data/service/celeborn/core or core.62
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#

---------------  T H R E A D  ---------------

Current thread (0x00007f9308001000):  JavaThread "leveldb" [_thread_in_native, id=878, stack(0x00007f9338cf0000,0x00007f93394f0000)]

siginfo: si_signo: 7 (SIGBUS), si_code: 2 (BUS_ADRERR), si_addr: 0x00007f97380d2220
```

Backport:

- https://github.com/apache/spark/pull/26636
- https://github.com/apache/spark/pull/31036

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

No.

Closes #2530 from SteNicholas/CELEBORN-1380.

Authored-by: SteNicholas <programgeek@163.com>
Signed-off-by: mingji <fengmingxiao.fmx@alibaba-inc.com>

LICENSE-binary

@@ -302,7 +302,7 @@ BSD 3-clause
 See licenses/LICENSE-protobuf.txt for details.
 com.google.protobuf:protobuf-java
 See licenses/LICENSE-leveldbjni.txt for details.
-org.openlabtesting.leveldbjni:leveldbjni-all
+org.fusesource.leveldbjni:leveldbjni-all
 
 
 Common Development and Distribution License (CDDL) 1.0

README.md

@@ -87,6 +87,7 @@ To compile for Spark 3.5 with Java21,  please use the following command
 ./build/make-distribution.sh --sbt-enabled -Pspark-3.5 -Pjdk-21
 ```
 
+> **_NOTE:_** Celeborn supports automatic builds on linux aarch64 platform via `aarch64` profile. `aarch64` profile requires glibc version 3.4.21. There is potential problematic frame `C  [libc.so.6+0x8412a]` for other glibc version like 2.x etc.
 
 ### Package Details
 Build procedure will create a compressed package.

common/pom.xml

@@ -87,7 +87,7 @@
       <artifactId>netty-all</artifactId>
     </dependency>
     <dependency>
-      <groupId>org.openlabtesting.leveldbjni</groupId>
+      <groupId>${leveldbjni.group}</groupId>
       <artifactId>leveldbjni-all</artifactId>
     </dependency>
     <dependency>

docs/deploy_on_k8s.md

@@ -38,6 +38,8 @@ You can find released version of Celeborn on [Downloading Page](https://celeborn
 Of course, you can build binary package from master branch or your own branch by using `./build/make-distribution.sh` in
 source code.
 
+> Notice: Celeborn supports automatic builds on linux aarch64 platform via `aarch64` profile. `aarch64` profile requires glibc version 3.4.21. There is potential problematic frame `C  [libc.so.6+0x8412a]` for other glibc version like 2.x etc.
+
 Anyway, you should unzip and into binary package.
 
 ### 2. Modify Celeborn Configurations

pom.xml

@@ -113,6 +113,9 @@
     <jetty.version>9.4.52.v20230823</jetty.version>
     <jakarta.servlet-api.version>4.0.4</jakarta.servlet-api.version>
 
+    <!-- org.fusesource.leveldbjni will be used except on arm64 platform. -->
+    <leveldbjni.group>org.fusesource.leveldbjni</leveldbjni.group>
+
     <shading.prefix>org.apache.celeborn.shaded</shading.prefix>
 
     <maven.plugin.antrun.version>3.0.0</maven.plugin.antrun.version>
@@ -394,7 +397,7 @@
         </exclusions>
       </dependency>
       <dependency>
-        <groupId>org.openlabtesting.leveldbjni</groupId>
+        <groupId>${leveldbjni.group}</groupId>
         <artifactId>leveldbjni-all</artifactId>
         <version>${leveldb.version}</version>
       </dependency>
@@ -1697,5 +1700,20 @@
         <module>tests/kubernetes-it</module>
       </modules>
     </profile>
+
+    <!-- use org.openlabtesting.leveldbjni on aarch64 platform except MacOS -->
+    <profile>
+      <id>aarch64</id>
+      <activation>
+        <os>
+          <family>linux</family>
+          <arch>aarch64</arch>
+        </os>
+      </activation>
+      <properties>
+        <!-- org.openlabtesting.leveldbjni requires glibc version 3.4.21 -->
+        <leveldbjni.group>org.openlabtesting.leveldbjni</leveldbjni.group>
+      </properties>
+    </profile>
   </profiles>
 </project>

project/CelebornBuild.scala

@@ -111,7 +111,15 @@ object Dependencies {
   val ioDropwizardMetricsJvm = "io.dropwizard.metrics" % "metrics-jvm" % metricsVersion
   val ioNetty = "io.netty" % "netty-all" % nettyVersion excludeAll(
     ExclusionRule("io.netty", "netty-handler-ssl-ocsp"))
-  val leveldbJniAll = "org.openlabtesting.leveldbjni" % "leveldbjni-all" % leveldbJniVersion
+  val leveldbJniGroup = if (System.getProperty("os.name").startsWith("Linux")
+    && System.getProperty("os.arch").equals("aarch64")) {
+    // use org.openlabtesting.leveldbjni on aarch64 platform except MacOS
+    // org.openlabtesting.leveldbjni requires glibc version 3.4.21
+    "org.openlabtesting.leveldbjni"
+  } else {
+    "org.fusesource.leveldbjni"
+  }
+  val leveldbJniAll = leveldbJniGroup % "leveldbjni-all" % leveldbJniVersion
   val log4j12Api = "org.apache.logging.log4j" % "log4j-1.2-api" % log4j2Version
   val log4jSlf4jImpl = "org.apache.logging.log4j" % "log4j-slf4j-impl" % log4j2Version
   val lz4Java = "org.lz4" % "lz4-java" % lz4JavaVersion

worker/pom.xml

@@ -49,7 +49,7 @@
       <version>${project.version}</version>
     </dependency>
     <dependency>
-      <groupId>org.openlabtesting.leveldbjni</groupId>
+      <groupId>${leveldbjni.group}</groupId>
       <artifactId>leveldbjni-all</artifactId>
     </dependency>
     <dependency>