39ab731b85
### What changes were proposed in this pull request? 1. replaced the usage of `HashMap` with `ConcurrentHashMap` for `partitionBatchIdMap` to ensure thread safety during parallel data processing 2. put the partition id and batch id into the `partitionBatchIdMap` before adding the task to prevent the possibility of a NPE ### Why are the changes needed? to fix NPE https://github.com/apache/incubator-celeborn/actions/runs/5734532048/job/15540863715?pr=1785 ``` xception in thread "DataPusher-0" java.lang.NullPointerException at org.apache.celeborn.client.write.DataPushQueueSuiteJ$1.pushData(DataPushQueueSuiteJ.java:121) at org.apache.celeborn.client.write.DataPusher$1.run(DataPusher.java:125) Error: The operation was canceled. ``` ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Pass GA Closes #1789 from cfmcgrady/celeborn-875-followup. Authored-by: Fu Chen <cfmcgrady@gmail.com> Signed-off-by: zky.zhoukeyong <zky.zhoukeyong@alibaba-inc.com> |
||
|---|---|---|
| .. | ||
| java/org/apache/celeborn/client | ||
| resources | ||
| scala/org/apache/celeborn/client | ||