[SHUFFLE]FAILED_TO_UNCOMPRESS(5) errors when fetching shuffle data with sort-based shuffle

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[SHUFFLE]FAILED_TO_UNCOMPRESS(5) errors when fetching shuffle data with sort-based shuffle

wangfei
Hi all,
     Non-deterministic FAILED_TO_UNCOMPRESS(5) or ’Stream is corrupted’  errors may occur during shuffle read, described as this JIRA(https://issues.apache.org/jira/browse/SPARK-4105).
     There is not new comment for a long time in this JIRA.  So,  Is there anyone seen these errors in latest version, such as spark-2.3?
     Can anyone provide a reproducible case or  analyze the cause of these errors?
     Thanks.