Continuous task retry support

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Continuous task retry support

Basil Hariri

Hi all,

 

I found that task retries are currently not supported in continuous processing mode. Is there another way to recover from continuous task failures currently? If not, are there plans to support this in a future release?

Thanks,

Basil

Reply | Threaded
Open this post in threaded view
|

Re: Continuous task retry support

Yuanjian Li
I found that task retries are currently not supported in continuous processing mode. Is there another way to recover from continuous task failures currently?
Yes, currently task level retry is not supported in CP mode and the recover strategy instead by stage restart.

 If not, are there plans to support this in a future release?
 Actually task level retry in CP mode is easy to implement in map-only operators but need more discussion when we plan to support more shuffled stateful operators in CP. More discussion in https://github.com/apache/spark/pull/20675.

Basil Hariri <[hidden email]> 于2018年11月3日周六 上午3:09写道:

Hi all,

 

I found that task retries are currently not supported in continuous processing mode. Is there another way to recover from continuous task failures currently? If not, are there plans to support this in a future release?

Thanks,

Basil