Resolve _temporary directory uncleaned

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

Resolve _temporary directory uncleaned

hi devs,

In InsertIntoHiveTable and InsertIntoHiveDirCommand, we use
deleteExternalTmpPath to clean temporary directories after Job committed and
cancel deleteOnExit if succeeded.But sometimes (e.g., when speculative task
is enabled), temporary directories may be left uncleaned. This is happened
if there are still some tasks running after we called deleteExternalTmpPath.
Thus I tried to add JobCleaned Status to clean temporary directories. The
JobCleaned Status is happened with all stages has been cleaned in a job,
thus it is a good way to do some job cleanings.


There have been some discuss in

This PR has been submitted for about 3 months. Any one could review this?

Jacky Lee

Sent from:

To unsubscribe e-mail: [hidden email]