Need guidance on Spark Session Termination.

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Need guidance on Spark Session Termination.

Nasrulla Khan Haris

Hi fellow Spark-devs,

 

I am pretty new to spark core and I am looking for some answers to my use case.  I have a datasource v2 api connector, In my connector we create temporary files on the blob storage. Can you please suggest places where I can look if I want to delete the temporary files on storage at the end of Spark Session?

 

Thanks,

Nasrulla

 

Reply | Threaded
Open this post in threaded view
|

RE: Need guidance on Spark Session Termination.

Nasrulla Khan Haris

HI All,

 

Any Inputs here ?

 

Thanks,

Nasrulla

 

From: Nasrulla Khan Haris <[hidden email]>
Sent: Tuesday, May 7, 2019 3:58 PM
To: [hidden email]
Subject: Need guidance on Spark Session Termination.

 

Hi fellow Spark-devs,

 

I am pretty new to spark core and I am looking for some answers to my use case.  I have a datasource v2 api connector, In my connector we create temporary files on the blob storage. Can you please suggest places where I can look if I want to delete the temporary files on storage at the end of Spark Session?

 

Thanks,

Nasrulla