Heap Settings for History Server

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Heap Settings for History Server

Neelesh Salian
Hi folks,

I couldn't find much literature on this so I figured I could ask here.

Does anyone have experience in tuning the memory settings and interval times of the Spark History Server?
Let's say I have 500 applications at 0.5 G each with a spark.history.fs.update.interval  of 400s.
Is there a direct memory correlation that can help me set an optimum value?

Looking for some advice if anyone has tuned the History Server to render large amounts of applications.

Thanks.
--
Regards,
Neelesh S. Salian

Loading...