Executor exceptions stacktrace omitted by HotSpot in long running application

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Executor exceptions stacktrace omitted by HotSpot in long running application

ZHANG Wei
Hi,

I'm considering to improve the experience of hitting potential
exceptions stacktrace omitted in long running application[1], which is
a JVM HotSpot optimization as Shixiong(Ryan) commented[2].

There might be 2 options:
1. Adds `-XX:-OmitStackTraceInFastThrow` as a common Executor JVM
option.
2. Adds an Executor replacement workflow with a fresh new JVM when
exceptions stacktrace being omitted.

I prefer option 1, there might be performance concerns. But option 2
isn't supposed to increase the performance since which JVM is replaced
by a new one to force working on no stacktrace fast throw mode.

May I get any concern or options on this?

--
Thanks,
-z

[1] http://apache-spark-user-list.1001560.n3.nabble.com/Structured-Streaming-NullPointerException-in-long-running-query-td37453.html
[2] https://stackoverflow.com/a/3010106

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]