Support structured plan logging

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Support structured plan logging

bo yang
Hi All,

Are there any people interested in adding structured plan logging in Spark? Currently the logical/physical plan could be logged as plain text via explain() method, which has some issues, for example, string truncation and difficult for tool/program to use.

This PR fixes the truncation issue. A further step is to log the plan as structured content (e.g. json). Do other people feel similar need?

Thanks,
Bo