[Spark Debug] Search vars of interest

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

[Spark Debug] Search vars of interest

Joseph PENG
I am debug spark (itself not its application) by running `sbt  default test suite`. I have successfully set up everything, and IntelliJ debugger have caught every variables. However, I can not find the variables I need, because there are so many variables. 

For example, I want to find the input data frame, the transformed data frame, and the output data frame. Where should I look at?

What I tried:
1. use search function in IntelliJ to search variable name.
2. try to understand how scaliest works.

Can someone point out a direction?