the spark version is 2.1.0
This isn't related to the progress bar, it just happened while in that section of code. Something else is taking memory in the driver, usually a broadcast table or something else that requires a lot of memory and happens on the driver.
You should check your driver memory settings and the query plan (if this was SparkSQL) for this stage to investigate further.
On Thu, Feb 9, 2017 at 8:41 PM, John Fang <[hidden email]> wrote:
|Free forum by Nabble||Edit this page|