Re: cannot call explain or show on dataframe in structured streaming addBatch dataframe
There is a little bit of weirdness to how we override the default query planner to replace it with an incrementalizing planner. As such, calling any operation that changes the query plan (such as a LIMIT) would cause it to revert to the batch planner and return the wrong answer. We should fix this before the finalize the Sink API.
On Mon, Jun 19, 2017 at 9:32 AM, assaf.mendelson <[hidden email]> wrote:
I am playing around with structured streaming and looked at the code for ConsoleSink.