is it possible to apply AQE rules only on some of nodes?

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

is it possible to apply AQE rules only on some of nodes?

CodingCat
Hi, Spark devs

I am wondering if it is possible to apply AQE on part of the physical plan? e.g. I only want to apply coalesce partitions on a particular ShuffleQueryStageExec?

I didn't find a very straightforward way to achieve this, but is there a way to workaround the current limitation?

Thanks!

Nan