GenerateExec Whole stage code generation is disabled

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

GenerateExec Whole stage code generation is disabled

Eugene Koifman

Hi,

 

Are there any plans to support this?

If not, could anyone explain why it should not be supported?

 

 

It looks like it’s been done in https://github.com/apache/spark/commit/b5c5bd98ea5e8dbfebcf86c5459bdf765f5ceb53#diff-0314224342bb8c30143ab784b3805d19

but there is no clear explanation as to why

 

It’s still disabled in current master

https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/GenerateExec.scala#L127

 

 

Thanks,

Eugene