Aggregate pushdown for data source

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

Aggregate pushdown for data source

Arun Khetarpal
Hi Folks: 

I have implemented a data source v2 API for an internal source. As a consequence of generating the data source, we have bunch of statistical information about the source which i can potentially use, only if spark pushes down the aggregates down to the data source itself. 

I see that there is already a comprehensive Jira for the same: Are there any more blockers for this Jira? I'll be more than happy to contribute if no one else had picked it up.