Design for continuous processing shuffle

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Design for continuous processing shuffle

Joseph Torres
Hi all,

A few of us have been working on a design for how to do shuffling in continuous processing. Feel free to chip in if you have any comments or questions.

doc:



Jose
Reply | Threaded
Open this post in threaded view
|

Re: Design for continuous processing shuffle

Yuanjian Li
Hi Joseph and devs,

Happy to see the discussion of CP shuffle, as comment in https://issues.apache.org/jira/browse/SPARK-20928?focusedCommentId=16245556&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16245556, the team also do some design and demo work over CP shuffle, the docs show more detailed work:




在 2018年5月5日,02:27,Joseph Torres <[hidden email]> 写道:

Hi all,

A few of us have been working on a design for how to do shuffling in continuous processing. Feel free to chip in if you have any comments or questions.

doc:



Jose