Fwd: Handling Skewness and Heterogeneity

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

Fwd: Handling Skewness and Heterogeneity

Anis Nasir
Dear all,

Can you please comment on the below mentioned use case. 

Thanking you in advance


---------- Forwarded message ---------
From: Anis Nasir <[hidden email]>
Date: Tue, 14 Feb 2017 at 17:01
Subject: Handling Skewness and Heterogeneity
To: <[hidden email]>

Dear All,

I have few use cases for spark streaming where spark cluster consist of heterogenous machines. 

Additionally, there is skew present in both the input distribution (e.g., each tuple is drawn from a zipf distribution) and the service time (e.g., service time required for each tuple comes from a zipf distribution). 

I want to know who spark will handle such use cases.

Any help will be highly appreciated!