[Spark Streaming] mapStateByKey Python SDK

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

[Spark Streaming] mapStateByKey Python SDK

I am a user of Spark Streaming Python SDK. In many cases stateful operations
are the only way to approach problems I am solving. UpdateStateByKey method,
which I believe is the only way to introduce a stateful operator, requires
some extra code to extract result from the states and to remove values for
keys not updated for a long time. Yet I can see there is a method in Scala
SDK, mapStateByKey, which solves this problems. Is there a plan to introduce
it into Python SDK?


Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/

To unsubscribe e-mail: [hidden email]