[DISCUSS] Shall we mark spark streaming component as deprecated.

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

[DISCUSS] Shall we mark spark streaming component as deprecated.

Prashant Sharma
Hi All,

It is noticed that some of the users of Spark streaming do not immediately realise that it is a deprecated component and it would be scary, if they end up with it in production. Now that we are in a position to release about Spark 3.0.0, may be we should discuss - should the spark streaming carry an explicit notice? That it is deprecated and not under active development.

I have opened an issue already, but I think a mailing list discussion would be more appropriate. https://issues.apache.org/jira/browse/SPARK-31006

Thanks,
Prashant.

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Shall we mark spark streaming component as deprecated.

Ismaël Mejía
Is it really ready to be deprecated? The fact that we cannot do multiple
aggregations with structured streaming [1] is a serious runtime limitation that
the DStream API does not have,
Is it worth to deprecate without having an equivalent set of features?

[1] https://issues.apache.org/jira/browse/SPARK-26655


On Mon, Mar 2, 2020 at 11:52 AM Prashant Sharma <[hidden email]> wrote:

>
> Hi All,
>
> It is noticed that some of the users of Spark streaming do not immediately realise that it is a deprecated component and it would be scary, if they end up with it in production. Now that we are in a position to release about Spark 3.0.0, may be we should discuss - should the spark streaming carry an explicit notice? That it is deprecated and not under active development.
>
> I have opened an issue already, but I think a mailing list discussion would be more appropriate. https://issues.apache.org/jira/browse/SPARK-31006
>
> Thanks,
> Prashant.
>

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Shall we mark spark streaming component as deprecated.

Sean Owen-2
In reply to this post by Prashant Sharma
Er, who says it's deprecated? I have never heard anything like that.
Why would it be?

On Mon, Mar 2, 2020 at 4:52 AM Prashant Sharma <[hidden email]> wrote:

>
> Hi All,
>
> It is noticed that some of the users of Spark streaming do not immediately realise that it is a deprecated component and it would be scary, if they end up with it in production. Now that we are in a position to release about Spark 3.0.0, may be we should discuss - should the spark streaming carry an explicit notice? That it is deprecated and not under active development.
>
> I have opened an issue already, but I think a mailing list discussion would be more appropriate. https://issues.apache.org/jira/browse/SPARK-31006
>
> Thanks,
> Prashant.
>

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Shall we mark spark streaming component as deprecated.

Prashant Sharma
I may have speculated, or believed the unauthorised sources, nevertheless I am happy to be corrected. 

On Mon, Mar 2, 2020 at 8:05 PM Sean Owen <[hidden email]> wrote:
Er, who says it's deprecated? I have never heard anything like that.
Why would it be?

On Mon, Mar 2, 2020 at 4:52 AM Prashant Sharma <[hidden email]> wrote:
>
> Hi All,
>
> It is noticed that some of the users of Spark streaming do not immediately realise that it is a deprecated component and it would be scary, if they end up with it in production. Now that we are in a position to release about Spark 3.0.0, may be we should discuss - should the spark streaming carry an explicit notice? That it is deprecated and not under active development.
>
> I have opened an issue already, but I think a mailing list discussion would be more appropriate. https://issues.apache.org/jira/browse/SPARK-31006
>
> Thanks,
> Prashant.
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Shall we mark spark streaming component as deprecated.

rxin
It's a good discussion to have though: should we deprecate dstream, and what do we need to do to make that happen? My experience working with a lot of Spark users is that in general I recommend them staying away from dstream, due to a lot of design and architectural issues.



On Mon, Mar 02, 2020 at 9:32 AM, Prashant Sharma <[hidden email]> wrote:
I may have speculated, or believed the unauthorised sources, nevertheless I am happy to be corrected. 

On Mon, Mar 2, 2020 at 8:05 PM Sean Owen <[hidden email]> wrote:
Er, who says it's deprecated? I have never heard anything like that.
Why would it be?

On Mon, Mar 2, 2020 at 4:52 AM Prashant Sharma <[hidden email]> wrote:
>
> Hi All,
>
> It is noticed that some of the users of Spark streaming do not immediately realise that it is a deprecated component and it would be scary, if they end up with it in production. Now that we are in a position to release about Spark 3.0.0, may be we should discuss - should the spark streaming carry an explicit notice? That it is deprecated and not under active development.
>
> I have opened an issue already, but I think a mailing list discussion would be more appropriate. https://issues.apache.org/jira/browse/SPARK-31006
>
> Thanks,
> Prashant.
>


smime.p7s (6K) Download Attachment