Kicking off the process around Spark 2.2.1

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Kicking off the process around Spark 2.2.1

Holden Karau
Now that Spark 2.1.2 is out it seems like now is a good time to get started on the Spark 2.2.1 release. There are some streaming fixes I’m aware of that would be good to get into a release, is there anything else people are working on for 2.2.1 we should be tracking?

To switch it up I’d like to suggest Felix to be the RM for this since there are also likely some R packaging changes to be included in the release. This also gives us a chance to see if my updated release documentation if enough for a new RM to get started from.

What do folks think?
--
Reply | Threaded
Open this post in threaded view
|

Re: Kicking off the process around Spark 2.2.1

Sean Owen
It would be reasonably consistent with the timing of other x.y.1 releases, and more release managers sounds useful, yeah.

Note also that in theory the code freeze for 2.3.0 starts in about 2 weeks.

On Wed, Oct 25, 2017 at 12:29 PM Holden Karau <[hidden email]> wrote:
Now that Spark 2.1.2 is out it seems like now is a good time to get started on the Spark 2.2.1 release. There are some streaming fixes I’m aware of that would be good to get into a release, is there anything else people are working on for 2.2.1 we should be tracking?

To switch it up I’d like to suggest Felix to be the RM for this since there are also likely some R packaging changes to be included in the release. This also gives us a chance to see if my updated release documentation if enough for a new RM to get started from.

What do folks think?
--
Reply | Threaded
Open this post in threaded view
|

Re: Kicking off the process around Spark 2.2.1

Felix Cheung
Yes! I can take on RM for 2.2.1.

We are still working out what to do with temp files created by Hive and Java that cause the policy issue with CRAN and will report back shortly, hopefully.


From: Sean Owen <[hidden email]>
Sent: Wednesday, October 25, 2017 4:39:15 AM
To: Holden Karau
Cc: Felix Cheung; [hidden email]
Subject: Re: Kicking off the process around Spark 2.2.1
 
It would be reasonably consistent with the timing of other x.y.1 releases, and more release managers sounds useful, yeah.

Note also that in theory the code freeze for 2.3.0 starts in about 2 weeks.

On Wed, Oct 25, 2017 at 12:29 PM Holden Karau <[hidden email]> wrote:
Now that Spark 2.1.2 is out it seems like now is a good time to get started on the Spark 2.2.1 release. There are some streaming fixes I’m aware of that would be good to get into a release, is there anything else people are working on for 2.2.1 we should be tracking?

To switch it up I’d like to suggest Felix to be the RM for this since there are also likely some R packaging changes to be included in the release. This also gives us a chance to see if my updated release documentation if enough for a new RM to get started from.

What do folks think?
--
Reply | Threaded
Open this post in threaded view
|

RE: Kicking off the process around Spark 2.2.1

Kevin Grealish-2

Any update on expected 2.2.1 (or 2.3.0) release process?

 

From: Felix Cheung [mailto:[hidden email]]
Sent: Thursday, October 26, 2017 10:04 AM
To: Sean Owen <[hidden email]>; Holden Karau <[hidden email]>
Cc: [hidden email]
Subject: Re: Kicking off the process around Spark 2.2.1

 

Yes! I can take on RM for 2.2.1.

 

We are still working out what to do with temp files created by Hive and Java that cause the policy issue with CRAN and will report back shortly, hopefully.

 


From: Sean Owen <[hidden email]>
Sent: Wednesday, October 25, 2017 4:39:15 AM
To: Holden Karau
Cc: Felix Cheung; [hidden email]
Subject: Re: Kicking off the process around Spark 2.2.1

 

It would be reasonably consistent with the timing of other x.y.1 releases, and more release managers sounds useful, yeah.

 

Note also that in theory the code freeze for 2.3.0 starts in about 2 weeks.

 

On Wed, Oct 25, 2017 at 12:29 PM Holden Karau <[hidden email]> wrote:

Now that Spark 2.1.2 is out it seems like now is a good time to get started on the Spark 2.2.1 release. There are some streaming fixes I’m aware of that would be good to get into a release, is there anything else people are working on for 2.2.1 we should be tracking?

 

To switch it up I’d like to suggest Felix to be the RM for this since there are also likely some R packaging changes to be included in the release. This also gives us a chance to see if my updated release documentation if enough for a new RM to get started from.

 

What do folks think?

--

Reply | Threaded
Open this post in threaded view
|

Re: Kicking off the process around Spark 2.2.1

Felix Cheung
For the 2.2.1, we are still working through a few bugs. Hopefully it won't be long.



From: Kevin Grealish <[hidden email]>
Sent: Thursday, November 2, 2017 9:51:56 AM
To: Felix Cheung; Sean Owen; Holden Karau
Cc: [hidden email]
Subject: RE: Kicking off the process around Spark 2.2.1
 

Any update on expected 2.2.1 (or 2.3.0) release process?

 

From: Felix Cheung [mailto:[hidden email]]
Sent: Thursday, October 26, 2017 10:04 AM
To: Sean Owen <[hidden email]>; Holden Karau <[hidden email]>
Cc: [hidden email]
Subject: Re: Kicking off the process around Spark 2.2.1

 

Yes! I can take on RM for 2.2.1.

 

We are still working out what to do with temp files created by Hive and Java that cause the policy issue with CRAN and will report back shortly, hopefully.

 


From: Sean Owen <[hidden email]>
Sent: Wednesday, October 25, 2017 4:39:15 AM
To: Holden Karau
Cc: Felix Cheung; [hidden email]
Subject: Re: Kicking off the process around Spark 2.2.1

 

It would be reasonably consistent with the timing of other x.y.1 releases, and more release managers sounds useful, yeah.

 

Note also that in theory the code freeze for 2.3.0 starts in about 2 weeks.

 

On Wed, Oct 25, 2017 at 12:29 PM Holden Karau <[hidden email]> wrote:

Now that Spark 2.1.2 is out it seems like now is a good time to get started on the Spark 2.2.1 release. There are some streaming fixes I’m aware of that would be good to get into a release, is there anything else people are working on for 2.2.1 we should be tracking?

 

To switch it up I’d like to suggest Felix to be the RM for this since there are also likely some R packaging changes to be included in the release. This also gives us a chance to see if my updated release documentation if enough for a new RM to get started from.

 

What do folks think?

--

Reply | Threaded
Open this post in threaded view
|

Re: Kicking off the process around Spark 2.2.1

Holden Karau
If it’s desired I’d be happy to start on 2.3 once 2.2.1 is finished.

On Thu, Nov 2, 2017 at 10:24 AM Felix Cheung <[hidden email]> wrote:
For the 2.2.1, we are still working through a few bugs. Hopefully it won't be long.



From: Kevin Grealish <[hidden email]>
Sent: Thursday, November 2, 2017 9:51:56 AM
To: Felix Cheung; Sean Owen; Holden Karau
Cc: [hidden email]
Subject: RE: Kicking off the process around Spark 2.2.1
 

Any update on expected 2.2.1 (or 2.3.0) release process?

 

From: Felix Cheung [mailto:[hidden email]]
Sent: Thursday, October 26, 2017 10:04 AM
To: Sean Owen <[hidden email]>; Holden Karau <[hidden email]>
Cc: [hidden email]
Subject: Re: Kicking off the process around Spark 2.2.1

 

Yes! I can take on RM for 2.2.1.

 

We are still working out what to do with temp files created by Hive and Java that cause the policy issue with CRAN and will report back shortly, hopefully.

 


From: Sean Owen <[hidden email]>
Sent: Wednesday, October 25, 2017 4:39:15 AM
To: Holden Karau
Cc: Felix Cheung; [hidden email]
Subject: Re: Kicking off the process around Spark 2.2.1

 

It would be reasonably consistent with the timing of other x.y.1 releases, and more release managers sounds useful, yeah.

 

Note also that in theory the code freeze for 2.3.0 starts in about 2 weeks.

 

On Wed, Oct 25, 2017 at 12:29 PM Holden Karau <[hidden email]> wrote:

Now that Spark 2.1.2 is out it seems like now is a good time to get started on the Spark 2.2.1 release. There are some streaming fixes I’m aware of that would be good to get into a release, is there anything else people are working on for 2.2.1 we should be tracking?

 

To switch it up I’d like to suggest Felix to be the RM for this since there are also likely some R packaging changes to be included in the release. This also gives us a chance to see if my updated release documentation if enough for a new RM to get started from.

 

What do folks think?

--

--
Reply | Threaded
Open this post in threaded view
|

Re: Kicking off the process around Spark 2.2.1

Felix Cheung
I think it will be great to set a feature freeze date for 2.3.0 first, as a minor release. There are a few new stuff that would be good to have and then we will likely need time to stabilize, before cutting RCs.


From: Holden Karau <[hidden email]>
Sent: Thursday, November 2, 2017 10:38:48 AM
To: Felix Cheung; Kevin Grealish; Sean Owen
Cc: [hidden email]
Subject: Re: Kicking off the process around Spark 2.2.1
 
If it’s desired I’d be happy to start on 2.3 once 2.2.1 is finished.

On Thu, Nov 2, 2017 at 10:24 AM Felix Cheung <[hidden email]> wrote:
For the 2.2.1, we are still working through a few bugs. Hopefully it won't be long.



From: Kevin Grealish <[hidden email]>
Sent: Thursday, November 2, 2017 9:51:56 AM
To: Felix Cheung; Sean Owen; Holden Karau
Cc: [hidden email]
Subject: RE: Kicking off the process around Spark 2.2.1
 

Any update on expected 2.2.1 (or 2.3.0) release process?

 

From: Felix Cheung [mailto:[hidden email]]
Sent: Thursday, October 26, 2017 10:04 AM
To: Sean Owen <[hidden email]>; Holden Karau <[hidden email]>
Cc: [hidden email]
Subject: Re: Kicking off the process around Spark 2.2.1

 

Yes! I can take on RM for 2.2.1.

 

We are still working out what to do with temp files created by Hive and Java that cause the policy issue with CRAN and will report back shortly, hopefully.

 


From: Sean Owen <[hidden email]>
Sent: Wednesday, October 25, 2017 4:39:15 AM
To: Holden Karau
Cc: Felix Cheung; [hidden email]
Subject: Re: Kicking off the process around Spark 2.2.1

 

It would be reasonably consistent with the timing of other x.y.1 releases, and more release managers sounds useful, yeah.

 

Note also that in theory the code freeze for 2.3.0 starts in about 2 weeks.

 

On Wed, Oct 25, 2017 at 12:29 PM Holden Karau <[hidden email]> wrote:

Now that Spark 2.1.2 is out it seems like now is a good time to get started on the Spark 2.2.1 release. There are some streaming fixes I’m aware of that would be good to get into a release, is there anything else people are working on for 2.2.1 we should be tracking?

 

To switch it up I’d like to suggest Felix to be the RM for this since there are also likely some R packaging changes to be included in the release. This also gives us a chance to see if my updated release documentation if enough for a new RM to get started from.

 

What do folks think?

--

--
Reply | Threaded
Open this post in threaded view
|

Re: Kicking off the process around Spark 2.2.1

Holden Karau
I’m fine with picking a feature freeze, although then we should branch close to that point. Is there interest in still seeing 2.3 try and go out around the nominal schedule?

Personally, from a release stand point, I’d rather see 2.2.1 go out first so we don’t end up with 2.3 potentially going out with missing fixes from 2.2.1 (especially with the CRAN stuff - it would be unfortunate to have 2.2.1 available in CRAN but not be able to provide 2.3 in the same manner).

On Thu, Nov 2, 2017 at 11:06 AM Felix Cheung <[hidden email]> wrote:
I think it will be great to set a feature freeze date for 2.3.0 first, as a minor release. There are a few new stuff that would be good to have and then we will likely need time to stabilize, before cutting RCs.


From: Holden Karau <[hidden email]>
Sent: Thursday, November 2, 2017 10:38:48 AM
To: Felix Cheung; Kevin Grealish; Sean Owen

Cc: [hidden email]
Subject: Re: Kicking off the process around Spark 2.2.1
If it’s desired I’d be happy to start on 2.3 once 2.2.1 is finished.

On Thu, Nov 2, 2017 at 10:24 AM Felix Cheung <[hidden email]> wrote:
For the 2.2.1, we are still working through a few bugs. Hopefully it won't be long.



From: Kevin Grealish <[hidden email]>
Sent: Thursday, November 2, 2017 9:51:56 AM
To: Felix Cheung; Sean Owen; Holden Karau
Cc: [hidden email]
Subject: RE: Kicking off the process around Spark 2.2.1
 

Any update on expected 2.2.1 (or 2.3.0) release process?

 

From: Felix Cheung [mailto:[hidden email]]
Sent: Thursday, October 26, 2017 10:04 AM
To: Sean Owen <[hidden email]>; Holden Karau <[hidden email]>
Cc: [hidden email]
Subject: Re: Kicking off the process around Spark 2.2.1

 

Yes! I can take on RM for 2.2.1.

 

We are still working out what to do with temp files created by Hive and Java that cause the policy issue with CRAN and will report back shortly, hopefully.

 


From: Sean Owen <[hidden email]>
Sent: Wednesday, October 25, 2017 4:39:15 AM
To: Holden Karau
Cc: Felix Cheung; [hidden email]
Subject: Re: Kicking off the process around Spark 2.2.1

 

It would be reasonably consistent with the timing of other x.y.1 releases, and more release managers sounds useful, yeah.

 

Note also that in theory the code freeze for 2.3.0 starts in about 2 weeks.

 

On Wed, Oct 25, 2017 at 12:29 PM Holden Karau <[hidden email]> wrote:

Now that Spark 2.1.2 is out it seems like now is a good time to get started on the Spark 2.2.1 release. There are some streaming fixes I’m aware of that would be good to get into a release, is there anything else people are working on for 2.2.1 we should be tracking?

 

To switch it up I’d like to suggest Felix to be the RM for this since there are also likely some R packaging changes to be included in the release. This also gives us a chance to see if my updated release documentation if enough for a new RM to get started from.

 

What do folks think?

--

--
--
Reply | Threaded
Open this post in threaded view
|

Re: Kicking off the process around Spark 2.2.1

Sean Owen
In reply to this post by Felix Cheung
The feature freeze is "mid November" : http://spark.apache.org/versioning-policy.html
Let's say... Nov 15? any body have a better date?

Although it'd be nice to get 2.2.1 out sooner than later in all events, and kind of makes sense to get out first, they need not go in order. It just might be distracting to deal with 2 at once. 

(BTW there was still one outstanding issue from the last release: https://issues.apache.org/jira/browse/SPARK-22401 )

On Thu, Nov 2, 2017 at 6:06 PM Felix Cheung <[hidden email]> wrote:
I think it will be great to set a feature freeze date for 2.3.0 first, as a minor release. There are a few new stuff that would be good to have and then we will likely need time to stabilize, before cutting RCs.

Reply | Threaded
Open this post in threaded view
|

Re: Kicking off the process around Spark 2.2.1

rxin
Why tie a maintenance release to a feature release? They are supposed to be independent and we should be able to make a lot of maintenance releases as needed. 

On Thu, Nov 2, 2017 at 7:13 PM Sean Owen <[hidden email]> wrote:
The feature freeze is "mid November" : http://spark.apache.org/versioning-policy.html
Let's say... Nov 15? any body have a better date?

Although it'd be nice to get 2.2.1 out sooner than later in all events, and kind of makes sense to get out first, they need not go in order. It just might be distracting to deal with 2 at once. 

(BTW there was still one outstanding issue from the last release: https://issues.apache.org/jira/browse/SPARK-22401 )

On Thu, Nov 2, 2017 at 6:06 PM Felix Cheung <[hidden email]> wrote:
I think it will be great to set a feature freeze date for 2.3.0 first, as a minor release. There are a few new stuff that would be good to have and then we will likely need time to stabilize, before cutting RCs.

Reply | Threaded
Open this post in threaded view
|

Re: Kicking off the process around Spark 2.2.1

Holden Karau
I agree, except in this case we probably want some of the fixes that are going into the maintenance release to be present in the new feature release (like the CRAN issue).

On Thu, Nov 2, 2017 at 12:12 PM, Reynold Xin <[hidden email]> wrote:
Why tie a maintenance release to a feature release? They are supposed to be independent and we should be able to make a lot of maintenance releases as needed. 

On Thu, Nov 2, 2017 at 7:13 PM Sean Owen <[hidden email]> wrote:
The feature freeze is "mid November" : http://spark.apache.org/versioning-policy.html
Let's say... Nov 15? any body have a better date?

Although it'd be nice to get 2.2.1 out sooner than later in all events, and kind of makes sense to get out first, they need not go in order. It just might be distracting to deal with 2 at once. 

(BTW there was still one outstanding issue from the last release: https://issues.apache.org/jira/browse/SPARK-22401 )

On Thu, Nov 2, 2017 at 6:06 PM Felix Cheung <[hidden email]> wrote:
I think it will be great to set a feature freeze date for 2.3.0 first, as a minor release. There are a few new stuff that would be good to have and then we will likely need time to stabilize, before cutting RCs.




--
Reply | Threaded
Open this post in threaded view
|

Re: Kicking off the process around Spark 2.2.1

Felix Cheung
Ok I think we are there, after getting rounds of fixes through in the last few weeks.

I’m going to kick off a separate thread on this to be absolutely clear.


From: [hidden email] <[hidden email]> on behalf of Holden Karau <[hidden email]>
Sent: Thursday, November 2, 2017 12:47:13 PM
To: Reynold Xin
Cc: Felix Cheung; Sean Owen; [hidden email]
Subject: Re: Kicking off the process around Spark 2.2.1
 
I agree, except in this case we probably want some of the fixes that are going into the maintenance release to be present in the new feature release (like the CRAN issue).

On Thu, Nov 2, 2017 at 12:12 PM, Reynold Xin <[hidden email]> wrote:
Why tie a maintenance release to a feature release? They are supposed to be independent and we should be able to make a lot of maintenance releases as needed. 

On Thu, Nov 2, 2017 at 7:13 PM Sean Owen <[hidden email]> wrote:
The feature freeze is "mid November" : http://spark.apache.org/versioning-policy.html
Let's say... Nov 15? any body have a better date?

Although it'd be nice to get 2.2.1 out sooner than later in all events, and kind of makes sense to get out first, they need not go in order. It just might be distracting to deal with 2 at once. 

(BTW there was still one outstanding issue from the last release: https://issues.apache.org/jira/browse/SPARK-22401 )

On Thu, Nov 2, 2017 at 6:06 PM Felix Cheung <[hidden email]> wrote:
I think it will be great to set a feature freeze date for 2.3.0 first, as a minor release. There are a few new stuff that would be good to have and then we will likely need time to stabilize, before cutting RCs.




--