Cutting the RC for Spark 2.2.1 release

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

Cutting the RC for Spark 2.2.1 release

Felix Cheung
Hi!

As we are closing down on the few known issues I think we are ready to tag and cut the 2.2.1 release.

If you are aware of any issue that you think should go into this release please feel free to ping me and mark the JIRA as targeting 2.2.1. I will be scrubbing JIRA in the next few days.

So unless we hear otherwise, I’m going to tag and build the RC starting Saturday EOD (PT). Please be patient since I’m going to be new at this :) but will keep the dev@ posted for any update.

Yours
RM for 2.2.1


Reply | Threaded
Open this post in threaded view
|

Re: Cutting the RC for Spark 2.2.1 release

Holden Karau
Thanks for stepping up and running the 2.2.1 release :)

On Wed, Nov 8, 2017 at 3:57 PM Felix Cheung <[hidden email]> wrote:
Hi!

As we are closing down on the few known issues I think we are ready to tag and cut the 2.2.1 release.

If you are aware of any issue that you think should go into this release please feel free to ping me and mark the JIRA as targeting 2.2.1. I will be scrubbing JIRA in the next few days.

So unless we hear otherwise, I’m going to tag and build the RC starting Saturday EOD (PT). Please be patient since I’m going to be new at this :) but will keep the dev@ posted for any update.

Yours
RM for 2.2.1


--
Reply | Threaded
Open this post in threaded view
|

Re: Cutting the RC for Spark 2.2.1 release

Dongjoon Hyun-2
It's great, Felix!

As of today, `branch-2.2` seems to be broken due to SPARK-22211 (Scala UT failure) and SPARK-22417 (Python UT failure).
I pinged you at both.

Bests,
Dongjoon.


On Wed, Nov 8, 2017 at 5:51 PM, Holden Karau <[hidden email]> wrote:
Thanks for stepping up and running the 2.2.1 release :)

On Wed, Nov 8, 2017 at 3:57 PM Felix Cheung <[hidden email]> wrote:
Hi!

As we are closing down on the few known issues I think we are ready to tag and cut the 2.2.1 release.

If you are aware of any issue that you think should go into this release please feel free to ping me and mark the JIRA as targeting 2.2.1. I will be scrubbing JIRA in the next few days.

So unless we hear otherwise, I’m going to tag and build the RC starting Saturday EOD (PT). Please be patient since I’m going to be new at this :) but will keep the dev@ posted for any update.

Yours
RM for 2.2.1


--

Reply | Threaded
Open this post in threaded view
|

Re: Cutting the RC for Spark 2.2.1 release

Felix Cheung
Thanks Dongjoon! I will track that.


From: Dongjoon Hyun <[hidden email]>
Sent: Wednesday, November 8, 2017 7:41:20 PM
To: Holden Karau
Cc: Felix Cheung; [hidden email]
Subject: Re: Cutting the RC for Spark 2.2.1 release
 
It's great, Felix!

As of today, `branch-2.2` seems to be broken due to SPARK-22211 (Scala UT failure) and SPARK-22417 (Python UT failure).
I pinged you at both.

Bests,
Dongjoon.


On Wed, Nov 8, 2017 at 5:51 PM, Holden Karau <[hidden email]> wrote:
Thanks for stepping up and running the 2.2.1 release :)

On Wed, Nov 8, 2017 at 3:57 PM Felix Cheung <[hidden email]> wrote:
Hi!

As we are closing down on the few known issues I think we are ready to tag and cut the 2.2.1 release.

If you are aware of any issue that you think should go into this release please feel free to ping me and mark the JIRA as targeting 2.2.1. I will be scrubbing JIRA in the next few days.

So unless we hear otherwise, I’m going to tag and build the RC starting Saturday EOD (PT). Please be patient since I’m going to be new at this :) but will keep the dev@ posted for any update.

Yours
RM for 2.2.1


--

Reply | Threaded
Open this post in threaded view
|

Re: Cutting the RC for Spark 2.2.1 release

Felix Cheung
In reply to this post by Felix Cheung
Quick update:

We merged 6 fixes Friday and 7 fixes today (thanks!), since some are hand-merged I’m waiting for clean builds from Jenkins and test passes. As of now it looks like we need to take one more fix for Scala 2.10.

With any luck we should be tagging for build tomorrow morning (PT).

There should not be any issue targetting 2.2.1 except for SPARK-22042. As it is not a regression and it seems it might take a while, we won’t be blocking the release.
 
_____________________________
From: Felix Cheung <[hidden email]>
Sent: Wednesday, November 8, 2017 3:57 PM
Subject: Cutting the RC for Spark 2.2.1 release
To: <[hidden email]>


Hi!

As we are closing down on the few known issues I think we are ready to tag and cut the 2.2.1 release.

If you are aware of any issue that you think should go into this release please feel free to ping me and mark the JIRA as targeting 2.2.1. I will be scrubbing JIRA in the next few days.

So unless we hear otherwise, I’m going to tag and build the RC starting Saturday EOD (PT). Please be patient since I’m going to be new at this :) but will keep the dev@ posted for any update.

Yours
RM for 2.2.1




Reply | Threaded
Open this post in threaded view
|

Re: Cutting the RC for Spark 2.2.1 release

Felix Cheung
Build/test looks good but I’m hitting a new issue with sonatype when tagging

"Host name 'repo1.maven.org' does not match the certificate subject provided by the peer (CN=repo.maven.apache.org, O="Sonatype, Inc", L=Fulton, ST=MD, C=US)"

https://issues.sonatype.org/browse/MVNCENTRAL-1369

Stay tuned.


From: Felix Cheung <[hidden email]>
Sent: Monday, November 13, 2017 12:00:41 AM
To: [hidden email]
Subject: Re: Cutting the RC for Spark 2.2.1 release
 
Quick update:

We merged 6 fixes Friday and 7 fixes today (thanks!), since some are hand-merged I’m waiting for clean builds from Jenkins and test passes. As of now it looks like we need to take one more fix for Scala 2.10.

With any luck we should be tagging for build tomorrow morning (PT).

There should not be any issue targetting 2.2.1 except for SPARK-22042. As it is not a regression and it seems it might take a while, we won’t be blocking the release.
 
_____________________________
From: Felix Cheung <[hidden email]>
Sent: Wednesday, November 8, 2017 3:57 PM
Subject: Cutting the RC for Spark 2.2.1 release
To: <[hidden email]>


Hi!

As we are closing down on the few known issues I think we are ready to tag and cut the 2.2.1 release.

If you are aware of any issue that you think should go into this release please feel free to ping me and mark the JIRA as targeting 2.2.1. I will be scrubbing JIRA in the next few days.

So unless we hear otherwise, I’m going to tag and build the RC starting Saturday EOD (PT). Please be patient since I’m going to be new at this :) but will keep the dev@ posted for any update.

Yours
RM for 2.2.1




Reply | Threaded
Open this post in threaded view
|

Re: Cutting the RC for Spark 2.2.1 release

Holden Karau
Which script is this from?

On Mon, Nov 13, 2017 at 10:37 AM Felix Cheung <[hidden email]> wrote:
Build/test looks good but I’m hitting a new issue with sonatype when tagging

"Host name 'repo1.maven.org' does not match the certificate subject provided by the peer (CN=repo.maven.apache.org, O="Sonatype, Inc", L=Fulton, ST=MD, C=US)"


Stay tuned.


From: Felix Cheung <[hidden email]>
Sent: Monday, November 13, 2017 12:00:41 AM
To: [hidden email]
Subject: Re: Cutting the RC for Spark 2.2.1 release
 
Quick update:

We merged 6 fixes Friday and 7 fixes today (thanks!), since some are hand-merged I’m waiting for clean builds from Jenkins and test passes. As of now it looks like we need to take one more fix for Scala 2.10.

With any luck we should be tagging for build tomorrow morning (PT).

There should not be any issue targetting 2.2.1 except for SPARK-22042. As it is not a regression and it seems it might take a while, we won’t be blocking the release.
 
_____________________________
From: Felix Cheung <[hidden email]>
Sent: Wednesday, November 8, 2017 3:57 PM
Subject: Cutting the RC for Spark 2.2.1 release
To: <[hidden email]>


Hi!

As we are closing down on the few known issues I think we are ready to tag and cut the 2.2.1 release.

If you are aware of any issue that you think should go into this release please feel free to ping me and mark the JIRA as targeting 2.2.1. I will be scrubbing JIRA in the next few days.

So unless we hear otherwise, I’m going to tag and build the RC starting Saturday EOD (PT). Please be patient since I’m going to be new at this :) but will keep the dev@ posted for any update.

Yours
RM for 2.2.1




--
Reply | Threaded
Open this post in threaded view
|

Re: Cutting the RC for Spark 2.2.1 release

Felix Cheung
Anything to build with maven on a clean machine.
It couldn’t connect to maven central repo.


From: Holden Karau <[hidden email]>
Sent: Monday, November 13, 2017 10:38:03 AM
To: Felix Cheung
Cc: [hidden email]
Subject: Re: Cutting the RC for Spark 2.2.1 release
 
Which script is this from?

On Mon, Nov 13, 2017 at 10:37 AM Felix Cheung <[hidden email]> wrote:
Build/test looks good but I’m hitting a new issue with sonatype when tagging

"Host name 'repo1.maven.org' does not match the certificate subject provided by the peer (CN=repo.maven.apache.org, O="Sonatype, Inc", L=Fulton, ST=MD, C=US)"


Stay tuned.


From: Felix Cheung <[hidden email]>
Sent: Monday, November 13, 2017 12:00:41 AM
To: [hidden email]
Subject: Re: Cutting the RC for Spark 2.2.1 release
 
Quick update:

We merged 6 fixes Friday and 7 fixes today (thanks!), since some are hand-merged I’m waiting for clean builds from Jenkins and test passes. As of now it looks like we need to take one more fix for Scala 2.10.

With any luck we should be tagging for build tomorrow morning (PT).

There should not be any issue targetting 2.2.1 except for SPARK-22042. As it is not a regression and it seems it might take a while, we won’t be blocking the release.
 
_____________________________
From: Felix Cheung <[hidden email]>
Sent: Wednesday, November 8, 2017 3:57 PM
Subject: Cutting the RC for Spark 2.2.1 release
To: <[hidden email]>


Hi!

As we are closing down on the few known issues I think we are ready to tag and cut the 2.2.1 release.

If you are aware of any issue that you think should go into this release please feel free to ping me and mark the JIRA as targeting 2.2.1. I will be scrubbing JIRA in the next few days.

So unless we hear otherwise, I’m going to tag and build the RC starting Saturday EOD (PT). Please be patient since I’m going to be new at this :) but will keep the dev@ posted for any update.

Yours
RM for 2.2.1




--
Reply | Threaded
Open this post in threaded view
|

Re: Cutting the RC for Spark 2.2.1 release

Sean Owen
I'm not seeing a problem building, myself. However we could change the location of the Maven Repository in our POM to https://repo.maven.apache.org/maven2/ without any consequence.

The only reason we overrode it was to force it to use HTTPS which still doesn't look like the default (!): https://maven.apache.org/guides/introduction/introduction-to-the-pom.html#Super_POM

On a related note, we could also update the POM to inherit from the latest Apache parent POM, while we're at it, to get the latest declarations relevant to the ASF. Doesn't need to happen in 2.2.x

On Mon, Nov 13, 2017 at 12:39 PM Felix Cheung <[hidden email]> wrote:
Anything to build with maven on a clean machine.
It couldn’t connect to maven central repo.

Reply | Threaded
Open this post in threaded view
|

Re: Cutting the RC for Spark 2.2.1 release

Felix Cheung
I did change it, but getting unknown host?

[ERROR] Non-resolvable parent POM for org.apache.spark:spark-parent_2.11:2.2.1-SNAPSHOT: Could not transfer artifact org.apache:apache:pom:14 from/to central (https://repo.maven.org/maven2): repo.maven.org: Name or service not known and 'parent.relativePath' points at wrong local POM @ line 22, column 11: Unknown host repo.maven.org: Name or service not known -> [Help 2]

_____________________________
From: Sean Owen <[hidden email]>
Sent: Monday, November 13, 2017 10:48 AM
Subject: Re: Cutting the RC for Spark 2.2.1 release
To: Felix Cheung <[hidden email]>
Cc: Holden Karau <[hidden email]>, <[hidden email]>


I'm not seeing a problem building, myself. However we could change the location of the Maven Repository in our POM to https://repo.maven.apache.org/maven2/ without any consequence.

The only reason we overrode it was to force it to use HTTPS which still doesn't look like the default (!): https://maven.apache.org/guides/introduction/introduction-to-the-pom.html#Super_POM

On a related note, we could also update the POM to inherit from the latest Apache parent POM, while we're at it, to get the latest declarations relevant to the ASF. Doesn't need to happen in 2.2.x

On Mon, Nov 13, 2017 at 12:39 PM Felix Cheung <[hidden email]> wrote:
Anything to build with maven on a clean machine.
It couldn’t connect to maven central repo.



Reply | Threaded
Open this post in threaded view
|

Re: Cutting the RC for Spark 2.2.1 release

Sean Owen
It's repo.maven.apache.org ?

On Mon, Nov 13, 2017 at 12:52 PM Felix Cheung <[hidden email]> wrote:
I did change it, but getting unknown host?

[ERROR] Non-resolvable parent POM for org.apache.spark:spark-parent_2.11:2.2.1-SNAPSHOT: Could not transfer artifact org.apache:apache:pom:14 from/to central (https://repo.maven.org/maven2): repo.maven.org: Name or service not known and 'parent.relativePath' points at wrong local POM @ line 22, column 11: Unknown host repo.maven.org: Name or service not known -> [Help 2]


Reply | Threaded
Open this post in threaded view
|

Re: Cutting the RC for Spark 2.2.1 release

Felix Cheung
Ouch ;) yes that works and RC1 is tagged.



From: Sean Owen <[hidden email]>
Sent: Monday, November 13, 2017 10:54:48 AM
To: Felix Cheung
Cc: Holden Karau; [hidden email]
Subject: Re: Cutting the RC for Spark 2.2.1 release
 
It's repo.maven.apache.org ?

On Mon, Nov 13, 2017 at 12:52 PM Felix Cheung <[hidden email]> wrote:
I did change it, but getting unknown host?

[ERROR] Non-resolvable parent POM for org.apache.spark:spark-parent_2.11:2.2.1-SNAPSHOT: Could not transfer artifact org.apache:apache:pom:14 from/to central (https://repo.maven.org/maven2): repo.maven.org: Name or service not known and 'parent.relativePath' points at wrong local POM @ line 22, column 11: Unknown host repo.maven.org: Name or service not known -> [Help 2]


Reply | Threaded
Open this post in threaded view
|

Re: Cutting the RC for Spark 2.2.1 release

Felix Cheung
Now I’m seeing an error with Closing nexus staging repository.

staged_repo_id=orgapachespark-1254

< HTTP/1.1 401 Unauthorized
< Date: Tue, 14 Nov 2017 12:32:57 GMT
< Server: Nexus/2.13.0-01
< X-Frame-Options: SAMEORIGIN
< X-Content-Type-Options: nosniff
* Authentication problem. Ignoring this.
< WWW-Authenticate: BASIC realm="Sonatype Nexus Repository Manager API"
< Content-Length: 0

Does working with Nexus require special permission in LDAP?
I couldn’t login to the web interface at repository.apache.org either.

From: Felix Cheung <[hidden email]>
Sent: Monday, November 13, 2017 11:23:44 AM
To: Sean Owen
Cc: Holden Karau; [hidden email]
Subject: Re: Cutting the RC for Spark 2.2.1 release
 
Ouch ;) yes that works and RC1 is tagged.



From: Sean Owen <[hidden email]>
Sent: Monday, November 13, 2017 10:54:48 AM
To: Felix Cheung
Cc: Holden Karau; [hidden email]
Subject: Re: Cutting the RC for Spark 2.2.1 release
 
It's repo.maven.apache.org ?

On Mon, Nov 13, 2017 at 12:52 PM Felix Cheung <[hidden email]> wrote:
I did change it, but getting unknown host?

[ERROR] Non-resolvable parent POM for org.apache.spark:spark-parent_2.11:2.2.1-SNAPSHOT: Could not transfer artifact org.apache:apache:pom:14 from/to central (https://repo.maven.org/maven2): repo.maven.org: Name or service not known and 'parent.relativePath' points at wrong local POM @ line 22, column 11: Unknown host repo.maven.org: Name or service not known -> [Help 2]