[VOTE] Spark 2.1.2 (RC2)

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

[VOTE] Spark 2.1.2 (RC2)

Holden Karau
Please vote on releasing the following candidate as Apache Spark version 2.1.2. The vote is open until Wednesday October 4th at 23:59 PST and passes if a majority of at least 3 +1 PMC votes are cast.

[ ] +1 Release this package as Apache Spark 2.1.2
[ ] -1 Do not release this package because ...


To learn more about Apache Spark, please see https://spark.apache.org/

The tag to be voted on is v2.1.2-rc2 (fabbb7f59e47590114366d14e15fbbff8c88593c)

List of JIRA tickets resolved in this release can be found with this filter.

The release files, including signatures, digests, etc. can be found at:

Release artifacts are signed with a key from:

The staging repository for this release can be found at:

The documentation corresponding to this release can be found at:


FAQ

How can I help test this release?

If you are a Spark user, you can help us test this release by taking an existing Spark workload and running on this release candidate, then reporting any regressions.

If you're working in PySpark you can set up a virtual env and install the current RC and see if anything important breaks, in the Java/Scala you can add the staging repository to your projects resolvers and test with the RC (make sure to clean up the artifact cache before/after so you don't end up building with a out of date RC going forward).

What should happen to JIRA tickets still targeting 2.1.2?

Committers should look at those and triage. Extremely important bug fixes, documentation, and API tweaks that impact compatibility should be worked on immediately. Everything else please retarget to 2.1.3.

But my bug isn't fixed!??!

In order to make timely releases, we will typically not hold the release unless the bug in question is a regression from 2.1.1. That being said if there is something which is a regression form 2.1.1 that has not been correctly targeted please ping a committer to help target the issue (you can see the open issues listed as impacting Spark 2.1.1 & 2.1.2)

What are the unresolved issues targeted for 2.1.2?

At this time there are no open unresolved issues.

Is there anything different about this release?

This is the first release in awhile not built on the AMPLAB Jenkins. This is good because it means future releases can more easily be built and signed securely (and I've been updating the documentation in https://github.com/apache/spark-website/pull/66 as I progress), however the chances of a mistake are higher with any change like this. If there something you normally take for granted as correct when checking a release, please double check this time :)

Should I be committing code to branch-2.1?

Thanks for asking! Please treat this stage in the RC process as "code freeze" so bug fixes only. If you're uncertain if something should be back ported please reach out. If you do commit to branch-2.1 please tag your JIRA issue fix version for 2.1.3 and if we cut another RC I'll move the 2.1.3 fixed into 2.1.2 as appropriate.

Why the longer voting window?

Since there is a large industry big data conference this week I figured I'd add a little bit of extra buffer time just to make sure everyone has a chance to take a look.

--
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] Spark 2.1.2 (RC2)

Sean Owen
+1

I tested the source release.
Hashes and signature (your signature) check out, project builds and tests pass with -Phadoop-2.7 -Pyarn -Phive -Pmesos on Debian 9.
List of issues look good and there are no open issues at all for 2.1.2.

Great work on improving the build process and docs.


On Wed, Sep 27, 2017 at 5:47 AM Holden Karau <[hidden email]> wrote:
Please vote on releasing the following candidate as Apache Spark version 2.1.2. The vote is open until Wednesday October 4th at 23:59 PST and passes if a majority of at least 3 +1 PMC votes are cast.

[ ] +1 Release this package as Apache Spark 2.1.2
[ ] -1 Do not release this package because ...


To learn more about Apache Spark, please see https://spark.apache.org/

The tag to be voted on is v2.1.2-rc2 (fabbb7f59e47590114366d14e15fbbff8c88593c)

List of JIRA tickets resolved in this release can be found with this filter.

The release files, including signatures, digests, etc. can be found at:

Release artifacts are signed with a key from:

The staging repository for this release can be found at:

The documentation corresponding to this release can be found at:


FAQ

How can I help test this release?

If you are a Spark user, you can help us test this release by taking an existing Spark workload and running on this release candidate, then reporting any regressions.

If you're working in PySpark you can set up a virtual env and install the current RC and see if anything important breaks, in the Java/Scala you can add the staging repository to your projects resolvers and test with the RC (make sure to clean up the artifact cache before/after so you don't end up building with a out of date RC going forward).

What should happen to JIRA tickets still targeting 2.1.2?

Committers should look at those and triage. Extremely important bug fixes, documentation, and API tweaks that impact compatibility should be worked on immediately. Everything else please retarget to 2.1.3.

But my bug isn't fixed!??!

In order to make timely releases, we will typically not hold the release unless the bug in question is a regression from 2.1.1. That being said if there is something which is a regression form 2.1.1 that has not been correctly targeted please ping a committer to help target the issue (you can see the open issues listed as impacting Spark 2.1.1 & 2.1.2)

What are the unresolved issues targeted for 2.1.2?

At this time there are no open unresolved issues.

Is there anything different about this release?

This is the first release in awhile not built on the AMPLAB Jenkins. This is good because it means future releases can more easily be built and signed securely (and I've been updating the documentation in https://github.com/apache/spark-website/pull/66 as I progress), however the chances of a mistake are higher with any change like this. If there something you normally take for granted as correct when checking a release, please double check this time :)

Should I be committing code to branch-2.1?

Thanks for asking! Please treat this stage in the RC process as "code freeze" so bug fixes only. If you're uncertain if something should be back ported please reach out. If you do commit to branch-2.1 please tag your JIRA issue fix version for 2.1.3 and if we cut another RC I'll move the 2.1.3 fixed into 2.1.2 as appropriate.

Why the longer voting window?

Since there is a large industry big data conference this week I figured I'd add a little bit of extra buffer time just to make sure everyone has a chance to take a look.
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] Spark 2.1.2 (RC2)

Denny Lee
+1 (non-binding)


On Wed, Sep 27, 2017 at 6:54 AM Sean Owen <[hidden email]> wrote:
+1

I tested the source release.
Hashes and signature (your signature) check out, project builds and tests pass with -Phadoop-2.7 -Pyarn -Phive -Pmesos on Debian 9.
List of issues look good and there are no open issues at all for 2.1.2.

Great work on improving the build process and docs.


On Wed, Sep 27, 2017 at 5:47 AM Holden Karau <[hidden email]> wrote:
Please vote on releasing the following candidate as Apache Spark version 2.1.2. The vote is open until Wednesday October 4th at 23:59 PST and passes if a majority of at least 3 +1 PMC votes are cast.

[ ] +1 Release this package as Apache Spark 2.1.2
[ ] -1 Do not release this package because ...


To learn more about Apache Spark, please see https://spark.apache.org/

The tag to be voted on is v2.1.2-rc2 (fabbb7f59e47590114366d14e15fbbff8c88593c)

List of JIRA tickets resolved in this release can be found with this filter.

The release files, including signatures, digests, etc. can be found at:

Release artifacts are signed with a key from:

The staging repository for this release can be found at:

The documentation corresponding to this release can be found at:


FAQ

How can I help test this release?

If you are a Spark user, you can help us test this release by taking an existing Spark workload and running on this release candidate, then reporting any regressions.

If you're working in PySpark you can set up a virtual env and install the current RC and see if anything important breaks, in the Java/Scala you can add the staging repository to your projects resolvers and test with the RC (make sure to clean up the artifact cache before/after so you don't end up building with a out of date RC going forward).

What should happen to JIRA tickets still targeting 2.1.2?

Committers should look at those and triage. Extremely important bug fixes, documentation, and API tweaks that impact compatibility should be worked on immediately. Everything else please retarget to 2.1.3.

But my bug isn't fixed!??!

In order to make timely releases, we will typically not hold the release unless the bug in question is a regression from 2.1.1. That being said if there is something which is a regression form 2.1.1 that has not been correctly targeted please ping a committer to help target the issue (you can see the open issues listed as impacting Spark 2.1.1 & 2.1.2)

What are the unresolved issues targeted for 2.1.2?

At this time there are no open unresolved issues.

Is there anything different about this release?

This is the first release in awhile not built on the AMPLAB Jenkins. This is good because it means future releases can more easily be built and signed securely (and I've been updating the documentation in https://github.com/apache/spark-website/pull/66 as I progress), however the chances of a mistake are higher with any change like this. If there something you normally take for granted as correct when checking a release, please double check this time :)

Should I be committing code to branch-2.1?

Thanks for asking! Please treat this stage in the RC process as "code freeze" so bug fixes only. If you're uncertain if something should be back ported please reach out. If you do commit to branch-2.1 please tag your JIRA issue fix version for 2.1.3 and if we cut another RC I'll move the 2.1.3 fixed into 2.1.2 as appropriate.

Why the longer voting window?

Since there is a large industry big data conference this week I figured I'd add a little bit of extra buffer time just to make sure everyone has a chance to take a look.
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] Spark 2.1.2 (RC2)

Dongjoon Hyun-2
+1 (non-binding)

Bests,
Dongjoon.


On Wed, Sep 27, 2017 at 7:54 AM, Denny Lee <[hidden email]> wrote:
+1 (non-binding)


On Wed, Sep 27, 2017 at 6:54 AM Sean Owen <[hidden email]> wrote:
+1

I tested the source release.
Hashes and signature (your signature) check out, project builds and tests pass with -Phadoop-2.7 -Pyarn -Phive -Pmesos on Debian 9.
List of issues look good and there are no open issues at all for 2.1.2.

Great work on improving the build process and docs.


On Wed, Sep 27, 2017 at 5:47 AM Holden Karau <[hidden email]> wrote:
Please vote on releasing the following candidate as Apache Spark version 2.1.2. The vote is open until Wednesday October 4th at 23:59 PST and passes if a majority of at least 3 +1 PMC votes are cast.

[ ] +1 Release this package as Apache Spark 2.1.2
[ ] -1 Do not release this package because ...


To learn more about Apache Spark, please see https://spark.apache.org/

The tag to be voted on is v2.1.2-rc2 (fabbb7f59e47590114366d14e15fbbff8c88593c)

List of JIRA tickets resolved in this release can be found with this filter.

The release files, including signatures, digests, etc. can be found at:

Release artifacts are signed with a key from:

The staging repository for this release can be found at:

The documentation corresponding to this release can be found at:


FAQ

How can I help test this release?

If you are a Spark user, you can help us test this release by taking an existing Spark workload and running on this release candidate, then reporting any regressions.

If you're working in PySpark you can set up a virtual env and install the current RC and see if anything important breaks, in the Java/Scala you can add the staging repository to your projects resolvers and test with the RC (make sure to clean up the artifact cache before/after so you don't end up building with a out of date RC going forward).

What should happen to JIRA tickets still targeting 2.1.2?

Committers should look at those and triage. Extremely important bug fixes, documentation, and API tweaks that impact compatibility should be worked on immediately. Everything else please retarget to 2.1.3.

But my bug isn't fixed!??!

In order to make timely releases, we will typically not hold the release unless the bug in question is a regression from 2.1.1. That being said if there is something which is a regression form 2.1.1 that has not been correctly targeted please ping a committer to help target the issue (you can see the open issues listed as impacting Spark 2.1.1 & 2.1.2)

What are the unresolved issues targeted for 2.1.2?

At this time there are no open unresolved issues.

Is there anything different about this release?

This is the first release in awhile not built on the AMPLAB Jenkins. This is good because it means future releases can more easily be built and signed securely (and I've been updating the documentation in https://github.com/apache/spark-website/pull/66 as I progress), however the chances of a mistake are higher with any change like this. If there something you normally take for granted as correct when checking a release, please double check this time :)

Should I be committing code to branch-2.1?

Thanks for asking! Please treat this stage in the RC process as "code freeze" so bug fixes only. If you're uncertain if something should be back ported please reach out. If you do commit to branch-2.1 please tag your JIRA issue fix version for 2.1.3 and if we cut another RC I'll move the 2.1.3 fixed into 2.1.2 as appropriate.

Why the longer voting window?

Since there is a large industry big data conference this week I figured I'd add a little bit of extra buffer time just to make sure everyone has a chance to take a look.

Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] Spark 2.1.2 (RC2)

Kazuaki Ishizaki
+1 (non-binding)

I tested it on Ubuntu 16.04 and OpenJDK8 on ppc64le. All of the tests for core/sql-core/sql-catalyst/mllib/mllib-local have passed.

$ java -version
openjdk version "1.8.0_131"
OpenJDK Runtime Environment (build 1.8.0_131-8u131-b11-2ubuntu1.16.04.3-b11)
OpenJDK 64-Bit Server VM (build 25.131-b11, mixed mode)

% build/mvn -DskipTests -Phive -Phive-thriftserver -Pyarn -Phadoop-2.7 -T 24 clean package install
% build/mvn -Phive -Phive-thriftserver -Pyarn -Phadoop-2.7 test -pl core -pl 'sql/core' -pl 'sql/catalyst' -pl mllib -pl mllib-local
...
Run completed in 12 minutes, 42 seconds.
Total number of tests run: 1035
Suites: completed 166, aborted 0
Tests: succeeded 1035, failed 0, canceled 0, ignored 5, pending 0
All tests passed.
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Spark Project Core ................................. SUCCESS [17:14 min]
[INFO] Spark Project ML Local Library ..................... SUCCESS [  4.067 s]
[INFO] Spark Project Catalyst ............................. SUCCESS [08:23 min]
[INFO] Spark Project SQL .................................. SUCCESS [10:50 min]
[INFO] Spark Project ML Library ........................... SUCCESS [15:45 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 52:20 min
[INFO] Finished at: 2017-09-28T12:16:46+09:00
[INFO] Final Memory: 103M/309M
[INFO] ------------------------------------------------------------------------
[WARNING] The requested profile "hive" could not be activated because it does not exist.

Kazuaki Ishizaki



From:        Dongjoon Hyun <[hidden email]>
To:        Denny Lee <[hidden email]>
Cc:        Sean Owen <[hidden email]>, Holden Karau <[hidden email]>, "[hidden email]" <[hidden email]>
Date:        2017/09/28 07:57
Subject:        Re: [VOTE] Spark 2.1.2 (RC2)




+1 (non-binding)

Bests,
Dongjoon.


On Wed, Sep 27, 2017 at 7:54 AM, Denny Lee <denny.g.lee@...> wrote:
+1 (non-binding)


On Wed, Sep 27, 2017 at 6:54 AM Sean Owen <sowen@...> wrote:
+1

I tested the source release.
Hashes and signature (your signature) check out, project builds and tests pass with -Phadoop-2.7 -Pyarn -Phive -Pmesos on Debian 9.
List of issues look good and there are no open issues at all for 2.1.2.

Great work on improving the build process and docs.


On Wed, Sep 27, 2017 at 5:47 AM Holden Karau <holden@...> wrote:
Please vote on releasing the following candidate as Apache Spark version 2.1.2. The vote is open until Wednesday October 4th at 23:59 PST and passes if a majority of at least 3 +1 PMC votes are cast.

[ ] +1 Release this package as Apache Spark 2.1.2
[ ] -1 Do not release this package because ...


To learn more about Apache Spark, please see https://spark.apache.org/

The tag to be voted on is v2.1.2-rc2 (fabbb7f59e47590114366d14e15fbbff8c88593c)

List of JIRA tickets resolved in this release can be found with this filter.

The release files, including signatures, digests, etc. can be found at:
https://home.apache.org/~holden/spark-2.1.2-rc2-bin/

Release artifacts are signed with a key from:
https://people.apache.org/~holden/holdens_keys.asc

The staging repository for this release can be found at:
https://repository.apache.org/content/repositories/orgapachespark-1251

The documentation corresponding to this release can be found at:
https://people.apache.org/~holden/spark-2.1.2-rc2-docs/


FAQ

How can I help test this release?

If you are a Spark user, you can help us test this release by taking an existing Spark workload and running on this release candidate, then reporting any regressions.

If you're working in PySpark you can set up a virtual env and install the current RC and see if anything important breaks, in the Java/Scala you can add the staging repository to your projects resolvers and test with the RC (make sure to clean up the artifact cache before/after so you don't end up building with a out of date RC going forward).

What should happen to JIRA tickets still targeting 2.1.2?

Committers should look at those and triage. Extremely important bug fixes, documentation, and API tweaks that impact compatibility should be worked on immediately. Everything else please retarget to 2.1.3.

But my bug isn't fixed!??!

In order to make timely releases, we will typically not hold the release unless the bug in question is a regression from 2.1.1. That being said if there is something which is a regression form 2.1.1 that has not been correctly targeted please ping a committer to help target the issue (you can see the open issues listed as impacting Spark 2.1.1 & 2.1.2)

What are the unresolved issues targeted for 2.1.2?

At this time there are no open unresolved issues.

Is there anything different about this release?

This is the first release in awhile not built on the AMPLAB Jenkins. This is good because it means future releases can more easily be built and signed securely (and I've been updating the documentation in https://github.com/apache/spark-website/pull/66as I progress), however the chances of a mistake are higher with any change like this. If there something you normally take for granted as correct when checking a release, please double check this time :)

Should I be committing code to branch-2.1?

Thanks for asking! Please treat this stage in the RC process as "code freeze" so bug fixes only. If you're uncertain if something should be back ported please reach out. If you do commit to branch-2.1 please tag your JIRA issue fix version for 2.1.3 and if we cut another RC I'll move the 2.1.3 fixed into 2.1.2 as appropriate.

Why the longer voting window?

Since there is a large industry big data conference this week I figured I'd add a little bit of extra buffer time just to make sure everyone has a chance to take a look.

--
Twitter: https://twitter.com/holdenkarau


Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] Spark 2.1.2 (RC2)

rxin
In reply to this post by Holden Karau
+1


On Tue, Sep 26, 2017 at 9:47 PM, Holden Karau <[hidden email]> wrote:
Please vote on releasing the following candidate as Apache Spark version 2.1.2. The vote is open until Wednesday October 4th at 23:59 PST and passes if a majority of at least 3 +1 PMC votes are cast.

[ ] +1 Release this package as Apache Spark 2.1.2
[ ] -1 Do not release this package because ...


To learn more about Apache Spark, please see https://spark.apache.org/

The tag to be voted on is v2.1.2-rc2 (fabbb7f59e47590114366d14e15fbbff8c88593c)

List of JIRA tickets resolved in this release can be found with this filter.

The release files, including signatures, digests, etc. can be found at:

Release artifacts are signed with a key from:

The staging repository for this release can be found at:

The documentation corresponding to this release can be found at:


FAQ

How can I help test this release?

If you are a Spark user, you can help us test this release by taking an existing Spark workload and running on this release candidate, then reporting any regressions.

If you're working in PySpark you can set up a virtual env and install the current RC and see if anything important breaks, in the Java/Scala you can add the staging repository to your projects resolvers and test with the RC (make sure to clean up the artifact cache before/after so you don't end up building with a out of date RC going forward).

What should happen to JIRA tickets still targeting 2.1.2?

Committers should look at those and triage. Extremely important bug fixes, documentation, and API tweaks that impact compatibility should be worked on immediately. Everything else please retarget to 2.1.3.

But my bug isn't fixed!??!

In order to make timely releases, we will typically not hold the release unless the bug in question is a regression from 2.1.1. That being said if there is something which is a regression form 2.1.1 that has not been correctly targeted please ping a committer to help target the issue (you can see the open issues listed as impacting Spark 2.1.1 & 2.1.2)

What are the unresolved issues targeted for 2.1.2?

At this time there are no open unresolved issues.

Is there anything different about this release?

This is the first release in awhile not built on the AMPLAB Jenkins. This is good because it means future releases can more easily be built and signed securely (and I've been updating the documentation in https://github.com/apache/spark-website/pull/66 as I progress), however the chances of a mistake are higher with any change like this. If there something you normally take for granted as correct when checking a release, please double check this time :)

Should I be committing code to branch-2.1?

Thanks for asking! Please treat this stage in the RC process as "code freeze" so bug fixes only. If you're uncertain if something should be back ported please reach out. If you do commit to branch-2.1 please tag your JIRA issue fix version for 2.1.3 and if we cut another RC I'll move the 2.1.3 fixed into 2.1.2 as appropriate.

Why the longer voting window?

Since there is a large industry big data conference this week I figured I'd add a little bit of extra buffer time just to make sure everyone has a chance to take a look.

--

Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] Spark 2.1.2 (RC2)

Luciano Resende
In reply to this post by Holden Karau
+1 (non-binding)

Minor comments:
The apache infra has a staging repository to add release candidates, and it might be better/simpler to use that instead of home.a.o. See https://dist.apache.org/repos/dist/dev/spark/.



On Tue, Sep 26, 2017 at 9:47 PM, Holden Karau <[hidden email]> wrote:
Please vote on releasing the following candidate as Apache Spark version 2.1.2. The vote is open until Wednesday October 4th at 23:59 PST and passes if a majority of at least 3 +1 PMC votes are cast.

[ ] +1 Release this package as Apache Spark 2.1.2
[ ] -1 Do not release this package because ...


To learn more about Apache Spark, please see https://spark.apache.org/

The tag to be voted on is v2.1.2-rc2 (fabbb7f59e47590114366d14e15fbbff8c88593c)

List of JIRA tickets resolved in this release can be found with this filter.

The release files, including signatures, digests, etc. can be found at:

Release artifacts are signed with a key from:

The staging repository for this release can be found at:

The documentation corresponding to this release can be found at:


FAQ

How can I help test this release?

If you are a Spark user, you can help us test this release by taking an existing Spark workload and running on this release candidate, then reporting any regressions.

If you're working in PySpark you can set up a virtual env and install the current RC and see if anything important breaks, in the Java/Scala you can add the staging repository to your projects resolvers and test with the RC (make sure to clean up the artifact cache before/after so you don't end up building with a out of date RC going forward).

What should happen to JIRA tickets still targeting 2.1.2?

Committers should look at those and triage. Extremely important bug fixes, documentation, and API tweaks that impact compatibility should be worked on immediately. Everything else please retarget to 2.1.3.

But my bug isn't fixed!??!

In order to make timely releases, we will typically not hold the release unless the bug in question is a regression from 2.1.1. That being said if there is something which is a regression form 2.1.1 that has not been correctly targeted please ping a committer to help target the issue (you can see the open issues listed as impacting Spark 2.1.1 & 2.1.2)

What are the unresolved issues targeted for 2.1.2?

At this time there are no open unresolved issues.

Is there anything different about this release?

This is the first release in awhile not built on the AMPLAB Jenkins. This is good because it means future releases can more easily be built and signed securely (and I've been updating the documentation in https://github.com/apache/spark-website/pull/66 as I progress), however the chances of a mistake are higher with any change like this. If there something you normally take for granted as correct when checking a release, please double check this time :)

Should I be committing code to branch-2.1?

Thanks for asking! Please treat this stage in the RC process as "code freeze" so bug fixes only. If you're uncertain if something should be back ported please reach out. If you do commit to branch-2.1 please tag your JIRA issue fix version for 2.1.3 and if we cut another RC I'll move the 2.1.3 fixed into 2.1.2 as appropriate.

Why the longer voting window?

Since there is a large industry big data conference this week I figured I'd add a little bit of extra buffer time just to make sure everyone has a chance to take a look.

--



--
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] Spark 2.1.2 (RC2)

Felix Cheung
-1

(Sorry) spark-2.1.2-bin-hadoop2.7.tgz is missing the R directory, not sure why yet.

Tested on multiple platform as source package, (against 2.1.1 jar) seemed fine except this WARNING on R-devel

* checking for code/documentation mismatches ... WARNING
Codoc mismatches from documentation object 'attach':
attach
  Code: function(what, pos = 2L, name = deparse(substitute(what),
                 backtick = FALSE), warn.conflicts = TRUE)
  Docs: function(what, pos = 2L, name = deparse(substitute(what)),
                 warn.conflicts = TRUE)
  Mismatches in argument default values:
    Name: 'name' Code: deparse(substitute(what), backtick = FALSE) Docs: deparse(substitute(what))

Checked the latest release R 3.4.1 and the signature change wasn't there. This likely indicated an upcoming change in the next R release that could insur this new warning when we attempt to publish the package.

Not sure what we can do now since we work with multiple versions of R and they will have different signatures then.

From: Luciano Resende <[hidden email]>
Sent: Thursday, September 28, 2017 10:29:18 PM
To: Holden Karau
Cc: [hidden email]
Subject: Re: [VOTE] Spark 2.1.2 (RC2)
 
+1 (non-binding)

Minor comments:
The apache infra has a staging repository to add release candidates, and it might be better/simpler to use that instead of home.a.o. See https://dist.apache.org/repos/dist/dev/spark/.



On Tue, Sep 26, 2017 at 9:47 PM, Holden Karau <[hidden email]> wrote:
Please vote on releasing the following candidate as Apache Spark version 2.1.2. The vote is open until Wednesday October 4th at 23:59 PST and passes if a majority of at least 3 +1 PMC votes are cast.

[ ] +1 Release this package as Apache Spark 2.1.2
[ ] -1 Do not release this package because ...


To learn more about Apache Spark, please see https://spark.apache.org/

The tag to be voted on is v2.1.2-rc2 (fabbb7f59e47590114366d14e15fbbff8c88593c)

List of JIRA tickets resolved in this release can be found with this filter.

The release files, including signatures, digests, etc. can be found at:

Release artifacts are signed with a key from:

The staging repository for this release can be found at:

The documentation corresponding to this release can be found at:


FAQ

How can I help test this release?

If you are a Spark user, you can help us test this release by taking an existing Spark workload and running on this release candidate, then reporting any regressions.

If you're working in PySpark you can set up a virtual env and install the current RC and see if anything important breaks, in the Java/Scala you can add the staging repository to your projects resolvers and test with the RC (make sure to clean up the artifact cache before/after so you don't end up building with a out of date RC going forward).

What should happen to JIRA tickets still targeting 2.1.2?

Committers should look at those and triage. Extremely important bug fixes, documentation, and API tweaks that impact compatibility should be worked on immediately. Everything else please retarget to 2.1.3.

But my bug isn't fixed!??!

In order to make timely releases, we will typically not hold the release unless the bug in question is a regression from 2.1.1. That being said if there is something which is a regression form 2.1.1 that has not been correctly targeted please ping a committer to help target the issue (you can see the open issues listed as impacting Spark 2.1.1 & 2.1.2)

What are the unresolved issues targeted for 2.1.2?

At this time there are no open unresolved issues.

Is there anything different about this release?

This is the first release in awhile not built on the AMPLAB Jenkins. This is good because it means future releases can more easily be built and signed securely (and I've been updating the documentation in https://github.com/apache/spark-website/pull/66 as I progress), however the chances of a mistake are higher with any change like this. If there something you normally take for granted as correct when checking a release, please double check this time :)

Should I be committing code to branch-2.1?

Thanks for asking! Please treat this stage in the RC process as "code freeze" so bug fixes only. If you're uncertain if something should be back ported please reach out. If you do commit to branch-2.1 please tag your JIRA issue fix version for 2.1.3 and if we cut another RC I'll move the 2.1.3 fixed into 2.1.2 as appropriate.

Why the longer voting window?

Since there is a large industry big data conference this week I figured I'd add a little bit of extra buffer time just to make sure everyone has a chance to take a look.

--



--
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] Spark 2.1.2 (RC2)

Holden Karau
This vote is canceled and will be replaced with an RC3 once Felix and I figure out the R packaging issue.

On Fri, Sep 29, 2017 at 1:03 AM Felix Cheung <[hidden email]> wrote:
-1

(Sorry) spark-2.1.2-bin-hadoop2.7.tgz is missing the R directory, not sure why yet.

Tested on multiple platform as source package, (against 2.1.1 jar) seemed fine except this WARNING on R-devel

* checking for code/documentation mismatches ... WARNING
Codoc mismatches from documentation object 'attach':
attach
  Code: function(what, pos = 2L, name = deparse(substitute(what),
                 backtick = FALSE), warn.conflicts = TRUE)
  Docs: function(what, pos = 2L, name = deparse(substitute(what)),
                 warn.conflicts = TRUE)
  Mismatches in argument default values:
    Name: 'name' Code: deparse(substitute(what), backtick = FALSE) Docs: deparse(substitute(what))

Checked the latest release R 3.4.1 and the signature change wasn't there. This likely indicated an upcoming change in the next R release that could insur this new warning when we attempt to publish the package.

Not sure what we can do now since we work with multiple versions of R and they will have different signatures then.

From: Luciano Resende <[hidden email]>
Sent: Thursday, September 28, 2017 10:29:18 PM
To: Holden Karau
Cc: [hidden email]

Subject: Re: [VOTE] Spark 2.1.2 (RC2)
+1 (non-binding)

Minor comments:
The apache infra has a staging repository to add release candidates, and it might be better/simpler to use that instead of home.a.o. See https://dist.apache.org/repos/dist/dev/spark/.



On Tue, Sep 26, 2017 at 9:47 PM, Holden Karau <[hidden email]> wrote:
Please vote on releasing the following candidate as Apache Spark version 2.1.2. The vote is open until Wednesday October 4th at 23:59 PST and passes if a majority of at least 3 +1 PMC votes are cast.

[ ] +1 Release this package as Apache Spark 2.1.2
[ ] -1 Do not release this package because ...


To learn more about Apache Spark, please see https://spark.apache.org/

The tag to be voted on is v2.1.2-rc2 (fabbb7f59e47590114366d14e15fbbff8c88593c)

List of JIRA tickets resolved in this release can be found with this filter.

The release files, including signatures, digests, etc. can be found at:

Release artifacts are signed with a key from:

The staging repository for this release can be found at:

The documentation corresponding to this release can be found at:


FAQ

How can I help test this release?

If you are a Spark user, you can help us test this release by taking an existing Spark workload and running on this release candidate, then reporting any regressions.

If you're working in PySpark you can set up a virtual env and install the current RC and see if anything important breaks, in the Java/Scala you can add the staging repository to your projects resolvers and test with the RC (make sure to clean up the artifact cache before/after so you don't end up building with a out of date RC going forward).

What should happen to JIRA tickets still targeting 2.1.2?

Committers should look at those and triage. Extremely important bug fixes, documentation, and API tweaks that impact compatibility should be worked on immediately. Everything else please retarget to 2.1.3.

But my bug isn't fixed!??!

In order to make timely releases, we will typically not hold the release unless the bug in question is a regression from 2.1.1. That being said if there is something which is a regression form 2.1.1 that has not been correctly targeted please ping a committer to help target the issue (you can see the open issues listed as impacting Spark 2.1.1 & 2.1.2)

What are the unresolved issues targeted for 2.1.2?

At this time there are no open unresolved issues.

Is there anything different about this release?

This is the first release in awhile not built on the AMPLAB Jenkins. This is good because it means future releases can more easily be built and signed securely (and I've been updating the documentation in https://github.com/apache/spark-website/pull/66 as I progress), however the chances of a mistake are higher with any change like this. If there something you normally take for granted as correct when checking a release, please double check this time :)

Should I be committing code to branch-2.1?

Thanks for asking! Please treat this stage in the RC process as "code freeze" so bug fixes only. If you're uncertain if something should be back ported please reach out. If you do commit to branch-2.1 please tag your JIRA issue fix version for 2.1.3 and if we cut another RC I'll move the 2.1.3 fixed into 2.1.2 as appropriate.

Why the longer voting window?

Since there is a large industry big data conference this week I figured I'd add a little bit of extra buffer time just to make sure everyone has a chance to take a look.

--



--
--
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] Spark 2.1.2 (RC2)

Holden Karau
As a follow up the JIRA for this is at https://issues.apache.org/jira/browse/SPARK-22167

On Fri, Sep 29, 2017 at 2:50 AM, Holden Karau <[hidden email]> wrote:
This vote is canceled and will be replaced with an RC3 once Felix and I figure out the R packaging issue.

On Fri, Sep 29, 2017 at 1:03 AM Felix Cheung <[hidden email]> wrote:
-1

(Sorry) spark-2.1.2-bin-hadoop2.7.tgz is missing the R directory, not sure why yet.

Tested on multiple platform as source package, (against 2.1.1 jar) seemed fine except this WARNING on R-devel

* checking for code/documentation mismatches ... WARNING
Codoc mismatches from documentation object 'attach':
attach
  Code: function(what, pos = 2L, name = deparse(substitute(what),
                 backtick = FALSE), warn.conflicts = TRUE)
  Docs: function(what, pos = 2L, name = deparse(substitute(what)),
                 warn.conflicts = TRUE)
  Mismatches in argument default values:
    Name: 'name' Code: deparse(substitute(what), backtick = FALSE) Docs: deparse(substitute(what))

Checked the latest release R 3.4.1 and the signature change wasn't there. This likely indicated an upcoming change in the next R release that could insur this new warning when we attempt to publish the package.

Not sure what we can do now since we work with multiple versions of R and they will have different signatures then.

From: Luciano Resende <[hidden email]>
Sent: Thursday, September 28, 2017 10:29:18 PM
To: Holden Karau
Cc: [hidden email]

Subject: Re: [VOTE] Spark 2.1.2 (RC2)
+1 (non-binding)

Minor comments:
The apache infra has a staging repository to add release candidates, and it might be better/simpler to use that instead of home.a.o. See https://dist.apache.org/repos/dist/dev/spark/.



On Tue, Sep 26, 2017 at 9:47 PM, Holden Karau <[hidden email]> wrote:
Please vote on releasing the following candidate as Apache Spark version 2.1.2. The vote is open until Wednesday October 4th at 23:59 PST and passes if a majority of at least 3 +1 PMC votes are cast.

[ ] +1 Release this package as Apache Spark 2.1.2
[ ] -1 Do not release this package because ...


To learn more about Apache Spark, please see https://spark.apache.org/

The tag to be voted on is v2.1.2-rc2 (fabbb7f59e47590114366d14e15fbbff8c88593c)

List of JIRA tickets resolved in this release can be found with this filter.

The release files, including signatures, digests, etc. can be found at:

Release artifacts are signed with a key from:

The staging repository for this release can be found at:

The documentation corresponding to this release can be found at:


FAQ

How can I help test this release?

If you are a Spark user, you can help us test this release by taking an existing Spark workload and running on this release candidate, then reporting any regressions.

If you're working in PySpark you can set up a virtual env and install the current RC and see if anything important breaks, in the Java/Scala you can add the staging repository to your projects resolvers and test with the RC (make sure to clean up the artifact cache before/after so you don't end up building with a out of date RC going forward).

What should happen to JIRA tickets still targeting 2.1.2?

Committers should look at those and triage. Extremely important bug fixes, documentation, and API tweaks that impact compatibility should be worked on immediately. Everything else please retarget to 2.1.3.

But my bug isn't fixed!??!

In order to make timely releases, we will typically not hold the release unless the bug in question is a regression from 2.1.1. That being said if there is something which is a regression form 2.1.1 that has not been correctly targeted please ping a committer to help target the issue (you can see the open issues listed as impacting Spark 2.1.1 & 2.1.2)

What are the unresolved issues targeted for 2.1.2?

At this time there are no open unresolved issues.

Is there anything different about this release?

This is the first release in awhile not built on the AMPLAB Jenkins. This is good because it means future releases can more easily be built and signed securely (and I've been updating the documentation in https://github.com/apache/spark-website/pull/66 as I progress), however the chances of a mistake are higher with any change like this. If there something you normally take for granted as correct when checking a release, please double check this time :)

Should I be committing code to branch-2.1?

Thanks for asking! Please treat this stage in the RC process as "code freeze" so bug fixes only. If you're uncertain if something should be back ported please reach out. If you do commit to branch-2.1 please tag your JIRA issue fix version for 2.1.3 and if we cut another RC I'll move the 2.1.3 fixed into 2.1.2 as appropriate.

Why the longer voting window?

Since there is a large industry big data conference this week I figured I'd add a little bit of extra buffer time just to make sure everyone has a chance to take a look.

--



--
--



--
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] Spark 2.1.2 (RC2)

Ryan Blue
+1 (non-binding)

Checked all signatures/checksums for binaries and source, spot-checked maven artifacts. Thanks for fixing the signatures, Holden!

On Fri, Sep 29, 2017 at 8:25 AM, Holden Karau <[hidden email]> wrote:
As a follow up the JIRA for this is at https://issues.apache.org/jira/browse/SPARK-22167

On Fri, Sep 29, 2017 at 2:50 AM, Holden Karau <[hidden email]> wrote:
This vote is canceled and will be replaced with an RC3 once Felix and I figure out the R packaging issue.

On Fri, Sep 29, 2017 at 1:03 AM Felix Cheung <[hidden email]> wrote:
-1

(Sorry) spark-2.1.2-bin-hadoop2.7.tgz is missing the R directory, not sure why yet.

Tested on multiple platform as source package, (against 2.1.1 jar) seemed fine except this WARNING on R-devel

* checking for code/documentation mismatches ... WARNING
Codoc mismatches from documentation object 'attach':
attach
  Code: function(what, pos = 2L, name = deparse(substitute(what),
                 backtick = FALSE), warn.conflicts = TRUE)
  Docs: function(what, pos = 2L, name = deparse(substitute(what)),
                 warn.conflicts = TRUE)
  Mismatches in argument default values:
    Name: 'name' Code: deparse(substitute(what), backtick = FALSE) Docs: deparse(substitute(what))

Checked the latest release R 3.4.1 and the signature change wasn't there. This likely indicated an upcoming change in the next R release that could insur this new warning when we attempt to publish the package.

Not sure what we can do now since we work with multiple versions of R and they will have different signatures then.

From: Luciano Resende <[hidden email]>
Sent: Thursday, September 28, 2017 10:29:18 PM
To: Holden Karau
Cc: [hidden email]

Subject: Re: [VOTE] Spark 2.1.2 (RC2)
+1 (non-binding)

Minor comments:
The apache infra has a staging repository to add release candidates, and it might be better/simpler to use that instead of home.a.o. See https://dist.apache.org/repos/dist/dev/spark/.



On Tue, Sep 26, 2017 at 9:47 PM, Holden Karau <[hidden email]> wrote:
Please vote on releasing the following candidate as Apache Spark version 2.1.2. The vote is open until Wednesday October 4th at 23:59 PST and passes if a majority of at least 3 +1 PMC votes are cast.

[ ] +1 Release this package as Apache Spark 2.1.2
[ ] -1 Do not release this package because ...


To learn more about Apache Spark, please see https://spark.apache.org/

The tag to be voted on is v2.1.2-rc2 (fabbb7f59e47590114366d14e15fbbff8c88593c)

List of JIRA tickets resolved in this release can be found with this filter.

The release files, including signatures, digests, etc. can be found at:

Release artifacts are signed with a key from:

The staging repository for this release can be found at:

The documentation corresponding to this release can be found at:


FAQ

How can I help test this release?

If you are a Spark user, you can help us test this release by taking an existing Spark workload and running on this release candidate, then reporting any regressions.

If you're working in PySpark you can set up a virtual env and install the current RC and see if anything important breaks, in the Java/Scala you can add the staging repository to your projects resolvers and test with the RC (make sure to clean up the artifact cache before/after so you don't end up building with a out of date RC going forward).

What should happen to JIRA tickets still targeting 2.1.2?

Committers should look at those and triage. Extremely important bug fixes, documentation, and API tweaks that impact compatibility should be worked on immediately. Everything else please retarget to 2.1.3.

But my bug isn't fixed!??!

In order to make timely releases, we will typically not hold the release unless the bug in question is a regression from 2.1.1. That being said if there is something which is a regression form 2.1.1 that has not been correctly targeted please ping a committer to help target the issue (you can see the open issues listed as impacting Spark 2.1.1 & 2.1.2)

What are the unresolved issues targeted for 2.1.2?

At this time there are no open unresolved issues.

Is there anything different about this release?

This is the first release in awhile not built on the AMPLAB Jenkins. This is good because it means future releases can more easily be built and signed securely (and I've been updating the documentation in https://github.com/apache/spark-website/pull/66 as I progress), however the chances of a mistake are higher with any change like this. If there something you normally take for granted as correct when checking a release, please double check this time :)

Should I be committing code to branch-2.1?

Thanks for asking! Please treat this stage in the RC process as "code freeze" so bug fixes only. If you're uncertain if something should be back ported please reach out. If you do commit to branch-2.1 please tag your JIRA issue fix version for 2.1.3 and if we cut another RC I'll move the 2.1.3 fixed into 2.1.2 as appropriate.

Why the longer voting window?

Since there is a large industry big data conference this week I figured I'd add a little bit of extra buffer time just to make sure everyone has a chance to take a look.

--



--
--



--



--
Ryan Blue
Software Engineer
Netflix
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] Spark 2.1.2 (RC2)

vaquarkhan
+1 (non-binding)

Regards,
Vaquar khan

On Fri, Sep 29, 2017 at 1:52 PM, Ryan Blue <[hidden email]> wrote:
+1 (non-binding)

Checked all signatures/checksums for binaries and source, spot-checked maven artifacts. Thanks for fixing the signatures, Holden!

On Fri, Sep 29, 2017 at 8:25 AM, Holden Karau <[hidden email]> wrote:
As a follow up the JIRA for this is at https://issues.apache.org/jira/browse/SPARK-22167

On Fri, Sep 29, 2017 at 2:50 AM, Holden Karau <[hidden email]> wrote:
This vote is canceled and will be replaced with an RC3 once Felix and I figure out the R packaging issue.

On Fri, Sep 29, 2017 at 1:03 AM Felix Cheung <[hidden email]> wrote:
-1

(Sorry) spark-2.1.2-bin-hadoop2.7.tgz is missing the R directory, not sure why yet.

Tested on multiple platform as source package, (against 2.1.1 jar) seemed fine except this WARNING on R-devel

* checking for code/documentation mismatches ... WARNING
Codoc mismatches from documentation object 'attach':
attach
  Code: function(what, pos = 2L, name = deparse(substitute(what),
                 backtick = FALSE), warn.conflicts = TRUE)
  Docs: function(what, pos = 2L, name = deparse(substitute(what)),
                 warn.conflicts = TRUE)
  Mismatches in argument default values:
    Name: 'name' Code: deparse(substitute(what), backtick = FALSE) Docs: deparse(substitute(what))

Checked the latest release R 3.4.1 and the signature change wasn't there. This likely indicated an upcoming change in the next R release that could insur this new warning when we attempt to publish the package.

Not sure what we can do now since we work with multiple versions of R and they will have different signatures then.

From: Luciano Resende <[hidden email]>
Sent: Thursday, September 28, 2017 10:29:18 PM
To: Holden Karau
Cc: [hidden email]

Subject: Re: [VOTE] Spark 2.1.2 (RC2)
+1 (non-binding)

Minor comments:
The apache infra has a staging repository to add release candidates, and it might be better/simpler to use that instead of home.a.o. See https://dist.apache.org/repos/dist/dev/spark/.



On Tue, Sep 26, 2017 at 9:47 PM, Holden Karau <[hidden email]> wrote:
Please vote on releasing the following candidate as Apache Spark version 2.1.2. The vote is open until Wednesday October 4th at 23:59 PST and passes if a majority of at least 3 +1 PMC votes are cast.

[ ] +1 Release this package as Apache Spark 2.1.2
[ ] -1 Do not release this package because ...


To learn more about Apache Spark, please see https://spark.apache.org/

The tag to be voted on is v2.1.2-rc2 (fabbb7f59e47590114366d14e15fbbff8c88593c)

List of JIRA tickets resolved in this release can be found with this filter.

The release files, including signatures, digests, etc. can be found at:

Release artifacts are signed with a key from:

The staging repository for this release can be found at:

The documentation corresponding to this release can be found at:


FAQ

How can I help test this release?

If you are a Spark user, you can help us test this release by taking an existing Spark workload and running on this release candidate, then reporting any regressions.

If you're working in PySpark you can set up a virtual env and install the current RC and see if anything important breaks, in the Java/Scala you can add the staging repository to your projects resolvers and test with the RC (make sure to clean up the artifact cache before/after so you don't end up building with a out of date RC going forward).

What should happen to JIRA tickets still targeting 2.1.2?

Committers should look at those and triage. Extremely important bug fixes, documentation, and API tweaks that impact compatibility should be worked on immediately. Everything else please retarget to 2.1.3.

But my bug isn't fixed!??!

In order to make timely releases, we will typically not hold the release unless the bug in question is a regression from 2.1.1. That being said if there is something which is a regression form 2.1.1 that has not been correctly targeted please ping a committer to help target the issue (you can see the open issues listed as impacting Spark 2.1.1 & 2.1.2)

What are the unresolved issues targeted for 2.1.2?

At this time there are no open unresolved issues.

Is there anything different about this release?

This is the first release in awhile not built on the AMPLAB Jenkins. This is good because it means future releases can more easily be built and signed securely (and I've been updating the documentation in https://github.com/apache/spark-website/pull/66 as I progress), however the chances of a mistake are higher with any change like this. If there something you normally take for granted as correct when checking a release, please double check this time :)

Should I be committing code to branch-2.1?

Thanks for asking! Please treat this stage in the RC process as "code freeze" so bug fixes only. If you're uncertain if something should be back ported please reach out. If you do commit to branch-2.1 please tag your JIRA issue fix version for 2.1.3 and if we cut another RC I'll move the 2.1.3 fixed into 2.1.2 as appropriate.

Why the longer voting window?

Since there is a large industry big data conference this week I figured I'd add a little bit of extra buffer time just to make sure everyone has a chance to take a look.

--



--
--



--



--
Ryan Blue
Software Engineer
Netflix



--
Regards,
Vaquar Khan
+1 -224-436-0783
Greater Chicago