Re: [VOTE] Release Apache Parquet 1.8.2 RC1

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [VOTE] Release Apache Parquet 1.8.2 RC1

Julien Le Dem
Hi Spark dev,
Here is the voting thread for parquet 1.8.2 release.
Cheng or someone else we would appreciate you verify it as well and reply to the thread.

On Mon, Jan 23, 2017 at 11:40 AM, Julien Le Dem <[hidden email]> wrote:
+1
We would appreciate someone from the Spark project (Cheng?) to verify the release as well.
CC'ing spark


On Mon, Jan 23, 2017 at 10:15 AM, Ryan Blue <[hidden email]> wrote:
+1

On Mon, Jan 23, 2017 at 10:15 AM, Daniel Weeks <[hidden email]>
wrote:

> +1 checked sums, built, tested
>
> On Mon, Jan 23, 2017 at 9:58 AM, Ryan Blue <[hidden email]>
> wrote:
>
> > Gabor, that md5 matches what I get. Are you sure you used the right file?
> > It isn’t the same format that md5sum produces, but if you check the
> octets
> > the hash matches..
> >
> > [blue@work Downloads]$ md5sum apache-parquet-1.8.2.tar.gz
> > b3743995bee616118c28f324598684ba  apache-parquet-1.8.2.tar.gz
> >
> > rb
> > ​
> >
> > On Thu, Jan 19, 2017 at 8:06 AM, Gabor Szadovszky <
> > [hidden email]> wrote:
> >
> > > Hi Ryan,
> > >
> > > I’ve downloaded the tar and checked the signature and the checksums.
> SHA
> > > and ASC are fine. MD5 is not and the content does not seem to be a
> common
> > > MD5 either:
> > > apache-parquet-1.8.2.tar.gz: B3 74 39 95 BE E6 16 11  8C 28 F3 24 59 86
> > 84
> > > BA
> > >
> > > The artifacts on Nexus are good with all the related signatures and
> > > checksums. The source zip properly contains the files from the repo
> with
> > > the tag apache-parquet-1.8.2.
> > >
> > > Regards,
> > > Gabor
> > >
> > > > On 19 Jan 2017, at 04:09, Ryan Blue <[hidden email]> wrote:
> > > >
> > > > Hi everyone,
> > > >
> > > > I propose the following RC to be released as official Apache Parquet
> > > 1.8.2
> > > > release.
> > > >
> > > > The commit id is c6522788629e590a53eb79874b95f6c3ff11f16c
> > > > * This corresponds to the tag: apache-parquet-1.8.2
> > > > * https://github.com/apache/parquet-mr/tree/c6522788
> > > > *
> > > > https://git-wip-us.apache.org/repos/asf/projects/repo?p=
> > > parquet-mr.git&a=commit&h=c6522788
> > > >
> > > > The release tarball, signature, and checksums are here:
> > > > * https://dist.apache.org/repos/dist/dev/parquet/apache-
> > > parquet-1.8.2-rc1
> > > >
> > > > You can find the KEYS file here:
> > > > * https://dist.apache.org/repos/dist/dev/parquet/KEYS
> > > >
> > > > Binary artifacts are staged in Nexus here:
> > > > *
> > > > https://repository.apache.org/content/groups/staging/org/
> > > apache/parquet/parquet/1.8.2/
> > > >
> > > > This is a patch release with backports from the master branch. For a
> > > > detailed summary, see the spreadsheet here:
> > > >
> > > > *
> > > > https://docs.google.com/spreadsheets/d/1NAuY3c77Egs6REu-
> > > UVkQqPswpVYVgZTTnY3bM0SPVRs/edit#gid=0
> > > >
> > > > Please download, verify, and test.
> > > >
> > > > Please vote by the end of Monday, 18 January.
> > > >
> > > > [ ] +1 Release this as Apache Parquet 1.8.2
> > > > [ ] +0
> > > > [ ] -1 Do not release this because...
> > > >
> > > >
> > > >
> > > > --
> > > > Ryan Blue
> > >
> > >
> >
> >
> > --
> > Ryan Blue
> > Software Engineer
> > Netflix
> >
>



--
Ryan Blue
Software Engineer
Netflix



--
Julien



--
Julien
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [VOTE] Release Apache Parquet 1.8.2 RC1

Cheng Lian

Sorry for being late, I'm building a Spark branch based on the most recent master to test out 1.8.2-rc1, will post my result here ASAP.

Cheng


On 1/23/17 11:43 AM, Julien Le Dem wrote:
Hi Spark dev,
Here is the voting thread for parquet 1.8.2 release.
Cheng or someone else we would appreciate you verify it as well and reply to the thread.

On Mon, Jan 23, 2017 at 11:40 AM, Julien Le Dem <[hidden email]> wrote:
+1
Followed: https://cwiki.apache.org/confluence/display/PARQUET/How+To+Verify+A+Release
checked sums, ran the build and tests.
We would appreciate someone from the Spark project (Cheng?) to verify the release as well.
CC'ing spark


On Mon, Jan 23, 2017 at 10:15 AM, Ryan Blue <[hidden email]> wrote:
+1

On Mon, Jan 23, 2017 at 10:15 AM, Daniel Weeks [hidden email]
wrote:

> +1 checked sums, built, tested
>
> On Mon, Jan 23, 2017 at 9:58 AM, Ryan Blue [hidden email]
> wrote:
>
> > Gabor, that md5 matches what I get. Are you sure you used the right file?
> > It isn’t the same format that md5sum produces, but if you check the
> octets
> > the hash matches..
> >
> > [blue@work Downloads]$ md5sum apache-parquet-1.8.2.tar.gz
> > b3743995bee616118c28f324598684ba  apache-parquet-1.8.2.tar.gz
> >
> > rb
> > ​
> >
> > On Thu, Jan 19, 2017 at 8:06 AM, Gabor Szadovszky <
> > [hidden email]> wrote:
> >
> > > Hi Ryan,
> > >
> > > I’ve downloaded the tar and checked the signature and the checksums.
> SHA
> > > and ASC are fine. MD5 is not and the content does not seem to be a
> common
> > > MD5 either:
> > > apache-parquet-1.8.2.tar.gz: B3 74 39 95 BE E6 16 11  8C 28 F3 24 59 86
> > 84
> > > BA
> > >
> > > The artifacts on Nexus are good with all the related signatures and
> > > checksums. The source zip properly contains the files from the repo
> with
> > > the tag apache-parquet-1.8.2.
> > >
> > > Regards,
> > > Gabor
> > >
> > > > On 19 Jan 2017, at 04:09, Ryan Blue <[hidden email]> wrote:
> > > >
> > > > Hi everyone,
> > > >
> > > > I propose the following RC to be released as official Apache Parquet
> > > 1.8.2
> > > > release.
> > > >
> > > > The commit id is c6522788629e590a53eb79874b95f6c3ff11f16c
> > > > * This corresponds to the tag: apache-parquet-1.8.2
> > > > * https://github.com/apache/parquet-mr/tree/c6522788
> > > > *
> > > > https://git-wip-us.apache.org/repos/asf/projects/repo?p=
> > > parquet-mr.git&a=commit&h=c6522788
> > > >
> > > > The release tarball, signature, and checksums are here:
> > > > * https://dist.apache.org/repos/dist/dev/parquet/apache-
> > > parquet-1.8.2-rc1
> > > >
> > > > You can find the KEYS file here:
> > > > * https://dist.apache.org/repos/dist/dev/parquet/KEYS
> > > >
> > > > Binary artifacts are staged in Nexus here:
> > > > *
> > > > https://repository.apache.org/content/groups/staging/org/
> > > apache/parquet/parquet/1.8.2/
> > > >
> > > > This is a patch release with backports from the master branch. For a
> > > > detailed summary, see the spreadsheet here:
> > > >
> > > > *
> > > > https://docs.google.com/spreadsheets/d/1NAuY3c77Egs6REu-
> > > UVkQqPswpVYVgZTTnY3bM0SPVRs/edit#gid=0
> > > >
> > > > Please download, verify, and test.
> > > >
> > > > Please vote by the end of Monday, 18 January.
> > > >
> > > > [ ] +1 Release this as Apache Parquet 1.8.2
> > > > [ ] +0
> > > > [ ] -1 Do not release this because...
> > > >
> > > >
> > > >
> > > > --
> > > > Ryan Blue
> > >
> > >
> >
> >
> > --
> > Ryan Blue
> > Software Engineer
> > Netflix
> >
>



--
Ryan Blue
Software Engineer
Netflix



--
Julien



--
Julien

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [VOTE] Release Apache Parquet 1.8.2 RC1

Julien Le Dem
Thank you Cheng!

On Mon, Jan 23, 2017 at 12:02 PM, Cheng Lian <[hidden email]> wrote:

Sorry for being late, I'm building a Spark branch based on the most recent master to test out 1.8.2-rc1, will post my result here ASAP.

Cheng


On 1/23/17 11:43 AM, Julien Le Dem wrote:
Hi Spark dev,
Here is the voting thread for parquet 1.8.2 release.
Cheng or someone else we would appreciate you verify it as well and reply to the thread.

On Mon, Jan 23, 2017 at 11:40 AM, Julien Le Dem <[hidden email]> wrote:
+1
Followed: https://cwiki.apache.org/confluence/display/PARQUET/How+To+Verify+A+Release
checked sums, ran the build and tests.
We would appreciate someone from the Spark project (Cheng?) to verify the release as well.
CC'ing spark


On Mon, Jan 23, 2017 at 10:15 AM, Ryan Blue <[hidden email]> wrote:
+1

On Mon, Jan 23, 2017 at 10:15 AM, Daniel Weeks [hidden email]
wrote:

> +1 checked sums, built, tested
>
> On Mon, Jan 23, 2017 at 9:58 AM, Ryan Blue [hidden email]
> wrote:
>
> > Gabor, that md5 matches what I get. Are you sure you used the right file?
> > It isn’t the same format that md5sum produces, but if you check the
> octets
> > the hash matches..
> >
> > [blue@work Downloads]$ md5sum apache-parquet-1.8.2.tar.gz
> > b3743995bee616118c28f324598684ba  apache-parquet-1.8.2.tar.gz
> >
> > rb
> > ​
> >
> > On Thu, Jan 19, 2017 at 8:06 AM, Gabor Szadovszky <
> > [hidden email]> wrote:
> >
> > > Hi Ryan,
> > >
> > > I’ve downloaded the tar and checked the signature and the checksums.
> SHA
> > > and ASC are fine. MD5 is not and the content does not seem to be a
> common
> > > MD5 either:
> > > apache-parquet-1.8.2.tar.gz: B3 74 39 95 BE E6 16 11  8C 28 F3 24 59 86
> > 84
> > > BA
> > >
> > > The artifacts on Nexus are good with all the related signatures and
> > > checksums. The source zip properly contains the files from the repo
> with
> > > the tag apache-parquet-1.8.2.
> > >
> > > Regards,
> > > Gabor
> > >
> > > > On 19 Jan 2017, at 04:09, Ryan Blue <[hidden email]> wrote:
> > > >
> > > > Hi everyone,
> > > >
> > > > I propose the following RC to be released as official Apache Parquet
> > > 1.8.2
> > > > release.
> > > >
> > > > The commit id is c6522788629e590a53eb79874b95f6c3ff11f16c
> > > > * This corresponds to the tag: apache-parquet-1.8.2
> > > > * https://github.com/apache/parquet-mr/tree/c6522788
> > > > *
> > > > https://git-wip-us.apache.org/repos/asf/projects/repo?p=
> > > parquet-mr.git&a=commit&h=c6522788
> > > >
> > > > The release tarball, signature, and checksums are here:
> > > > * https://dist.apache.org/repos/dist/dev/parquet/apache-
> > > parquet-1.8.2-rc1
> > > >
> > > > You can find the KEYS file here:
> > > > * https://dist.apache.org/repos/dist/dev/parquet/KEYS
> > > >
> > > > Binary artifacts are staged in Nexus here:
> > > > *
> > > > https://repository.apache.org/content/groups/staging/org/
> > > apache/parquet/parquet/1.8.2/
> > > >
> > > > This is a patch release with backports from the master branch. For a
> > > > detailed summary, see the spreadsheet here:
> > > >
> > > > *
> > > > https://docs.google.com/spreadsheets/d/1NAuY3c77Egs6REu-
> > > UVkQqPswpVYVgZTTnY3bM0SPVRs/edit#gid=0
> > > >
> > > > Please download, verify, and test.
> > > >
> > > > Please vote by the end of Monday, 18 January.
> > > >
> > > > [ ] +1 Release this as Apache Parquet 1.8.2
> > > > [ ] +0
> > > > [ ] -1 Do not release this because...
> > > >
> > > >
> > > >
> > > > --
> > > > Ryan Blue
> > >
> > >
> >
> >
> > --
> > Ryan Blue
> > Software Engineer
> > Netflix
> >
>



--
Ryan Blue
Software Engineer
Netflix



--
Julien



--
Julien




--
Julien
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [VOTE] Release Apache Parquet 1.8.2 RC1

Michael Heuer
In reply to this post by Julien Le Dem
Per comment
https://github.com/bigdatagenomics/adam/pull/1360#issuecomment-274681650

when bumping our build to 1.8.2-rc1 unit tests succeed but we encounter NoSuchMethodError errors at runtime in spark-submit.  Our build is a mess of transitive dependencies and exclusions, so I haven't yet been able to identify why bumping the parquet version would cause this problem.

I wanted to mention this here in case anyone else runs into it.

   michael


---------- Forwarded message ----------
From: Julien Le Dem <[hidden email]>
To: [hidden email], [hidden email]
Cc: 
Date: Mon, 23 Jan 2017 11:43:40 -0800
Subject: Re: [VOTE] Release Apache Parquet 1.8.2 RC1
Hi Spark dev,
Here is the voting thread for parquet 1.8.2 release.
Cheng or someone else we would appreciate you verify it as well and reply to the thread.

On Mon, Jan 23, 2017 at 11:40 AM, Julien Le Dem <[hidden email]> wrote:
+1
We would appreciate someone from the Spark project (Cheng?) to verify the release as well.
CC'ing spark


On Mon, Jan 23, 2017 at 10:15 AM, Ryan Blue <[hidden email]> wrote:
+1

On Mon, Jan 23, 2017 at 10:15 AM, Daniel Weeks <[hidden email]>
wrote:

> +1 checked sums, built, tested
>
> On Mon, Jan 23, 2017 at 9:58 AM, Ryan Blue <[hidden email]>
> wrote:
>
> > Gabor, that md5 matches what I get. Are you sure you used the right file?
> > It isn’t the same format that md5sum produces, but if you check the
> octets
> > the hash matches..
> >
> > [blue@work Downloads]$ md5sum apache-parquet-1.8.2.tar.gz
> > b3743995bee616118c28f324598684ba  apache-parquet-1.8.2.tar.gz
> >
> > rb
> > ​
> >
> > On Thu, Jan 19, 2017 at 8:06 AM, Gabor Szadovszky <
> > [hidden email]> wrote:
> >
> > > Hi Ryan,
> > >
> > > I’ve downloaded the tar and checked the signature and the checksums.
> SHA
> > > and ASC are fine. MD5 is not and the content does not seem to be a
> common
> > > MD5 either:
> > > apache-parquet-1.8.2.tar.gz: B3 74 39 95 BE E6 16 11  8C 28 F3 24 59 86
> > 84
> > > BA
> > >
> > > The artifacts on Nexus are good with all the related signatures and
> > > checksums. The source zip properly contains the files from the repo
> with
> > > the tag apache-parquet-1.8.2.
> > >
> > > Regards,
> > > Gabor
> > >
> > > > On 19 Jan 2017, at 04:09, Ryan Blue <[hidden email]> wrote:
> > > >
> > > > Hi everyone,
> > > >
> > > > I propose the following RC to be released as official Apache Parquet
> > > 1.8.2
> > > > release.
> > > >
> > > > The commit id is c6522788629e590a53eb79874b95f6c3ff11f16c
> > > > * This corresponds to the tag: apache-parquet-1.8.2
> > > > * https://github.com/apache/parquet-mr/tree/c6522788
> > > > *
> > > > https://git-wip-us.apache.org/repos/asf/projects/repo?p=
> > > parquet-mr.git&a=commit&h=c6522788
> > > >
> > > > The release tarball, signature, and checksums are here:
> > > > * https://dist.apache.org/repos/dist/dev/parquet/apache-
> > > parquet-1.8.2-rc1
> > > >
> > > > You can find the KEYS file here:
> > > > * https://dist.apache.org/repos/dist/dev/parquet/KEYS
> > > >
> > > > Binary artifacts are staged in Nexus here:
> > > > *
> > > > https://repository.apache.org/content/groups/staging/org/
> > > apache/parquet/parquet/1.8.2/
> > > >
> > > > This is a patch release with backports from the master branch. For a
> > > > detailed summary, see the spreadsheet here:
> > > >
> > > > *
> > > > https://docs.google.com/spreadsheets/d/1NAuY3c77Egs6REu-
> > > UVkQqPswpVYVgZTTnY3bM0SPVRs/edit#gid=0
> > > >
> > > > Please download, verify, and test.
> > > >
> > > > Please vote by the end of Monday, 18 January.
> > > >
> > > > [ ] +1 Release this as Apache Parquet 1.8.2
> > > > [ ] +0
> > > > [ ] -1 Do not release this because...
> > > >
> > > >
> > > >
> > > > --
> > > > Ryan Blue
> > >
> > >
> >
> >
> > --
> > Ryan Blue
> > Software Engineer
> > Netflix
> >
>



--
Ryan Blue
Software Engineer
Netflix



--
Julien



--
Julien


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [VOTE] Release Apache Parquet 1.8.2 RC1

Ryan Blue
Michael, the problem you're hitting is that Parquet's dependency moved to Avro 1.8.0 from 1.7.7 and added a method that is missing when you use 1.7.7.

It looks like Spark is still using Avro 1.7.7. I think updating that to 1.8.x should fix the problem.

rb

On Tue, Jan 24, 2017 at 5:35 AM, Michael Heuer <[hidden email]> wrote:
Per comment
https://github.com/bigdatagenomics/adam/pull/1360#issuecomment-274681650

when bumping our build to 1.8.2-rc1 unit tests succeed but we encounter NoSuchMethodError errors at runtime in spark-submit.  Our build is a mess of transitive dependencies and exclusions, so I haven't yet been able to identify why bumping the parquet version would cause this problem.

I wanted to mention this here in case anyone else runs into it.

   michael


---------- Forwarded message ----------
From: Julien Le Dem <[hidden email]>
To: [hidden email], [hidden email]
Cc: 
Date: Mon, 23 Jan 2017 11:43:40 -0800
Subject: Re: [VOTE] Release Apache Parquet 1.8.2 RC1
Hi Spark dev,
Here is the voting thread for parquet 1.8.2 release.
Cheng or someone else we would appreciate you verify it as well and reply to the thread.

On Mon, Jan 23, 2017 at 11:40 AM, Julien Le Dem <[hidden email]> wrote:
+1
We would appreciate someone from the Spark project (Cheng?) to verify the release as well.
CC'ing spark


On Mon, Jan 23, 2017 at 10:15 AM, Ryan Blue <[hidden email]> wrote:
+1

On Mon, Jan 23, 2017 at 10:15 AM, Daniel Weeks <[hidden email]>
wrote:

> +1 checked sums, built, tested
>
> On Mon, Jan 23, 2017 at 9:58 AM, Ryan Blue <[hidden email]>
> wrote:
>
> > Gabor, that md5 matches what I get. Are you sure you used the right file?
> > It isn’t the same format that md5sum produces, but if you check the
> octets
> > the hash matches..
> >
> > [blue@work Downloads]$ md5sum apache-parquet-1.8.2.tar.gz
> > b3743995bee616118c28f324598684ba  apache-parquet-1.8.2.tar.gz
> >
> > rb
> > ​
> >
> > On Thu, Jan 19, 2017 at 8:06 AM, Gabor Szadovszky <
> > [hidden email]> wrote:
> >
> > > Hi Ryan,
> > >
> > > I’ve downloaded the tar and checked the signature and the checksums.
> SHA
> > > and ASC are fine. MD5 is not and the content does not seem to be a
> common
> > > MD5 either:
> > > apache-parquet-1.8.2.tar.gz: B3 74 39 95 BE E6 16 11  8C 28 F3 24 59 86
> > 84
> > > BA
> > >
> > > The artifacts on Nexus are good with all the related signatures and
> > > checksums. The source zip properly contains the files from the repo
> with
> > > the tag apache-parquet-1.8.2.
> > >
> > > Regards,
> > > Gabor
> > >
> > > > On 19 Jan 2017, at 04:09, Ryan Blue <[hidden email]> wrote:
> > > >
> > > > Hi everyone,
> > > >
> > > > I propose the following RC to be released as official Apache Parquet
> > > 1.8.2
> > > > release.
> > > >
> > > > The commit id is c6522788629e590a53eb79874b95f6c3ff11f16c
> > > > * This corresponds to the tag: apache-parquet-1.8.2
> > > > * https://github.com/apache/parquet-mr/tree/c6522788
> > > > *
> > > > https://git-wip-us.apache.org/repos/asf/projects/repo?p=
> > > parquet-mr.git&a=commit&h=c6522788
> > > >
> > > > The release tarball, signature, and checksums are here:
> > > > * https://dist.apache.org/repos/dist/dev/parquet/apache-
> > > parquet-1.8.2-rc1
> > > >
> > > > You can find the KEYS file here:
> > > > * https://dist.apache.org/repos/dist/dev/parquet/KEYS
> > > >
> > > > Binary artifacts are staged in Nexus here:
> > > > *
> > > > https://repository.apache.org/content/groups/staging/org/
> > > apache/parquet/parquet/1.8.2/
> > > >
> > > > This is a patch release with backports from the master branch. For a
> > > > detailed summary, see the spreadsheet here:
> > > >
> > > > *
> > > > https://docs.google.com/spreadsheets/d/1NAuY3c77Egs6REu-
> > > UVkQqPswpVYVgZTTnY3bM0SPVRs/edit#gid=0
> > > >
> > > > Please download, verify, and test.
> > > >
> > > > Please vote by the end of Monday, 18 January.
> > > >
> > > > [ ] +1 Release this as Apache Parquet 1.8.2
> > > > [ ] +0
> > > > [ ] -1 Do not release this because...
> > > >
> > > >
> > > >
> > > > --
> > > > Ryan Blue
> > >
> > >
> >
> >
> > --
> > Ryan Blue
> > Software Engineer
> > Netflix
> >
>



--
Ryan Blue
Software Engineer
Netflix



--
Julien



--
Julien





--
Ryan Blue
Software Engineer
Netflix
Loading...