Hi Spark dev,
Here is the voting thread for parquet 1.8.2 release.
Cheng or someone else we would appreciate you verify it as well and reply to the thread.
On Mon, Jan 23, 2017 at 11:40 AM, Julien Le Dem <[hidden email]> wrote:
Sorry for being late, I'm building a Spark branch based on the most recent master to test out 1.8.2-rc1, will post my result here ASAP.
On 1/23/17 11:43 AM, Julien Le Dem wrote:
Thank you Cheng!
On Mon, Jan 23, 2017 at 12:02 PM, Cheng Lian <[hidden email]> wrote:
In reply to this post by Julien Le Dem
and Jenkins failure
when bumping our build to 1.8.2-rc1 unit tests succeed but we encounter NoSuchMethodError errors at runtime in spark-submit. Our build is a mess of transitive dependencies and exclusions, so I haven't yet been able to identify why bumping the parquet version would cause this problem.
I wanted to mention this here in case anyone else runs into it.
---------- Forwarded message ----------
Michael, the problem you're hitting is that Parquet's dependency moved to Avro 1.8.0 from 1.7.7 and added a method that is missing when you use 1.7.7.
It looks like Spark is still using Avro 1.7.7. I think updating that to 1.8.x should fix the problem.
On Tue, Jan 24, 2017 at 5:35 AM, Michael Heuer <[hidden email]> wrote:
|Free forum by Nabble||Edit this page|