On Scala 2.12.7

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

On Scala 2.12.7

Sean Owen-3
I'm forking the discussion about Scala 2.12.7 from the 2.4.0 RC vote thread.

2.12.7 was released yesterday, and, is even labeled as fixing Spark
2.4 compatibility! https://www.scala-lang.org/news/2.12.7 We should
look into it, yes.

Darcy identified, and they fixed, this issue:
https://github.com/scala/scala/pull/7156 while finishing the work for
Scala 2.12.

However we already worked around this in Spark, no? at
https://github.com/apache/spark/commit/f29c2b5287563c0d6f55f936bd5a75707d7b2b1f

So we should go ahead and update to use 2.12.7, yes, and undo this workaround?
But this doesn't necessarily block a 2.4.0 release, if it's already
worked around.

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: On Scala 2.12.7

cloud0fan
My major concern is how it will affect end-users if Spark 2.4 is built with Scala versions prior to 2.12.7. Generally I'm hesitating to upgrade Scala version when we are very close to a release, and Scala 2.12 build of Spark 2.4 is beta anyway.

On Sat, Sep 29, 2018 at 6:46 AM Sean Owen <[hidden email]> wrote:
I'm forking the discussion about Scala 2.12.7 from the 2.4.0 RC vote thread.

2.12.7 was released yesterday, and, is even labeled as fixing Spark
2.4 compatibility! https://www.scala-lang.org/news/2.12.7 We should
look into it, yes.

Darcy identified, and they fixed, this issue:
https://github.com/scala/scala/pull/7156 while finishing the work for
Scala 2.12.

However we already worked around this in Spark, no? at
https://github.com/apache/spark/commit/f29c2b5287563c0d6f55f936bd5a75707d7b2b1f

So we should go ahead and update to use 2.12.7, yes, and undo this workaround?
But this doesn't necessarily block a 2.4.0 release, if it's already
worked around.

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: On Scala 2.12.7

Felix Cheung
Although like you said, spark support for scala 2.12 is beta anyway then shouldn’t we get it to a working state by basing on 2.12.7? There shouldn’t be a stability issue since it is not officially “supported”


 

From: Wenchen Fan <[hidden email]>
Sent: Monday, October 1, 2018 7:43 PM
To: Sean Owen
Cc: [hidden email]; Spark dev list
Subject: Re: On Scala 2.12.7
 
My major concern is how it will affect end-users if Spark 2.4 is built with Scala versions prior to 2.12.7. Generally I'm hesitating to upgrade Scala version when we are very close to a release, and Scala 2.12 build of Spark 2.4 is beta anyway.

On Sat, Sep 29, 2018 at 6:46 AM Sean Owen <[hidden email]> wrote:
I'm forking the discussion about Scala 2.12.7 from the 2.4.0 RC vote thread.

2.12.7 was released yesterday, and, is even labeled as fixing Spark
2.4 compatibility! https://www.scala-lang.org/news/2.12.7 We should
look into it, yes.

Darcy identified, and they fixed, this issue:
https://github.com/scala/scala/pull/7156 while finishing the work for
Scala 2.12.

However we already worked around this in Spark, no? at
https://github.com/apache/spark/commit/f29c2b5287563c0d6f55f936bd5a75707d7b2b1f

So we should go ahead and update to use 2.12.7, yes, and undo this workaround?
But this doesn't necessarily block a 2.4.0 release, if it's already
worked around.

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: On Scala 2.12.7

Sean Owen-3
In reply to this post by cloud0fan
I tested both ways, and it actually works fine. It calls into question
whether there's really a fix we need with 2.12.7, but, I hear two
informed opinions (Darcy and the scala release notes) that it was
relevant. As we have no prior 2.12 support, I guess my feeling was
indeed to get this update in to the first 2.12-supporting release.

On Mon, Oct 1, 2018 at 9:43 PM Wenchen Fan <[hidden email]> wrote:
>
> My major concern is how it will affect end-users if Spark 2.4 is built with Scala versions prior to 2.12.7. Generally I'm hesitating to upgrade Scala version when we are very close to a release, and Scala 2.12 build of Spark 2.4 is beta anyway.
>

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: On Scala 2.12.7

cloud0fan
SGTM then. Is there anything we need to do to pick up the 2.12.7 upgrade? like updating Jenkins config?

On Tue, Oct 2, 2018 at 10:53 AM Sean Owen <[hidden email]> wrote:
I tested both ways, and it actually works fine. It calls into question
whether there's really a fix we need with 2.12.7, but, I hear two
informed opinions (Darcy and the scala release notes) that it was
relevant. As we have no prior 2.12 support, I guess my feeling was
indeed to get this update in to the first 2.12-supporting release.

On Mon, Oct 1, 2018 at 9:43 PM Wenchen Fan <[hidden email]> wrote:
>
> My major concern is how it will affect end-users if Spark 2.4 is built with Scala versions prior to 2.12.7. Generally I'm hesitating to upgrade Scala version when we are very close to a release, and Scala 2.12 build of Spark 2.4 is beta anyway.
>
Reply | Threaded
Open this post in threaded view
|

Re: On Scala 2.12.7

Sean Owen-3
I don't think so. Spark brings the Scala dependency, and I don't think
the installed scalac matters in this respect.
Darcy there was an open question about whether this enables you to
back out the workaround you created for 2.12.6. I tried removing it
and it failed again, so left it in as still needed.
On Mon, Oct 1, 2018 at 10:20 PM Wenchen Fan <[hidden email]> wrote:

>
> SGTM then. Is there anything we need to do to pick up the 2.12.7 upgrade? like updating Jenkins config?
>
> On Tue, Oct 2, 2018 at 10:53 AM Sean Owen <[hidden email]> wrote:
>>
>> I tested both ways, and it actually works fine. It calls into question
>> whether there's really a fix we need with 2.12.7, but, I hear two
>> informed opinions (Darcy and the scala release notes) that it was
>> relevant. As we have no prior 2.12 support, I guess my feeling was
>> indeed to get this update in to the first 2.12-supporting release.
>>
>> On Mon, Oct 1, 2018 at 9:43 PM Wenchen Fan <[hidden email]> wrote:
>> >
>> > My major concern is how it will affect end-users if Spark 2.4 is built with Scala versions prior to 2.12.7. Generally I'm hesitating to upgrade Scala version when we are very close to a release, and Scala 2.12 build of Spark 2.4 is beta anyway.
>> >

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]