[FYI] Kubernetes GA Preparation (SPARK-33005)

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

[FYI] Kubernetes GA Preparation (SPARK-33005)

Dongjoon Hyun-2
Hi, All.

K8s GA preparation is on the way like the following.

    https://issues.apache.org/jira/browse/SPARK-33005

Apache Spark 3.1/3.2 is scheduled for December 2020
and mid of 2021 (TBD). If you hit K8s issues, please
file a JIRA issue. To give more visibility to your issue,
you can create yours as a subtask of SPARK-33005
or link to it.

Bests,
Dongjoon.
Reply | Threaded
Open this post in threaded view
|

Re: [FYI] Kubernetes GA Preparation (SPARK-33005)

Dr. Kent Yao
Thanks, Dongjon,
   I pined two long-standing issues to the umbrella.
   
   https://issues.apache.org/jira/browse/SPARK-28895
   https://issues.apache.org/jira/browse/SPARK-28992
   
   This helps fix some problems of running Spark on k8s application with
HDFS.
   Hoping to get reviewed soon

Best,
Kent Yao



--
Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [FYI] Kubernetes GA Preparation (SPARK-33005)

Dongjoon Hyun-2
Thank you!

Bests,
Dongjoon

On Mon, Sep 28, 2020 at 8:07 PM Dr. Kent Yao <[hidden email]> wrote:
Thanks, Dongjon,

   I pined two long-standing issues to the umbrella.



   https://issues.apache.org/jira/browse/SPARK-28895

   https://issues.apache.org/jira/browse/SPARK-28992



   This helps fix some problems of running Spark on k8s application with

HDFS.

   Hoping to get reviewed soon



Best,

Kent Yao







--

Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/



---------------------------------------------------------------------

To unsubscribe e-mail: [hidden email]



Reply | Threaded
Open this post in threaded view
|

Re: [FYI] Kubernetes GA Preparation (SPARK-33005)

German Schiavon
Hi!
I just run to this same issue while testing k8s in local mode


Note that the tittle shouldn't be  "Unable to disable Kerberos when submitting jobs to Kubernetes" (based on the comments) and something more related with the spark.kubernetes.file.upload.path property

Should we add it too?

On Wed, 30 Sep 2020 at 05:28, Dongjoon Hyun <[hidden email]> wrote:
Thank you!

Bests,
Dongjoon

On Mon, Sep 28, 2020 at 8:07 PM Dr. Kent Yao <[hidden email]> wrote:
Thanks, Dongjon,

   I pined two long-standing issues to the umbrella.



   https://issues.apache.org/jira/browse/SPARK-28895

   https://issues.apache.org/jira/browse/SPARK-28992



   This helps fix some problems of running Spark on k8s application with

HDFS.

   Hoping to get reviewed soon



Best,

Kent Yao







--

Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/



---------------------------------------------------------------------

To unsubscribe e-mail: [hidden email]



Reply | Threaded
Open this post in threaded view
|

Re: [FYI] Kubernetes GA Preparation (SPARK-33005)

Dongjoon Hyun-2
Sure, German.

Please add your comment on that issue. If possible, please provide a reproducible example which you did.



On Sat, Oct 3, 2020 at 10:31 PM German Schiavon <[hidden email]> wrote:
Hi!
I just run to this same issue while testing k8s in local mode


Note that the tittle shouldn't be  "Unable to disable Kerberos when submitting jobs to Kubernetes" (based on the comments) and something more related with the spark.kubernetes.file.upload.path property

Should we add it too?

On Wed, 30 Sep 2020 at 05:28, Dongjoon Hyun <[hidden email]> wrote:
Thank you!

Bests,
Dongjoon

On Mon, Sep 28, 2020 at 8:07 PM Dr. Kent Yao <[hidden email]> wrote:
Thanks, Dongjon,

   I pined two long-standing issues to the umbrella.



   https://issues.apache.org/jira/browse/SPARK-28895

   https://issues.apache.org/jira/browse/SPARK-28992



   This helps fix some problems of running Spark on k8s application with

HDFS.

   Hoping to get reviewed soon



Best,

Kent Yao







--

Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/



---------------------------------------------------------------------

To unsubscribe e-mail: [hidden email]