Spark version for Mesos 0.27.0

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Spark version for Mesos 0.27.0

Thodoris Zois
Hello,

Could you please tell me which version of Spark works with Apache Mesos
 version 0.27.0? (I cannot find anything on docs at github)

Thank you very much,
Thodoris Zois

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Spark version for Mesos 0.27.0

Jacek Laskowski
Hi,

Mesos 0.27.0?! That's been a while. I'd search for the changes to pom.xml and see when the mesos dependency version changed. That'd give you the most precise answer. I think it could've been 1.5 or older.


On Fri, May 25, 2018 at 1:29 PM, Thodoris Zois <[hidden email]> wrote:
Hello,

Could you please tell me which version of Spark works with Apache Mesos
 version 0.27.0? (I cannot find anything on docs at github)

Thank you very much,
Thodoris Zois

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]


Reply | Threaded
Open this post in threaded view
|

Re: Spark version for Mesos 0.27.0

Thodoris Zois
Hello,

I need Mesos 0.27 for specific purposes and unfortunately I can’t use a newer version. Did you find anything? Could it be Spark 1.6? 

Except that, from which version Spark supports dynamic allocation on Mesos? 

- Thodoris

On 25 May 2018, at 16:06, Jacek Laskowski <[hidden email]> wrote:

Hi,

Mesos 0.27.0?! That's been a while. I'd search for the changes to pom.xml and see when the mesos dependency version changed. That'd give you the most precise answer. I think it could've been 1.5 or older.


On Fri, May 25, 2018 at 1:29 PM, Thodoris Zois <[hidden email]> wrote:
Hello,

Could you please tell me which version of Spark works with Apache Mesos
 version 0.27.0? (I cannot find anything on docs at github)

Thank you very much,
Thodoris Zois

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]


Reply | Threaded
Open this post in threaded view
|

Re: Spark version for Mesos 0.27.0

Szuromi Tamás
Hey,

I'm sure we used Spark 1.6 on Mesos 0.27 as well but at that time we used with fine-grained scheduling and not dynamic allocation. Also, newer Spark versions should work with older mesos versions like 0.27.
Why do you have Mesos 0.27 btw?

cheers,
Tamas

Thodoris Zois <[hidden email]> ezt írta (időpont: 2018. máj. 31., Cs, 1:56):
Hello,

I need Mesos 0.27 for specific purposes and unfortunately I can’t use a newer version. Did you find anything? Could it be Spark 1.6? 

Except that, from which version Spark supports dynamic allocation on Mesos? 

- Thodoris

On 25 May 2018, at 16:06, Jacek Laskowski <[hidden email]> wrote:

Hi,

Mesos 0.27.0?! That's been a while. I'd search for the changes to pom.xml and see when the mesos dependency version changed. That'd give you the most precise answer. I think it could've been 1.5 or older.


On Fri, May 25, 2018 at 1:29 PM, Thodoris Zois <[hidden email]> wrote:
Hello,

Could you please tell me which version of Spark works with Apache Mesos
 version 0.27.0? (I cannot find anything on docs at github)

Thank you very much,
Thodoris Zois

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]


Reply | Threaded
Open this post in threaded view
|

Re: Spark version for Mesos 0.27.0

Thodoris Zois
Hello,

The reason is that I want to make some tests and use the oversubscription feature of Mesos along with Spark. Intel and Mesosphere have built a project, called Serenity that actually measures the usage slack on each Mesos agent and returns resources to the cluster.

Unfortunately, Serenity is not compatible with newer versions of Mesos...

- Thodoris


On Thu, 2018-05-31 at 11:08 +0200, Szuromi Tamás wrote:
Hey,

I'm sure we used Spark 1.6 on Mesos 0.27 as well but at that time we used with fine-grained scheduling and not dynamic allocation. Also, newer Spark versions should work with older mesos versions like 0.27.
Why do you have Mesos 0.27 btw?

cheers,
Tamas

Thodoris Zois <[hidden email]> ezt írta (időpont: 2018. máj. 31., Cs, 1:56):
Hello,

I need Mesos 0.27 for specific purposes and unfortunately I can’t use a newer version. Did you find anything? Could it be Spark 1.6? 

Except that, from which version Spark supports dynamic allocation on Mesos? 

- Thodoris

On 25 May 2018, at 16:06, Jacek Laskowski <[hidden email]> wrote:

Hi,

Mesos 0.27.0?! That's been a while. I'd search for the changes to pom.xml and see when the mesos dependency version changed. That'd give you the most precise answer. I think it could've been 1.5 or older.


On Fri, May 25, 2018 at 1:29 PM, Thodoris Zois <[hidden email]> wrote:
Hello,

Could you please tell me which version of Spark works with Apache Mesos
 version 0.27.0? (I cannot find anything on docs at github)

Thank you very much,
Thodoris Zois

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]


Reply | Threaded
Open this post in threaded view
|

Re: Spark version for Mesos 0.27.0

Szuromi Tamás
I see it in the Serenity docs. Anyway, I guess you are able to use the newest version of Spark with the Mesos 0.27 without any issues so don't have to dispense newer Spark features and fixes.

Thodoris Zois <[hidden email]> ezt írta (időpont: 2018. máj. 31., Cs, 11:22):
Hello,

The reason is that I want to make some tests and use the oversubscription feature of Mesos along with Spark. Intel and Mesosphere have built a project, called Serenity that actually measures the usage slack on each Mesos agent and returns resources to the cluster.

Unfortunately, Serenity is not compatible with newer versions of Mesos...

- Thodoris


On Thu, 2018-05-31 at 11:08 +0200, Szuromi Tamás wrote:
Hey,

I'm sure we used Spark 1.6 on Mesos 0.27 as well but at that time we used with fine-grained scheduling and not dynamic allocation. Also, newer Spark versions should work with older mesos versions like 0.27.
Why do you have Mesos 0.27 btw?

cheers,
Tamas

Thodoris Zois <[hidden email]> ezt írta (időpont: 2018. máj. 31., Cs, 1:56):
Hello,

I need Mesos 0.27 for specific purposes and unfortunately I can’t use a newer version. Did you find anything? Could it be Spark 1.6? 

Except that, from which version Spark supports dynamic allocation on Mesos? 

- Thodoris

On 25 May 2018, at 16:06, Jacek Laskowski <[hidden email]> wrote:

Hi,

Mesos 0.27.0?! That's been a while. I'd search for the changes to pom.xml and see when the mesos dependency version changed. That'd give you the most precise answer. I think it could've been 1.5 or older.


On Fri, May 25, 2018 at 1:29 PM, Thodoris Zois <[hidden email]> wrote:
Hello,

Could you please tell me which version of Spark works with Apache Mesos
 version 0.27.0? (I cannot find anything on docs at github)

Thank you very much,
Thodoris Zois

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]


Reply | Threaded
Open this post in threaded view
|

Re: Spark version for Mesos 0.27.0

Thodoris Zois
Ok! Thank you very much!

- Thodoris

On Thu, 2018-05-31 at 11:30 +0200, Szuromi Tamás wrote:
I see it in the Serenity docs. Anyway, I guess you are able to use the newest version of Spark with the Mesos 0.27 without any issues so don't have to dispense newer Spark features and fixes.

Thodoris Zois <[hidden email]> ezt írta (időpont: 2018. máj. 31., Cs, 11:22):
Hello,

The reason is that I want to make some tests and use the oversubscription feature of Mesos along with Spark. Intel and Mesosphere have built a project, called Serenity that actually measures the usage slack on each Mesos agent and returns resources to the cluster.

Unfortunately, Serenity is not compatible with newer versions of Mesos...

- Thodoris


On Thu, 2018-05-31 at 11:08 +0200, Szuromi Tamás wrote:
Hey,

I'm sure we used Spark 1.6 on Mesos 0.27 as well but at that time we used with fine-grained scheduling and not dynamic allocation. Also, newer Spark versions should work with older mesos versions like 0.27.
Why do you have Mesos 0.27 btw?

cheers,
Tamas

Thodoris Zois <[hidden email]> ezt írta (időpont: 2018. máj. 31., Cs, 1:56):
Hello,

I need Mesos 0.27 for specific purposes and unfortunately I can’t use a newer version. Did you find anything? Could it be Spark 1.6? 

Except that, from which version Spark supports dynamic allocation on Mesos? 

- Thodoris

On 25 May 2018, at 16:06, Jacek Laskowski <[hidden email]> wrote:

Hi,

Mesos 0.27.0?! That's been a while. I'd search for the changes to pom.xml and see when the mesos dependency version changed. That'd give you the most precise answer. I think it could've been 1.5 or older.


On Fri, May 25, 2018 at 1:29 PM, Thodoris Zois <[hidden email]> wrote:
Hello,

Could you please tell me which version of Spark works with Apache Mesos
 version 0.27.0? (I cannot find anything on docs at github)

Thank you very much,
Thodoris Zois

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]