Removing references to Master

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Removing references to Master

Tom Graves-2
Hey everyone,

I filed jira https://issues.apache.org/jira/browse/SPARK-32333 to remove references to Master.  I realize this is a bigger change then the slave jira but I wanted to get folks input on if they are ok with making the change and if so we would need to pick a name to use instead.  I think we should keep it backwards compatible at first as to not break anyone and depending on what we find might break it up into multiple smaller liras.

A few name possibilities:
 - ApplicationManager
 - StandaloneClusterManager
 - Coordinator
 - Primary
 - Controller

Thoughts or suggestions?

Thanks,
Tom


Reply | Threaded
Open this post in threaded view
|

Re: Removing references to Master

Holden Karau
I think this is a good idea, and yes keeping it backwards compatible initially is important since we missed the boat on Spark 3. I like the Controller/Leader one since I think that does a good job of reflecting the codes role.

On Tue, Aug 4, 2020 at 7:01 AM Tom Graves <[hidden email]> wrote:
Hey everyone,

I filed jira https://issues.apache.org/jira/browse/SPARK-32333 to remove references to Master.  I realize this is a bigger change then the slave jira but I wanted to get folks input on if they are ok with making the change and if so we would need to pick a name to use instead.  I think we should keep it backwards compatible at first as to not break anyone and depending on what we find might break it up into multiple smaller liras.

A few name possibilities:
 - ApplicationManager
 - StandaloneClusterManager
 - Coordinator
 - Primary
 - Controller

Thoughts or suggestions?

Thanks,
Tom




--
Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9 
Reply | Threaded
Open this post in threaded view
|

Re: Removing references to Master

RussS
I think we should use Scheduler or Comptroller or Leader; something that evokes better describes the purpose as a resource management service. I would rather we didn't use controller, coordinator, application manager, primary because I feel that those terms make it seem like the process is central to an Application's function when in reality it does nothing other than turn off or on containers and processes. The key example here for me would be, if the StandaloneResourceManager goes down, a running app is basically unaffected . The initial usage of "master" was misleading even in context of previous CS usage of the term imho and we should choose a much more limited term to describe it now that we have a chance for a rename. Of course, ymmv and really anything would be better than the current status quo which is both misleading and insensitive.

On Tue, Aug 4, 2020 at 9:08 AM Holden Karau <[hidden email]> wrote:
I think this is a good idea, and yes keeping it backwards compatible initially is important since we missed the boat on Spark 3. I like the Controller/Leader one since I think that does a good job of reflecting the codes role.

On Tue, Aug 4, 2020 at 7:01 AM Tom Graves <[hidden email]> wrote:
Hey everyone,

I filed jira https://issues.apache.org/jira/browse/SPARK-32333 to remove references to Master.  I realize this is a bigger change then the slave jira but I wanted to get folks input on if they are ok with making the change and if so we would need to pick a name to use instead.  I think we should keep it backwards compatible at first as to not break anyone and depending on what we find might break it up into multiple smaller liras.

A few name possibilities:
 - ApplicationManager
 - StandaloneClusterManager
 - Coordinator
 - Primary
 - Controller

Thoughts or suggestions?

Thanks,
Tom




--
Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9 
Reply | Threaded
Open this post in threaded view
|

Re: Removing references to Master

Tom Graves-2
Thanks for the replies so far, is there any other feedback here?    Of the replies so far I think Leader has been mentioned the most.

Tom

On Tuesday, August 4, 2020, 09:33:14 AM CDT, Russell Spitzer <[hidden email]> wrote:


I think we should use Scheduler or Comptroller or Leader; something that evokes better describes the purpose as a resource management service. I would rather we didn't use controller, coordinator, application manager, primary because I feel that those terms make it seem like the process is central to an Application's function when in reality it does nothing other than turn off or on containers and processes. The key example here for me would be, if the StandaloneResourceManager goes down, a running app is basically unaffected . The initial usage of "master" was misleading even in context of previous CS usage of the term imho and we should choose a much more limited term to describe it now that we have a chance for a rename. Of course, ymmv and really anything would be better than the current status quo which is both misleading and insensitive.

On Tue, Aug 4, 2020 at 9:08 AM Holden Karau <[hidden email]> wrote:
I think this is a good idea, and yes keeping it backwards compatible initially is important since we missed the boat on Spark 3. I like the Controller/Leader one since I think that does a good job of reflecting the codes role.

On Tue, Aug 4, 2020 at 7:01 AM Tom Graves <[hidden email]> wrote:
Hey everyone,

I filed jira https://issues.apache.org/jira/browse/SPARK-32333 to remove references to Master.  I realize this is a bigger change then the slave jira but I wanted to get folks input on if they are ok with making the change and if so we would need to pick a name to use instead.  I think we should keep it backwards compatible at first as to not break anyone and depending on what we find might break it up into multiple smaller liras.

A few name possibilities:
 - ApplicationManager
 - StandaloneClusterManager
 - Coordinator
 - Primary
 - Controller

Thoughts or suggestions?

Thanks,
Tom




--
Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9 
Reply | Threaded
Open this post in threaded view
|

Re: Removing references to Master

jpaulorio
What about updating the documentation as well? Does it depend on the codebase
changes or can we treat it as a separate issue? I volunteer to update both
Master and Slave terms when there's an agreement on what should be used as
replacement. Since  [SPARK-32004]
<https://issues.apache.org/jira/browse/SPARK-32004>   was already resolved,
can I start replacing slave with worker?



--
Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Removing references to Master

jpaulorio
So, it looks like slave was already replaced in the docs. Waiting for a definition on the replacement(s) for master so I can create a PR for the docs only.

On Sat, Jan 16, 2021 at 8:30 AM jpaulorio <[hidden email]> wrote:
What about updating the documentation as well? Does it depend on the codebase
changes or can we treat it as a separate issue? I volunteer to update both
Master and Slave terms when there's an agreement on what should be used as
replacement. Since  [SPARK-32004]
<https://issues.apache.org/jira/browse/SPARK-32004>   was already resolved,
can I start replacing slave with worker?



--
Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Removing references to Master

Tom Graves-2
thanks for the interest, I haven't had time to work on replacing Master, hopefully for the next release but time dependent, if you follow the lira - https://issues.apache.org/jira/browse/SPARK-32333 - I will post there when I start or if someone else picks it up should see activity there.

Tom

On Saturday, January 16, 2021, 07:56:14 AM CST, João Paulo Leonidas Fernandes Dias da Silva <[hidden email]> wrote:


So, it looks like slave was already replaced in the docs. Waiting for a definition on the replacement(s) for master so I can create a PR for the docs only.

On Sat, Jan 16, 2021 at 8:30 AM jpaulorio <[hidden email]> wrote:
What about updating the documentation as well? Does it depend on the codebase
changes or can we treat it as a separate issue? I volunteer to update both
Master and Slave terms when there's an agreement on what should be used as
replacement. Since  [SPARK-32004]
<https://issues.apache.org/jira/browse/SPARK-32004>   was already resolved,
can I start replacing slave with worker?



--
Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]