Removing references to Master

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Removing references to Master

Tom Graves-2
Hey everyone,

I filed jira https://issues.apache.org/jira/browse/SPARK-32333 to remove references to Master.  I realize this is a bigger change then the slave jira but I wanted to get folks input on if they are ok with making the change and if so we would need to pick a name to use instead.  I think we should keep it backwards compatible at first as to not break anyone and depending on what we find might break it up into multiple smaller liras.

A few name possibilities:
 - ApplicationManager
 - StandaloneClusterManager
 - Coordinator
 - Primary
 - Controller

Thoughts or suggestions?

Thanks,
Tom


Reply | Threaded
Open this post in threaded view
|

Re: Removing references to Master

Holden Karau
I think this is a good idea, and yes keeping it backwards compatible initially is important since we missed the boat on Spark 3. I like the Controller/Leader one since I think that does a good job of reflecting the codes role.

On Tue, Aug 4, 2020 at 7:01 AM Tom Graves <[hidden email]> wrote:
Hey everyone,

I filed jira https://issues.apache.org/jira/browse/SPARK-32333 to remove references to Master.  I realize this is a bigger change then the slave jira but I wanted to get folks input on if they are ok with making the change and if so we would need to pick a name to use instead.  I think we should keep it backwards compatible at first as to not break anyone and depending on what we find might break it up into multiple smaller liras.

A few name possibilities:
 - ApplicationManager
 - StandaloneClusterManager
 - Coordinator
 - Primary
 - Controller

Thoughts or suggestions?

Thanks,
Tom




--
Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9 
Reply | Threaded
Open this post in threaded view
|

Re: Removing references to Master

RussS
I think we should use Scheduler or Comptroller or Leader; something that evokes better describes the purpose as a resource management service. I would rather we didn't use controller, coordinator, application manager, primary because I feel that those terms make it seem like the process is central to an Application's function when in reality it does nothing other than turn off or on containers and processes. The key example here for me would be, if the StandaloneResourceManager goes down, a running app is basically unaffected . The initial usage of "master" was misleading even in context of previous CS usage of the term imho and we should choose a much more limited term to describe it now that we have a chance for a rename. Of course, ymmv and really anything would be better than the current status quo which is both misleading and insensitive.

On Tue, Aug 4, 2020 at 9:08 AM Holden Karau <[hidden email]> wrote:
I think this is a good idea, and yes keeping it backwards compatible initially is important since we missed the boat on Spark 3. I like the Controller/Leader one since I think that does a good job of reflecting the codes role.

On Tue, Aug 4, 2020 at 7:01 AM Tom Graves <[hidden email]> wrote:
Hey everyone,

I filed jira https://issues.apache.org/jira/browse/SPARK-32333 to remove references to Master.  I realize this is a bigger change then the slave jira but I wanted to get folks input on if they are ok with making the change and if so we would need to pick a name to use instead.  I think we should keep it backwards compatible at first as to not break anyone and depending on what we find might break it up into multiple smaller liras.

A few name possibilities:
 - ApplicationManager
 - StandaloneClusterManager
 - Coordinator
 - Primary
 - Controller

Thoughts or suggestions?

Thanks,
Tom




--
Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9 
Reply | Threaded
Open this post in threaded view
|

Re: Removing references to Master

Tom Graves-2
Thanks for the replies so far, is there any other feedback here?    Of the replies so far I think Leader has been mentioned the most.

Tom

On Tuesday, August 4, 2020, 09:33:14 AM CDT, Russell Spitzer <[hidden email]> wrote:


I think we should use Scheduler or Comptroller or Leader; something that evokes better describes the purpose as a resource management service. I would rather we didn't use controller, coordinator, application manager, primary because I feel that those terms make it seem like the process is central to an Application's function when in reality it does nothing other than turn off or on containers and processes. The key example here for me would be, if the StandaloneResourceManager goes down, a running app is basically unaffected . The initial usage of "master" was misleading even in context of previous CS usage of the term imho and we should choose a much more limited term to describe it now that we have a chance for a rename. Of course, ymmv and really anything would be better than the current status quo which is both misleading and insensitive.

On Tue, Aug 4, 2020 at 9:08 AM Holden Karau <[hidden email]> wrote:
I think this is a good idea, and yes keeping it backwards compatible initially is important since we missed the boat on Spark 3. I like the Controller/Leader one since I think that does a good job of reflecting the codes role.

On Tue, Aug 4, 2020 at 7:01 AM Tom Graves <[hidden email]> wrote:
Hey everyone,

I filed jira https://issues.apache.org/jira/browse/SPARK-32333 to remove references to Master.  I realize this is a bigger change then the slave jira but I wanted to get folks input on if they are ok with making the change and if so we would need to pick a name to use instead.  I think we should keep it backwards compatible at first as to not break anyone and depending on what we find might break it up into multiple smaller liras.

A few name possibilities:
 - ApplicationManager
 - StandaloneClusterManager
 - Coordinator
 - Primary
 - Controller

Thoughts or suggestions?

Thanks,
Tom




--
Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9