a stage can belong to more than one job please?

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

a stage can belong to more than one job please?

萝卜丝炒饭
Hi all,

I read same code of spark about stage.

The constructor of stage keep the first job  ID the stage was part of.
does that means a stage can belong to more than one job please? And I find the member jobIds is never used. It looks strange.


thanks adv
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: a stage can belong to more than one job please?

Mark Hamstra
Yes, a Stage can be part of more than one Job. The jobIds field of Stage is used repeatedly in the DAGScheduler.

On Tue, Jun 6, 2017 at 5:04 AM, 萝卜丝炒饭 <[hidden email]> wrote:
Hi all,

I read same code of spark about stage.

The constructor of stage keep the first job  ID the stage was part of.
does that means a stage can belong to more than one job please? And I find the member jobIds is never used. It looks strange.


thanks adv

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: a stage can belong to more than one job please?

萝卜丝炒饭
Hi  Mark,

Thanks.



---Original---
From: "Mark Hamstra"<[hidden email]>
Date: 2017/6/6 23:27:43
To: "dev"<[hidden email]>;
Cc: "user"<[hidden email]>;
Subject: Re: a stage can belong to more than one job please?

Yes, a Stage can be part of more than one Job. The jobIds field of Stage is used repeatedly in the DAGScheduler.

On Tue, Jun 6, 2017 at 5:04 AM, 萝卜丝炒饭 <[hidden email]> wrote:
Hi all,

I read same code of spark about stage.

The constructor of stage keep the first job  ID the stage was part of.
does that means a stage can belong to more than one job please? And I find the member jobIds is never used. It looks strange.


thanks adv

Loading...