Auto-closing PRs when there are no feedback or response from its author

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Auto-closing PRs when there are no feedback or response from its author

Hyukjin Kwon
Hi all,

I think we talked about this before. Roughly speaking, there are two cases of PRs:
  1. PRs waiting for review and 2. PRs waiting for author's reaction
We might not have to take an action but wait for reviewing for the first case.
However, we can ping and/or take an action for the second case.

I noticed (at Read the Docs, https://github.com/readthedocs/readthedocs.org/blob/master/.github/no-response.yml) there's one bot integrated with Github app that does exactly what we want (see https://github.com/probot/no-response).

1. Maintainers (committers) can add a tag to a PR (e.g., need-more-information)
2. If the PR author responds with a comment or update, the bot removes the tag
3. If the PR author does not respond, the bot closes the PR after waiting for the configured number of days.

We already have a kind of simple mechanism for windowing the number of JIRAs. I think it's time to have such mechanism in Github PR as well.

Although this repo doesn't look popular or widely used enough, seems exactly matched to what we want and less aggressive since this mechanism will only work when maintainers (committers) add a tag to a PR.

WDYT guys?

I cc'ed few people who I think were in the past similar discussions.

Reply | Threaded
Open this post in threaded view
|

Re: Auto-closing PRs when there are no feedback or response from its author

Sean Owen-2
I'm generally all for closing pretty old PRs. They can be reopened
easily. Closing a PR (a particular proposal for how to resolve an
issue) is less drastic than closing a JIRA (a description of an
issue). Closing them just delivers the reality, that nobody is going
to otherwise revisit it, and can actually prompt a few contributors to
update or revisit their proposal.

I wouldn't necessarily want to adopt new process or tools though. Is
it not sufficient to auto-close PRs that have a merge conflict and
haven't been updated in months? or just haven't been updated in a
year? Those are probably manual-ish processes, but, don't need to
happen more than a couple times a year.

If there's little overhead to adoption, cool, though I doubt people
will consistently use a new tag. I'd prefer any process or tool that
implements the above.


On Tue, Oct 8, 2019 at 8:19 PM Hyukjin Kwon <[hidden email]> wrote:

>
> Hi all,
>
> I think we talked about this before. Roughly speaking, there are two cases of PRs:
>   1. PRs waiting for review and 2. PRs waiting for author's reaction
> We might not have to take an action but wait for reviewing for the first case.
> However, we can ping and/or take an action for the second case.
>
> I noticed (at Read the Docs, https://github.com/readthedocs/readthedocs.org/blob/master/.github/no-response.yml) there's one bot integrated with Github app that does exactly what we want (see https://github.com/probot/no-response).
>
> 1. Maintainers (committers) can add a tag to a PR (e.g., need-more-information)
> 2. If the PR author responds with a comment or update, the bot removes the tag
> 3. If the PR author does not respond, the bot closes the PR after waiting for the configured number of days.
>
> We already have a kind of simple mechanism for windowing the number of JIRAs. I think it's time to have such mechanism in Github PR as well.
>
> Although this repo doesn't look popular or widely used enough, seems exactly matched to what we want and less aggressive since this mechanism will only work when maintainers (committers) add a tag to a PR.
>
> WDYT guys?
>
> I cc'ed few people who I think were in the past similar discussions.
>

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Auto-closing PRs when there are no feedback or response from its author

Dongjoon Hyun-2
Thank you for keeping eyes on this difficult issue, Hyukjin.

Although we try our best, there exist some corner cases always. For examples,

1. Although we close old JIRA issues on EOL-version only, but some issues doesn't have `Affected Versions` field  info at all.

2. Although we can do auto-close PRs that have a merge conflict and haven't been updated in months, but some PRs don't have conflicts.
    - https://github.com/apache/spark/pull/7842 (Actually, this is the oldest PR due to the above reason.)

So, I'm +1 for trying to add a new manual tagging process
because I believe it's better than no-activity status and that sounds softer than the direct closing due to the grace-period.

Bests,
Dongjoon.


On Tue, Oct 8, 2019 at 7:26 PM Sean Owen <[hidden email]> wrote:
I'm generally all for closing pretty old PRs. They can be reopened
easily. Closing a PR (a particular proposal for how to resolve an
issue) is less drastic than closing a JIRA (a description of an
issue). Closing them just delivers the reality, that nobody is going
to otherwise revisit it, and can actually prompt a few contributors to
update or revisit their proposal.

I wouldn't necessarily want to adopt new process or tools though. Is
it not sufficient to auto-close PRs that have a merge conflict and
haven't been updated in months? or just haven't been updated in a
year? Those are probably manual-ish processes, but, don't need to
happen more than a couple times a year.

If there's little overhead to adoption, cool, though I doubt people
will consistently use a new tag. I'd prefer any process or tool that
implements the above.


On Tue, Oct 8, 2019 at 8:19 PM Hyukjin Kwon <[hidden email]> wrote:
>
> Hi all,
>
> I think we talked about this before. Roughly speaking, there are two cases of PRs:
>   1. PRs waiting for review and 2. PRs waiting for author's reaction
> We might not have to take an action but wait for reviewing for the first case.
> However, we can ping and/or take an action for the second case.
>
> I noticed (at Read the Docs, https://github.com/readthedocs/readthedocs.org/blob/master/.github/no-response.yml) there's one bot integrated with Github app that does exactly what we want (see https://github.com/probot/no-response).
>
> 1. Maintainers (committers) can add a tag to a PR (e.g., need-more-information)
> 2. If the PR author responds with a comment or update, the bot removes the tag
> 3. If the PR author does not respond, the bot closes the PR after waiting for the configured number of days.
>
> We already have a kind of simple mechanism for windowing the number of JIRAs. I think it's time to have such mechanism in Github PR as well.
>
> Although this repo doesn't look popular or widely used enough, seems exactly matched to what we want and less aggressive since this mechanism will only work when maintainers (committers) add a tag to a PR.
>
> WDYT guys?
>
> I cc'ed few people who I think were in the past similar discussions.
>
Reply | Threaded
Open this post in threaded view
|

Re: Auto-closing PRs when there are no feedback or response from its author

Hyukjin Kwon
> 1. Although we close old JIRA issues on EOL-version only, but some issues doesn't have `Affected Versions` field  info at all.

For this case actually, I thought we resolved such cases all .. maybe some of them slipped out of my hand.
Few years ago, we made affected version a required field:
Screen Shot 2019-10-09 at 3.36.15 PM.png
It should be good to resolve them at least to let reporters to update the affected versions. and all such JIRAs will be old JIRAs anyway.


> 2. Although we can do auto-close PRs that have a merge conflict and haven't been updated in months, but some PRs don't have conflicts.
>     - https://github.com/apache/spark/pull/7842 (Actually, this is the oldest PR due to the above reason.)

Yea, this is a good point. This might be one of the reasons for that manual tagging way to identify case by case.



2019년 10월 9일 (수) 오후 3:02, Dongjoon Hyun <[hidden email]>님이 작성:
Thank you for keeping eyes on this difficult issue, Hyukjin.

Although we try our best, there exist some corner cases always. For examples,

1. Although we close old JIRA issues on EOL-version only, but some issues doesn't have `Affected Versions` field  info at all.

2. Although we can do auto-close PRs that have a merge conflict and haven't been updated in months, but some PRs don't have conflicts.
    - https://github.com/apache/spark/pull/7842 (Actually, this is the oldest PR due to the above reason.)

So, I'm +1 for trying to add a new manual tagging process
because I believe it's better than no-activity status and that sounds softer than the direct closing due to the grace-period.

Bests,
Dongjoon.


On Tue, Oct 8, 2019 at 7:26 PM Sean Owen <[hidden email]> wrote:
I'm generally all for closing pretty old PRs. They can be reopened
easily. Closing a PR (a particular proposal for how to resolve an
issue) is less drastic than closing a JIRA (a description of an
issue). Closing them just delivers the reality, that nobody is going
to otherwise revisit it, and can actually prompt a few contributors to
update or revisit their proposal.

I wouldn't necessarily want to adopt new process or tools though. Is
it not sufficient to auto-close PRs that have a merge conflict and
haven't been updated in months? or just haven't been updated in a
year? Those are probably manual-ish processes, but, don't need to
happen more than a couple times a year.

If there's little overhead to adoption, cool, though I doubt people
will consistently use a new tag. I'd prefer any process or tool that
implements the above.


On Tue, Oct 8, 2019 at 8:19 PM Hyukjin Kwon <[hidden email]> wrote:
>
> Hi all,
>
> I think we talked about this before. Roughly speaking, there are two cases of PRs:
>   1. PRs waiting for review and 2. PRs waiting for author's reaction
> We might not have to take an action but wait for reviewing for the first case.
> However, we can ping and/or take an action for the second case.
>
> I noticed (at Read the Docs, https://github.com/readthedocs/readthedocs.org/blob/master/.github/no-response.yml) there's one bot integrated with Github app that does exactly what we want (see https://github.com/probot/no-response).
>
> 1. Maintainers (committers) can add a tag to a PR (e.g., need-more-information)
> 2. If the PR author responds with a comment or update, the bot removes the tag
> 3. If the PR author does not respond, the bot closes the PR after waiting for the configured number of days.
>
> We already have a kind of simple mechanism for windowing the number of JIRAs. I think it's time to have such mechanism in Github PR as well.
>
> Although this repo doesn't look popular or widely used enough, seems exactly matched to what we want and less aggressive since this mechanism will only work when maintainers (committers) add a tag to a PR.
>
> WDYT guys?
>
> I cc'ed few people who I think were in the past similar discussions.
>
Reply | Threaded
Open this post in threaded view
|

Re: Auto-closing PRs when there are no feedback or response from its author

Hyukjin Kwon
In reply to this post by Sean Owen-2
Yes, the problem was that it is difficult to automate. I think this has been discussed twice(?) in the mailing list;
however, it ended up with doing nothing because it was difficult to automate.

I think in case of PRs unlike JIRAs, there are some more different cases that need manual judgement.

As an example, while JIRAs are easy to keep it updated in general, I think it might not be fair to request to keep updating and resolving
conflicts of a PR with indefinitely waiting for review. For some large PRs, it's kind of painful to keep it updated always.
It might be more reasonable to be updated per request when a committer has some time to review.

> If there's little overhead to adoption, cool, though I doubt people
> will consistently use a new tag.

Yea, this is a good point. But in fact the standard about when to use is quite simple - in a PR, leave a comment or review and tag this.
In case of readthedocs, they seem always tagging this whenever they leave a comment or responds.

In fact, I myself am not sure about how useful it would be but to me it looked worth trying. I remember we tried
such bots and dropped it back when it is found practically not quite useful.



2019년 10월 9일 (수) 오전 11:26, Sean Owen <[hidden email]>님이 작성:
I'm generally all for closing pretty old PRs. They can be reopened
easily. Closing a PR (a particular proposal for how to resolve an
issue) is less drastic than closing a JIRA (a description of an
issue). Closing them just delivers the reality, that nobody is going
to otherwise revisit it, and can actually prompt a few contributors to
update or revisit their proposal.

I wouldn't necessarily want to adopt new process or tools though. Is
it not sufficient to auto-close PRs that have a merge conflict and
haven't been updated in months? or just haven't been updated in a
year? Those are probably manual-ish processes, but, don't need to
happen more than a couple times a year.

If there's little overhead to adoption, cool, though I doubt people
will consistently use a new tag. I'd prefer any process or tool that
implements the above.


On Tue, Oct 8, 2019 at 8:19 PM Hyukjin Kwon <[hidden email]> wrote:
>
> Hi all,
>
> I think we talked about this before. Roughly speaking, there are two cases of PRs:
>   1. PRs waiting for review and 2. PRs waiting for author's reaction
> We might not have to take an action but wait for reviewing for the first case.
> However, we can ping and/or take an action for the second case.
>
> I noticed (at Read the Docs, https://github.com/readthedocs/readthedocs.org/blob/master/.github/no-response.yml) there's one bot integrated with Github app that does exactly what we want (see https://github.com/probot/no-response).
>
> 1. Maintainers (committers) can add a tag to a PR (e.g., need-more-information)
> 2. If the PR author responds with a comment or update, the bot removes the tag
> 3. If the PR author does not respond, the bot closes the PR after waiting for the configured number of days.
>
> We already have a kind of simple mechanism for windowing the number of JIRAs. I think it's time to have such mechanism in Github PR as well.
>
> Although this repo doesn't look popular or widely used enough, seems exactly matched to what we want and less aggressive since this mechanism will only work when maintainers (committers) add a tag to a PR.
>
> WDYT guys?
>
> I cc'ed few people who I think were in the past similar discussions.
>