In Apache Spark JIRA, spark/dev/github_jira_sync.py not running properly

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

In Apache Spark JIRA, spark/dev/github_jira_sync.py not running properly

Hyukjin Kwon
Hi all,

Looks 'spark/dev/github_jira_sync.py' is not running correctly somewhere.
Usually the JIRA's status should be updated to "IN PROGRESS" when
somebody opens a PR against a JIRA.
Looks now it only leaves a link and does not change JIRA's status.

Can someone else who knows where it's running can check this?

FWIW, I check every PR and JIRA almost every day but ever since this happened, this makes (at least to me) duplicately check the JIRAs.
Previously, if I check all the PRs and JIRAs, they were not duplicated because JIRAs having PRs have different status, "IN PROGRESS" but now all JIRAs have "OPEN" status.

Thanks.
Reply | Threaded
Open this post in threaded view
|

Re: In Apache Spark JIRA, spark/dev/github_jira_sync.py not running properly

Hyukjin Kwon
Can anyone take a look for this one? OPEN status JIRAs are being rapidly increased (from around 2400 to 2600)

2019년 4월 19일 (금) 오후 8:05, Hyukjin Kwon <[hidden email]>님이 작성:
Hi all,

Looks 'spark/dev/github_jira_sync.py' is not running correctly somewhere.
Usually the JIRA's status should be updated to "IN PROGRESS" when
somebody opens a PR against a JIRA.
Looks now it only leaves a link and does not change JIRA's status.

Can someone else who knows where it's running can check this?

FWIW, I check every PR and JIRA almost every day but ever since this happened, this makes (at least to me) duplicately check the JIRAs.
Previously, if I check all the PRs and JIRAs, they were not duplicated because JIRAs having PRs have different status, "IN PROGRESS" but now all JIRAs have "OPEN" status.

Thanks.
Reply | Threaded
Open this post in threaded view
|

Re: In Apache Spark JIRA, spark/dev/github_jira_sync.py not running properly

Josh Rosen
The code for this runs in http://spark-prs.appspot.com (see https://github.com/databricks/spark-pr-dashboard/blob/1e799c9e510fa8cdc9a6c084a777436bebeabe10/sparkprs/controllers/tasks.py#L137)

I checked the AppEngine logs and it looks like we're getting error responses, possibly due to a credentials issue:

Exception when starting progress on JIRA issue SPARK-27355 (/base/data/home/apps/s~spark-prs/live.412416057856832734/sparkprs/controllers/tasks.py:142) Traceback (most recent call last): File
Traceback (most recent call last):
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/sparkprs/controllers/tasks.py", line 138, in update_pr start_issue_progress("%s-%s" % (app.config['JIRA_PROJECT'], issue_number)) File
start_issue_progress("%s-%s" % (app.config['JIRA_PROJECT'], issue_number))
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/sparkprs/jira_api.py", line 27, in start_issue_progress jira_client = get_jira_client() File
jira_client = get_jira_client()
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/sparkprs/jira_api.py", line 18, in get_jira_client app.config['JIRA_PASSWORD'])) File
app.config['JIRA_PASSWORD']))
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/client.py", line 472, in __init__ si = self.server_info() File
si = self.server_info()
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/client.py", line 2133, in server_info j = self._get_json('serverInfo') File
j = self._get_json('serverInfo')
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/client.py", line 2549, in _get_json r = self._session.get(url, params=params) File
r = self._session.get(url, params=params)
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/resilientsession.py", line 151, in get return self.__verb('GET', url, **kwargs) File
return self.__verb('GET', url, **kwargs)
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/resilientsession.py", line 147, in __verb raise_on_error(response, verb=verb, **kwargs) File
raise_on_error(response, verb=verb, **kwargs)
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/resilientsession.py", line 57, in raise_on_error r.status_code, error, r.url, request=request, response=r, **kwargs) JIRAError: JiraError HTTP 403 url: https://issues.apache.org/jira/rest/api/2/serverInfo text: CAPTCHA_CHALLENGE; login-url=https://issues.apache.org/jira/login.jsp r.status_code, error, r.url, request=request, response=r, **kwargs)
JIRAError: JiraError HTTP 403 url: https://issues.apache.org/jira/rest/api/2/serverInfo
text: CAPTCHA_CHALLENGE; login-url=https://issues.apache.org/jira/login.jsp

It looks like ASF JIRA was throwing a captcha challenge at us, so I used the credentials to manually log in and complete the challenge.

Hopefully that's enough to fix things, but to prevent re-occurrence we might need to change the login credential type from username + password to instead use OAuth tokens.

On Wed, Apr 24, 2019 at 10:42 PM Hyukjin Kwon <[hidden email]> wrote:
Can anyone take a look for this one? OPEN status JIRAs are being rapidly increased (from around 2400 to 2600)

2019년 4월 19일 (금) 오후 8:05, Hyukjin Kwon <[hidden email]>님이 작성:
Hi all,

Looks 'spark/dev/github_jira_sync.py' is not running correctly somewhere.
Usually the JIRA's status should be updated to "IN PROGRESS" when
somebody opens a PR against a JIRA.
Looks now it only leaves a link and does not change JIRA's status.

Can someone else who knows where it's running can check this?

FWIW, I check every PR and JIRA almost every day but ever since this happened, this makes (at least to me) duplicately check the JIRAs.
Previously, if I check all the PRs and JIRAs, they were not duplicated because JIRAs having PRs have different status, "IN PROGRESS" but now all JIRAs have "OPEN" status.

Thanks.
Reply | Threaded
Open this post in threaded view
|

Re: In Apache Spark JIRA, spark/dev/github_jira_sync.py not running properly

Hyukjin Kwon
Thank you so much Josh .. !!

2019년 4월 25일 (목) 오후 3:04, Josh Rosen <[hidden email]>님이 작성:
The code for this runs in http://spark-prs.appspot.com (see https://github.com/databricks/spark-pr-dashboard/blob/1e799c9e510fa8cdc9a6c084a777436bebeabe10/sparkprs/controllers/tasks.py#L137)

I checked the AppEngine logs and it looks like we're getting error responses, possibly due to a credentials issue:

Exception when starting progress on JIRA issue SPARK-27355 (/base/data/home/apps/s~spark-prs/live.412416057856832734/sparkprs/controllers/tasks.py:142) Traceback (most recent call last): File
Traceback (most recent call last):
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/sparkprs/controllers/tasks.py", line 138, in update_pr start_issue_progress("%s-%s" % (app.config['JIRA_PROJECT'], issue_number)) File
start_issue_progress("%s-%s" % (app.config['JIRA_PROJECT'], issue_number))
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/sparkprs/jira_api.py", line 27, in start_issue_progress jira_client = get_jira_client() File
jira_client = get_jira_client()
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/sparkprs/jira_api.py", line 18, in get_jira_client app.config['JIRA_PASSWORD'])) File
app.config['JIRA_PASSWORD']))
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/client.py", line 472, in __init__ si = self.server_info() File
si = self.server_info()
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/client.py", line 2133, in server_info j = self._get_json('serverInfo') File
j = self._get_json('serverInfo')
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/client.py", line 2549, in _get_json r = self._session.get(url, params=params) File
r = self._session.get(url, params=params)
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/resilientsession.py", line 151, in get return self.__verb('GET', url, **kwargs) File
return self.__verb('GET', url, **kwargs)
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/resilientsession.py", line 147, in __verb raise_on_error(response, verb=verb, **kwargs) File
raise_on_error(response, verb=verb, **kwargs)
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/resilientsession.py", line 57, in raise_on_error r.status_code, error, r.url, request=request, response=r, **kwargs) JIRAError: JiraError HTTP 403 url: https://issues.apache.org/jira/rest/api/2/serverInfo text: CAPTCHA_CHALLENGE; login-url=https://issues.apache.org/jira/login.jsp r.status_code, error, r.url, request=request, response=r, **kwargs)
JIRAError: JiraError HTTP 403 url: https://issues.apache.org/jira/rest/api/2/serverInfo
text: CAPTCHA_CHALLENGE; login-url=https://issues.apache.org/jira/login.jsp

It looks like ASF JIRA was throwing a captcha challenge at us, so I used the credentials to manually log in and complete the challenge.

Hopefully that's enough to fix things, but to prevent re-occurrence we might need to change the login credential type from username + password to instead use OAuth tokens.

On Wed, Apr 24, 2019 at 10:42 PM Hyukjin Kwon <[hidden email]> wrote:
Can anyone take a look for this one? OPEN status JIRAs are being rapidly increased (from around 2400 to 2600)

2019년 4월 19일 (금) 오후 8:05, Hyukjin Kwon <[hidden email]>님이 작성:
Hi all,

Looks 'spark/dev/github_jira_sync.py' is not running correctly somewhere.
Usually the JIRA's status should be updated to "IN PROGRESS" when
somebody opens a PR against a JIRA.
Looks now it only leaves a link and does not change JIRA's status.

Can someone else who knows where it's running can check this?

FWIW, I check every PR and JIRA almost every day but ever since this happened, this makes (at least to me) duplicately check the JIRAs.
Previously, if I check all the PRs and JIRAs, they were not duplicated because JIRAs having PRs have different status, "IN PROGRESS" but now all JIRAs have "OPEN" status.

Thanks.
Reply | Threaded
Open this post in threaded view
|

Re: In Apache Spark JIRA, spark/dev/github_jira_sync.py not running properly

Hyukjin Kwon
Hi all,

Seems this issue is re-happening again. Seems the PR link is properly created in the corresponding JIRA but it doesn't change the JIRA's status from OPEN to IN-PROGRESS.

See, for instance,

https://issues.apache.org/jira/browse/SPARK-28443
https://issues.apache.org/jira/browse/SPARK-28440
https://issues.apache.org/jira/browse/SPARK-28436
https://issues.apache.org/jira/browse/SPARK-28434
https://issues.apache.org/jira/browse/SPARK-28433
https://issues.apache.org/jira/browse/SPARK-28431

Josh and Dongjoon, do you guys maybe have any idea?

2019년 4월 25일 (목) 오후 3:09, Hyukjin Kwon <[hidden email]>님이 작성:
Thank you so much Josh .. !!

2019년 4월 25일 (목) 오후 3:04, Josh Rosen <[hidden email]>님이 작성:
The code for this runs in http://spark-prs.appspot.com (see https://github.com/databricks/spark-pr-dashboard/blob/1e799c9e510fa8cdc9a6c084a777436bebeabe10/sparkprs/controllers/tasks.py#L137)

I checked the AppEngine logs and it looks like we're getting error responses, possibly due to a credentials issue:

Exception when starting progress on JIRA issue SPARK-27355 (/base/data/home/apps/s~spark-prs/live.412416057856832734/sparkprs/controllers/tasks.py:142) Traceback (most recent call last): File
Traceback (most recent call last):
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/sparkprs/controllers/tasks.py", line 138, in update_pr start_issue_progress("%s-%s" % (app.config['JIRA_PROJECT'], issue_number)) File
start_issue_progress("%s-%s" % (app.config['JIRA_PROJECT'], issue_number))
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/sparkprs/jira_api.py", line 27, in start_issue_progress jira_client = get_jira_client() File
jira_client = get_jira_client()
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/sparkprs/jira_api.py", line 18, in get_jira_client app.config['JIRA_PASSWORD'])) File
app.config['JIRA_PASSWORD']))
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/client.py", line 472, in __init__ si = self.server_info() File
si = self.server_info()
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/client.py", line 2133, in server_info j = self._get_json('serverInfo') File
j = self._get_json('serverInfo')
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/client.py", line 2549, in _get_json r = self._session.get(url, params=params) File
r = self._session.get(url, params=params)
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/resilientsession.py", line 151, in get return self.__verb('GET', url, **kwargs) File
return self.__verb('GET', url, **kwargs)
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/resilientsession.py", line 147, in __verb raise_on_error(response, verb=verb, **kwargs) File
raise_on_error(response, verb=verb, **kwargs)
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/resilientsession.py", line 57, in raise_on_error r.status_code, error, r.url, request=request, response=r, **kwargs) JIRAError: JiraError HTTP 403 url: https://issues.apache.org/jira/rest/api/2/serverInfo text: CAPTCHA_CHALLENGE; login-url=https://issues.apache.org/jira/login.jsp r.status_code, error, r.url, request=request, response=r, **kwargs)
JIRAError: JiraError HTTP 403 url: https://issues.apache.org/jira/rest/api/2/serverInfo
text: CAPTCHA_CHALLENGE; login-url=https://issues.apache.org/jira/login.jsp

It looks like ASF JIRA was throwing a captcha challenge at us, so I used the credentials to manually log in and complete the challenge.

Hopefully that's enough to fix things, but to prevent re-occurrence we might need to change the login credential type from username + password to instead use OAuth tokens.

On Wed, Apr 24, 2019 at 10:42 PM Hyukjin Kwon <[hidden email]> wrote:
Can anyone take a look for this one? OPEN status JIRAs are being rapidly increased (from around 2400 to 2600)

2019년 4월 19일 (금) 오후 8:05, Hyukjin Kwon <[hidden email]>님이 작성:
Hi all,

Looks 'spark/dev/github_jira_sync.py' is not running correctly somewhere.
Usually the JIRA's status should be updated to "IN PROGRESS" when
somebody opens a PR against a JIRA.
Looks now it only leaves a link and does not change JIRA's status.

Can someone else who knows where it's running can check this?

FWIW, I check every PR and JIRA almost every day but ever since this happened, this makes (at least to me) duplicately check the JIRAs.
Previously, if I check all the PRs and JIRAs, they were not duplicated because JIRAs having PRs have different status, "IN PROGRESS" but now all JIRAs have "OPEN" status.

Thanks.
Reply | Threaded
Open this post in threaded view
|

Re: In Apache Spark JIRA, spark/dev/github_jira_sync.py not running properly

Dongjoon Hyun-2
Hi, Hyukjin.

In short, there are two bots. And, the current situation happens when only one bot with `dev/github_jira_sync.py` works.

And, `dev/github_jira_sync.py` is irrelevant to the JIRA status change because it only use `add_remote_link` and `add_comment` API.
I know only this bot (in Apache Spark repository repo)

AFAIK, `deb/github_jira_sync.py`'s activity is done under JIRA ID `githubbot` (Name: `ASF GitHub Bot`).
And, the other bot's activity is done under JIRA ID `apachespark` (Name: `Apache Spark`).
The other bot is the one which Josh mentioned before. (in `databricks/spark-pr-dashboard` repo).

The root cause will be the same. The API key used by the bot is rejected by Apache JIRA and forwarded to CAPCHAR.

Bests,
Dongjoon.

On Thu, Jul 18, 2019 at 8:24 PM Hyukjin Kwon <[hidden email]> wrote:
Hi all,

Seems this issue is re-happening again. Seems the PR link is properly created in the corresponding JIRA but it doesn't change the JIRA's status from OPEN to IN-PROGRESS.

See, for instance,

https://issues.apache.org/jira/browse/SPARK-28443
https://issues.apache.org/jira/browse/SPARK-28440
https://issues.apache.org/jira/browse/SPARK-28436
https://issues.apache.org/jira/browse/SPARK-28434
https://issues.apache.org/jira/browse/SPARK-28433
https://issues.apache.org/jira/browse/SPARK-28431

Josh and Dongjoon, do you guys maybe have any idea?

2019년 4월 25일 (목) 오후 3:09, Hyukjin Kwon <[hidden email]>님이 작성:
Thank you so much Josh .. !!

2019년 4월 25일 (목) 오후 3:04, Josh Rosen <[hidden email]>님이 작성:
The code for this runs in http://spark-prs.appspot.com (see https://github.com/databricks/spark-pr-dashboard/blob/1e799c9e510fa8cdc9a6c084a777436bebeabe10/sparkprs/controllers/tasks.py#L137)

I checked the AppEngine logs and it looks like we're getting error responses, possibly due to a credentials issue:

Exception when starting progress on JIRA issue SPARK-27355 (/base/data/home/apps/s~spark-prs/live.412416057856832734/sparkprs/controllers/tasks.py:142) Traceback (most recent call last): File
Traceback (most recent call last):
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/sparkprs/controllers/tasks.py", line 138, in update_pr start_issue_progress("%s-%s" % (app.config['JIRA_PROJECT'], issue_number)) File
start_issue_progress("%s-%s" % (app.config['JIRA_PROJECT'], issue_number))
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/sparkprs/jira_api.py", line 27, in start_issue_progress jira_client = get_jira_client() File
jira_client = get_jira_client()
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/sparkprs/jira_api.py", line 18, in get_jira_client app.config['JIRA_PASSWORD'])) File
app.config['JIRA_PASSWORD']))
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/client.py", line 472, in __init__ si = self.server_info() File
si = self.server_info()
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/client.py", line 2133, in server_info j = self._get_json('serverInfo') File
j = self._get_json('serverInfo')
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/client.py", line 2549, in _get_json r = self._session.get(url, params=params) File
r = self._session.get(url, params=params)
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/resilientsession.py", line 151, in get return self.__verb('GET', url, **kwargs) File
return self.__verb('GET', url, **kwargs)
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/resilientsession.py", line 147, in __verb raise_on_error(response, verb=verb, **kwargs) File
raise_on_error(response, verb=verb, **kwargs)
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/resilientsession.py", line 57, in raise_on_error r.status_code, error, r.url, request=request, response=r, **kwargs) JIRAError: JiraError HTTP 403 url: https://issues.apache.org/jira/rest/api/2/serverInfo text: CAPTCHA_CHALLENGE; login-url=https://issues.apache.org/jira/login.jsp r.status_code, error, r.url, request=request, response=r, **kwargs)
JIRAError: JiraError HTTP 403 url: https://issues.apache.org/jira/rest/api/2/serverInfo
text: CAPTCHA_CHALLENGE; login-url=https://issues.apache.org/jira/login.jsp

It looks like ASF JIRA was throwing a captcha challenge at us, so I used the credentials to manually log in and complete the challenge.

Hopefully that's enough to fix things, but to prevent re-occurrence we might need to change the login credential type from username + password to instead use OAuth tokens.

On Wed, Apr 24, 2019 at 10:42 PM Hyukjin Kwon <[hidden email]> wrote:
Can anyone take a look for this one? OPEN status JIRAs are being rapidly increased (from around 2400 to 2600)

2019년 4월 19일 (금) 오후 8:05, Hyukjin Kwon <[hidden email]>님이 작성:
Hi all,

Looks 'spark/dev/github_jira_sync.py' is not running correctly somewhere.
Usually the JIRA's status should be updated to "IN PROGRESS" when
somebody opens a PR against a JIRA.
Looks now it only leaves a link and does not change JIRA's status.

Can someone else who knows where it's running can check this?

FWIW, I check every PR and JIRA almost every day but ever since this happened, this makes (at least to me) duplicately check the JIRAs.
Previously, if I check all the PRs and JIRAs, they were not duplicated because JIRAs having PRs have different status, "IN PROGRESS" but now all JIRAs have "OPEN" status.

Thanks.
Reply | Threaded
Open this post in threaded view
|

Re: In Apache Spark JIRA, spark/dev/github_jira_sync.py not running properly

Hyukjin Kwon
That's a great explanation. Thanks I didn't know that.

Josh, do you know who I should ping on this?

On Fri, 19 Jul 2019, 16:52 Dongjoon Hyun, <[hidden email]> wrote:
Hi, Hyukjin.

In short, there are two bots. And, the current situation happens when only one bot with `dev/github_jira_sync.py` works.

And, `dev/github_jira_sync.py` is irrelevant to the JIRA status change because it only use `add_remote_link` and `add_comment` API.
I know only this bot (in Apache Spark repository repo)

AFAIK, `deb/github_jira_sync.py`'s activity is done under JIRA ID `githubbot` (Name: `ASF GitHub Bot`).
And, the other bot's activity is done under JIRA ID `apachespark` (Name: `Apache Spark`).
The other bot is the one which Josh mentioned before. (in `databricks/spark-pr-dashboard` repo).

The root cause will be the same. The API key used by the bot is rejected by Apache JIRA and forwarded to CAPCHAR.

Bests,
Dongjoon.

On Thu, Jul 18, 2019 at 8:24 PM Hyukjin Kwon <[hidden email]> wrote:
Hi all,

Seems this issue is re-happening again. Seems the PR link is properly created in the corresponding JIRA but it doesn't change the JIRA's status from OPEN to IN-PROGRESS.

See, for instance,

https://issues.apache.org/jira/browse/SPARK-28443
https://issues.apache.org/jira/browse/SPARK-28440
https://issues.apache.org/jira/browse/SPARK-28436
https://issues.apache.org/jira/browse/SPARK-28434
https://issues.apache.org/jira/browse/SPARK-28433
https://issues.apache.org/jira/browse/SPARK-28431

Josh and Dongjoon, do you guys maybe have any idea?

2019년 4월 25일 (목) 오후 3:09, Hyukjin Kwon <[hidden email]>님이 작성:
Thank you so much Josh .. !!

2019년 4월 25일 (목) 오후 3:04, Josh Rosen <[hidden email]>님이 작성:
The code for this runs in http://spark-prs.appspot.com (see https://github.com/databricks/spark-pr-dashboard/blob/1e799c9e510fa8cdc9a6c084a777436bebeabe10/sparkprs/controllers/tasks.py#L137)

I checked the AppEngine logs and it looks like we're getting error responses, possibly due to a credentials issue:

Exception when starting progress on JIRA issue SPARK-27355 (/base/data/home/apps/s~spark-prs/live.412416057856832734/sparkprs/controllers/tasks.py:142) Traceback (most recent call last): File
Traceback (most recent call last):
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/sparkprs/controllers/tasks.py", line 138, in update_pr start_issue_progress("%s-%s" % (app.config['JIRA_PROJECT'], issue_number)) File
start_issue_progress("%s-%s" % (app.config['JIRA_PROJECT'], issue_number))
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/sparkprs/jira_api.py", line 27, in start_issue_progress jira_client = get_jira_client() File
jira_client = get_jira_client()
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/sparkprs/jira_api.py", line 18, in get_jira_client app.config['JIRA_PASSWORD'])) File
app.config['JIRA_PASSWORD']))
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/client.py", line 472, in __init__ si = self.server_info() File
si = self.server_info()
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/client.py", line 2133, in server_info j = self._get_json('serverInfo') File
j = self._get_json('serverInfo')
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/client.py", line 2549, in _get_json r = self._session.get(url, params=params) File
r = self._session.get(url, params=params)
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/resilientsession.py", line 151, in get return self.__verb('GET', url, **kwargs) File
return self.__verb('GET', url, **kwargs)
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/resilientsession.py", line 147, in __verb raise_on_error(response, verb=verb, **kwargs) File
raise_on_error(response, verb=verb, **kwargs)
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/resilientsession.py", line 57, in raise_on_error r.status_code, error, r.url, request=request, response=r, **kwargs) JIRAError: JiraError HTTP 403 url: https://issues.apache.org/jira/rest/api/2/serverInfo text: CAPTCHA_CHALLENGE; login-url=https://issues.apache.org/jira/login.jsp r.status_code, error, r.url, request=request, response=r, **kwargs)
JIRAError: JiraError HTTP 403 url: https://issues.apache.org/jira/rest/api/2/serverInfo
text: CAPTCHA_CHALLENGE; login-url=https://issues.apache.org/jira/login.jsp

It looks like ASF JIRA was throwing a captcha challenge at us, so I used the credentials to manually log in and complete the challenge.

Hopefully that's enough to fix things, but to prevent re-occurrence we might need to change the login credential type from username + password to instead use OAuth tokens.

On Wed, Apr 24, 2019 at 10:42 PM Hyukjin Kwon <[hidden email]> wrote:
Can anyone take a look for this one? OPEN status JIRAs are being rapidly increased (from around 2400 to 2600)

2019년 4월 19일 (금) 오후 8:05, Hyukjin Kwon <[hidden email]>님이 작성:
Hi all,

Looks 'spark/dev/github_jira_sync.py' is not running correctly somewhere.
Usually the JIRA's status should be updated to "IN PROGRESS" when
somebody opens a PR against a JIRA.
Looks now it only leaves a link and does not change JIRA's status.

Can someone else who knows where it's running can check this?

FWIW, I check every PR and JIRA almost every day but ever since this happened, this makes (at least to me) duplicately check the JIRAs.
Previously, if I check all the PRs and JIRAs, they were not duplicated because JIRAs having PRs have different status, "IN PROGRESS" but now all JIRAs have "OPEN" status.

Thanks.
Reply | Threaded
Open this post in threaded view
|

Re: In Apache Spark JIRA, spark/dev/github_jira_sync.py not running properly

Hyukjin Kwon
Just FYI, I had to come up with a better JQL to filter out the JIRAs that already have linked PRs.
In case it helps someone, I use this JQL now to look through the open JIRAs:
project = SPARK AND
status = Open AND
NOT issueFunction in linkedIssuesOfRemote("Github Pull Request *")
ORDER BY created DESC, priority DESC, updated DESC



2019년 7월 19일 (금) 오후 4:54, Hyukjin Kwon <[hidden email]>님이 작성:
That's a great explanation. Thanks I didn't know that.

Josh, do you know who I should ping on this?

On Fri, 19 Jul 2019, 16:52 Dongjoon Hyun, <[hidden email]> wrote:
Hi, Hyukjin.

In short, there are two bots. And, the current situation happens when only one bot with `dev/github_jira_sync.py` works.

And, `dev/github_jira_sync.py` is irrelevant to the JIRA status change because it only use `add_remote_link` and `add_comment` API.
I know only this bot (in Apache Spark repository repo)

AFAIK, `deb/github_jira_sync.py`'s activity is done under JIRA ID `githubbot` (Name: `ASF GitHub Bot`).
And, the other bot's activity is done under JIRA ID `apachespark` (Name: `Apache Spark`).
The other bot is the one which Josh mentioned before. (in `databricks/spark-pr-dashboard` repo).

The root cause will be the same. The API key used by the bot is rejected by Apache JIRA and forwarded to CAPCHAR.

Bests,
Dongjoon.

On Thu, Jul 18, 2019 at 8:24 PM Hyukjin Kwon <[hidden email]> wrote:
Hi all,

Seems this issue is re-happening again. Seems the PR link is properly created in the corresponding JIRA but it doesn't change the JIRA's status from OPEN to IN-PROGRESS.

See, for instance,

https://issues.apache.org/jira/browse/SPARK-28443
https://issues.apache.org/jira/browse/SPARK-28440
https://issues.apache.org/jira/browse/SPARK-28436
https://issues.apache.org/jira/browse/SPARK-28434
https://issues.apache.org/jira/browse/SPARK-28433
https://issues.apache.org/jira/browse/SPARK-28431

Josh and Dongjoon, do you guys maybe have any idea?

2019년 4월 25일 (목) 오후 3:09, Hyukjin Kwon <[hidden email]>님이 작성:
Thank you so much Josh .. !!

2019년 4월 25일 (목) 오후 3:04, Josh Rosen <[hidden email]>님이 작성:
The code for this runs in http://spark-prs.appspot.com (see https://github.com/databricks/spark-pr-dashboard/blob/1e799c9e510fa8cdc9a6c084a777436bebeabe10/sparkprs/controllers/tasks.py#L137)

I checked the AppEngine logs and it looks like we're getting error responses, possibly due to a credentials issue:

Exception when starting progress on JIRA issue SPARK-27355 (/base/data/home/apps/s~spark-prs/live.412416057856832734/sparkprs/controllers/tasks.py:142) Traceback (most recent call last): File
Traceback (most recent call last):
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/sparkprs/controllers/tasks.py", line 138, in update_pr start_issue_progress("%s-%s" % (app.config['JIRA_PROJECT'], issue_number)) File
start_issue_progress("%s-%s" % (app.config['JIRA_PROJECT'], issue_number))
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/sparkprs/jira_api.py", line 27, in start_issue_progress jira_client = get_jira_client() File
jira_client = get_jira_client()
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/sparkprs/jira_api.py", line 18, in get_jira_client app.config['JIRA_PASSWORD'])) File
app.config['JIRA_PASSWORD']))
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/client.py", line 472, in __init__ si = self.server_info() File
si = self.server_info()
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/client.py", line 2133, in server_info j = self._get_json('serverInfo') File
j = self._get_json('serverInfo')
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/client.py", line 2549, in _get_json r = self._session.get(url, params=params) File
r = self._session.get(url, params=params)
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/resilientsession.py", line 151, in get return self.__verb('GET', url, **kwargs) File
return self.__verb('GET', url, **kwargs)
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/resilientsession.py", line 147, in __verb raise_on_error(response, verb=verb, **kwargs) File
raise_on_error(response, verb=verb, **kwargs)
File "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/resilientsession.py", line 57, in raise_on_error r.status_code, error, r.url, request=request, response=r, **kwargs) JIRAError: JiraError HTTP 403 url: https://issues.apache.org/jira/rest/api/2/serverInfo text: CAPTCHA_CHALLENGE; login-url=https://issues.apache.org/jira/login.jsp r.status_code, error, r.url, request=request, response=r, **kwargs)
JIRAError: JiraError HTTP 403 url: https://issues.apache.org/jira/rest/api/2/serverInfo
text: CAPTCHA_CHALLENGE; login-url=https://issues.apache.org/jira/login.jsp

It looks like ASF JIRA was throwing a captcha challenge at us, so I used the credentials to manually log in and complete the challenge.

Hopefully that's enough to fix things, but to prevent re-occurrence we might need to change the login credential type from username + password to instead use OAuth tokens.

On Wed, Apr 24, 2019 at 10:42 PM Hyukjin Kwon <[hidden email]> wrote:
Can anyone take a look for this one? OPEN status JIRAs are being rapidly increased (from around 2400 to 2600)

2019년 4월 19일 (금) 오후 8:05, Hyukjin Kwon <[hidden email]>님이 작성:
Hi all,

Looks 'spark/dev/github_jira_sync.py' is not running correctly somewhere.
Usually the JIRA's status should be updated to "IN PROGRESS" when
somebody opens a PR against a JIRA.
Looks now it only leaves a link and does not change JIRA's status.

Can someone else who knows where it's running can check this?

FWIW, I check every PR and JIRA almost every day but ever since this happened, this makes (at least to me) duplicately check the JIRAs.
Previously, if I check all the PRs and JIRAs, they were not duplicated because JIRAs having PRs have different status, "IN PROGRESS" but now all JIRAs have "OPEN" status.

Thanks.