Some PRs not automatically linked to JIRAs

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Some PRs not automatically linked to JIRAs

Bryan Cutler
Hi Devs,

I've noticed a couple PRs recently have not been automatically linked to the related JIRAs.  This was one of mine (I linked it manually) https://issues.apache.org/jira/browse/SPARK-21583, but I've seen it happen elsewhere.  I think this is the script that does it, but it hasn't been changed recently https://github.com/apache/spark/blob/master/dev/github_jira_sync.py.  Anyone else seen this or know what's going on?

Thanks,
Bryan
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Some PRs not automatically linked to JIRAs

Sean Owen
Hyukjin mentioned this here earlier today and had run it manually, but yeah I'm not sure where it normally runs or why it hasn't. Shane not sure if you're the person to ask?

On Wed, Aug 2, 2017 at 7:47 PM Bryan Cutler <[hidden email]> wrote:
Hi Devs,

I've noticed a couple PRs recently have not been automatically linked to the related JIRAs.  This was one of mine (I linked it manually) https://issues.apache.org/jira/browse/SPARK-21583, but I've seen it happen elsewhere.  I think this is the script that does it, but it hasn't been changed recently https://github.com/apache/spark/blob/master/dev/github_jira_sync.py.  Anyone else seen this or know what's going on?

Thanks,
Bryan
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Some PRs not automatically linked to JIRAs

Hyukjin Kwon
I was wondering about this too..


Yes, actually, I have been manually adding some links by resembling the same steps in the script before.

I was thinking it'd rather be nicer to run this manually once and then I ran this against single JIRA

first - https://issues.apache.org/jira/browse/SPARK-21526 to show how it looks like and check if there

is any issue or objection just in case.


Will run this manually now once. I will revert all my action manually if there is any issue by doing this.


2017-08-03 3:50 GMT+09:00 Sean Owen <[hidden email]>:
Hyukjin mentioned this here earlier today and had run it manually, but yeah I'm not sure where it normally runs or why it hasn't. Shane not sure if you're the person to ask?


On Wed, Aug 2, 2017 at 7:47 PM Bryan Cutler <[hidden email]> wrote:
Hi Devs,

I've noticed a couple PRs recently have not been automatically linked to the related JIRAs.  This was one of mine (I linked it manually) https://issues.apache.org/jira/browse/SPARK-21583, but I've seen it happen elsewhere.  I think this is the script that does it, but it hasn't been changed recently https://github.com/apache/spark/blob/master/dev/github_jira_sync.py.  Anyone else seen this or know what's going on?

Thanks,
Bryan

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Some PRs not automatically linked to JIRAs

Bryan Cutler
Thanks Hyukjin!  I didn't see your previous message..  It looks like your manual run worked pretty well for the JIRAs I'm following, the only thing is that it didn't mark them as "in progress", but that's not a big deal.  Otherwise that helps until we can find out why it's not doing this automatically.  I'm not familiar with that script, can anyone run it to apply to a single JIRA they are working on?

On Wed, Aug 2, 2017 at 12:09 PM, Hyukjin Kwon <[hidden email]> wrote:
I was wondering about this too..


Yes, actually, I have been manually adding some links by resembling the same steps in the script before.

I was thinking it'd rather be nicer to run this manually once and then I ran this against single JIRA

first - https://issues.apache.org/jira/browse/SPARK-21526 to show how it looks like and check if there

is any issue or objection just in case.


Will run this manually now once. I will revert all my action manually if there is any issue by doing this.


2017-08-03 3:50 GMT+09:00 Sean Owen <[hidden email]>:
Hyukjin mentioned this here earlier today and had run it manually, but yeah I'm not sure where it normally runs or why it hasn't. Shane not sure if you're the person to ask?


On Wed, Aug 2, 2017 at 7:47 PM Bryan Cutler <[hidden email]> wrote:
Hi Devs,

I've noticed a couple PRs recently have not been automatically linked to the related JIRAs.  This was one of mine (I linked it manually) https://issues.apache.org/jira/browse/SPARK-21583, but I've seen it happen elsewhere.  I think this is the script that does it, but it hasn't been changed recently https://github.com/apache/spark/blob/master/dev/github_jira_sync.py.  Anyone else seen this or know what's going on?

Thanks,
Bryan


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Some PRs not automatically linked to JIRAs

Josh Rosen-2
Usually the backend of https://spark-prs.appspot.com does the linking while processing PR update tasks. It appears that the site's connections to JIRA have started failing:

ConnectionError: ('Connection aborted.', HTTPException('Deadline exceeded while waiting for HTTP response from URL: https://issues.apache.org/jira/rest/api/2/serverInfo',))

From Stackdriver's log-based metrics, I can spot that this problem started around July 24th. We're already using a much-higher-than-default URL fetch timeout, so it's possible that the problem is related to access credentials, IP blocks, outdated client libraries, or something else.

On Wed, Aug 2, 2017 at 1:10 PM Bryan Cutler <[hidden email]> wrote:
Thanks Hyukjin!  I didn't see your previous message..  It looks like your manual run worked pretty well for the JIRAs I'm following, the only thing is that it didn't mark them as "in progress", but that's not a big deal.  Otherwise that helps until we can find out why it's not doing this automatically.  I'm not familiar with that script, can anyone run it to apply to a single JIRA they are working on?

On Wed, Aug 2, 2017 at 12:09 PM, Hyukjin Kwon <[hidden email]> wrote:
I was wondering about this too..


Yes, actually, I have been manually adding some links by resembling the same steps in the script before.

I was thinking it'd rather be nicer to run this manually once and then I ran this against single JIRA

first - https://issues.apache.org/jira/browse/SPARK-21526 to show how it looks like and check if there

is any issue or objection just in case.


Will run this manually now once. I will revert all my action manually if there is any issue by doing this.


2017-08-03 3:50 GMT+09:00 Sean Owen <[hidden email]>:
Hyukjin mentioned this here earlier today and had run it manually, but yeah I'm not sure where it normally runs or why it hasn't. Shane not sure if you're the person to ask?


On Wed, Aug 2, 2017 at 7:47 PM Bryan Cutler <[hidden email]> wrote:
Hi Devs,

I've noticed a couple PRs recently have not been automatically linked to the related JIRAs.  This was one of mine (I linked it manually) https://issues.apache.org/jira/browse/SPARK-21583, but I've seen it happen elsewhere.  I think this is the script that does it, but it hasn't been changed recently https://github.com/apache/spark/blob/master/dev/github_jira_sync.py.  Anyone else seen this or know what's going on?

Thanks,
Bryan


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Some PRs not automatically linked to JIRAs

Hyukjin Kwon
I think it works for anyone who can leave a web link and comment.


For "in progress" Resolution, it looks I am unable to manually set. Please let me know if anyone knows.


For single JIRA, I manually modified the script to process single item before. I guess you know what the script does but want to rather test, right?

This looks requiring Python JIRA client, which I installed by pip install jira==1.0.7 and I personally tested (and just double checked this) after commenting out JIRA related codes:


diff --git a/dev/github_jira_sync.py b/dev/github_jira_sync.py
index acc9aeabbb9..c8adf0c4ade 100755
--- a/dev/github_jira_sync.py
+++ b/dev/github_jira_sync.py
@@ -101,8 +101,8 @@ def get_max_pr():
         return 0


-jira_client = jira.client.JIRA({'server': JIRA_API_BASE},
-                               basic_auth=(JIRA_USERNAME, JIRA_PASSWORD))
+# jira_client = jira.client.JIRA({'server': JIRA_API_BASE},
+#                                basic_auth=(JIRA_USERNAME, JIRA_PASSWORD))

 jira_prs = get_jira_prs()

@@ -124,14 +124,15 @@ for issue, pr in sorted(jira_prs, key=lambda kv: int(kv[1]['number'])):
     url = pr['html_url']
     title = "[Github] Pull Request #%s (%s)" % (pr['number'], pr['user']['login'])
     try:
-        existing_links = map(lambda l: l.raw['object']['url'], jira_client.remote_links(issue))
+        # existing_links = map(lambda l: l.raw['object']['url'], jira_client.remote_links(issue))
+        pass
     except:
         print("Failure reading JIRA %s (does it exist?)" % issue)
         print(sys.exc_info()[0])
         continue

-    if url in existing_links:
-        continue
+    # if url in existing_links:
+    #     continue

     icon = {"title": "Pull request #%s" % pr['number'],
             "url16x16": "https://assets-cdn.github.com/favicon.ico"}
@@ -139,12 +140,12 @@ for issue, pr in sorted(jira_prs, key=lambda kv: int(kv[1]['number'])):
     # For all possible fields see:
     # application = {"name": "Github pull requests", "type": "org.apache.spark.jira.github"}
-    jira_client.add_remote_link(issue, destination)
+    # jira_client.add_remote_link(issue, destination)

     comment = "User '%s' has created a pull request for this issue:" % pr['user']['login']
     comment += "\n%s" % pr['html_url']
-    if pr_num >= MIN_COMMENT_PR:
-        jira_client.add_comment(issue, comment)
+    # if pr_num >= MIN_COMMENT_PR:
+        # jira_client.add_comment(issue, comment)

     print("Added link %s <-> PR #%s" % (issue, pr['number']))
     num_updates += 1


and


rm .github-jira-max
./dev/github_jira_sync.py



2017-08-03 5:20 GMT+09:00 Josh Rosen <[hidden email]>:
Usually the backend of https://spark-prs.appspot.com does the linking while processing PR update tasks. It appears that the site's connections to JIRA have started failing:

ConnectionError: ('Connection aborted.', HTTPException('Deadline exceeded while waiting for HTTP response from URL: https://issues.apache.org/jira/rest/api/2/serverInfo',))

From Stackdriver's log-based metrics, I can spot that this problem started around July 24th. We're already using a much-higher-than-default URL fetch timeout, so it's possible that the problem is related to access credentials, IP blocks, outdated client libraries, or something else.

On Wed, Aug 2, 2017 at 1:10 PM Bryan Cutler <[hidden email]> wrote:
Thanks Hyukjin!  I didn't see your previous message..  It looks like your manual run worked pretty well for the JIRAs I'm following, the only thing is that it didn't mark them as "in progress", but that's not a big deal.  Otherwise that helps until we can find out why it's not doing this automatically.  I'm not familiar with that script, can anyone run it to apply to a single JIRA they are working on?

On Wed, Aug 2, 2017 at 12:09 PM, Hyukjin Kwon <[hidden email]> wrote:
I was wondering about this too..


Yes, actually, I have been manually adding some links by resembling the same steps in the script before.

I was thinking it'd rather be nicer to run this manually once and then I ran this against single JIRA

first - https://issues.apache.org/jira/browse/SPARK-21526 to show how it looks like and check if there

is any issue or objection just in case.


Will run this manually now once. I will revert all my action manually if there is any issue by doing this.


2017-08-03 3:50 GMT+09:00 Sean Owen <[hidden email]>:
Hyukjin mentioned this here earlier today and had run it manually, but yeah I'm not sure where it normally runs or why it hasn't. Shane not sure if you're the person to ask?


On Wed, Aug 2, 2017 at 7:47 PM Bryan Cutler <[hidden email]> wrote:
Hi Devs,

I've noticed a couple PRs recently have not been automatically linked to the related JIRAs.  This was one of mine (I linked it manually) https://issues.apache.org/jira/browse/SPARK-21583, but I've seen it happen elsewhere.  I think this is the script that does it, but it hasn't been changed recently https://github.com/apache/spark/blob/master/dev/github_jira_sync.py.  Anyone else seen this or know what's going on?

Thanks,
Bryan



Loading...