spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hyukjin Kwon <gurwls...@gmail.com>
Subject Re: Some PRs not automatically linked to JIRAs
Date Thu, 03 Aug 2017 01:06:55 GMT
I think it works for anyone who can leave a web link and comment.


For "in progress" Resolution, it looks I am unable to manually set. Please
let me know if anyone knows.


For single JIRA, I manually modified the script to process single item
before. I guess you know what the script does but want to rather test,
right?

This looks requiring Python JIRA client, which I installed by pip install
jira==1.0.7 and I personally tested (and just double checked this) after
commenting out JIRA related codes:


diff --git a/dev/github_jira_sync.py b/dev/github_jira_sync.py
index acc9aeabbb9..c8adf0c4ade 100755
--- a/dev/github_jira_sync.py
+++ b/dev/github_jira_sync.py
@@ -101,8 +101,8 @@ def get_max_pr():
         return 0


-jira_client = jira.client.JIRA({'server': JIRA_API_BASE},
-                               basic_auth=(JIRA_USERNAME, JIRA_PASSWORD))
+# jira_client = jira.client.JIRA({'server': JIRA_API_BASE},
+#                                basic_auth=(JIRA_USERNAME, JIRA_PASSWORD))

 jira_prs = get_jira_prs()

@@ -124,14 +124,15 @@ for issue, pr in sorted(jira_prs, key=lambda kv:
int(kv[1]['number'])):
     url = pr['html_url']
     title = "[Github] Pull Request #%s (%s)" % (pr['number'],
pr['user']['login'])
     try:
-        existing_links = map(lambda l: l.raw['object']['url'],
jira_client.remote_links(issue))
+        # existing_links = map(lambda l: l.raw['object']['url'],
jira_client.remote_links(issue))
+        pass
     except:
         print("Failure reading JIRA %s (does it exist?)" % issue)
         print(sys.exc_info()[0])
         continue

-    if url in existing_links:
-        continue
+    # if url in existing_links:
+    #     continue

     icon = {"title": "Pull request #%s" % pr['number'],
             "url16x16": "https://assets-cdn.github.com/favicon.ico"}
@@ -139,12 +140,12 @@ for issue, pr in sorted(jira_prs, key=lambda kv:
int(kv[1]['number'])):
     # For all possible fields see:
     #
https://developer.atlassian.com/display/JIRADEV/Fields+in+Remote+Issue+Links
     # application = {"name": "Github pull requests", "type":
"org.apache.spark.jira.github"}
-    jira_client.add_remote_link(issue, destination)
+    # jira_client.add_remote_link(issue, destination)

     comment = "User '%s' has created a pull request for this issue:" %
pr['user']['login']
     comment += "\n%s" % pr['html_url']
-    if pr_num >= MIN_COMMENT_PR:
-        jira_client.add_comment(issue, comment)
+    # if pr_num >= MIN_COMMENT_PR:
+        # jira_client.add_comment(issue, comment)

     print("Added link %s <-> PR #%s" % (issue, pr['number']))
     num_updates += 1


and


rm .github-jira-max
./dev/github_jira_sync.py



2017-08-03 5:20 GMT+09:00 Josh Rosen <joshrosen@databricks.com>:

> Usually the backend of https://spark-prs.appspot.com does the linking
> while processing PR update tasks. It appears that the site's connections to
> JIRA have started failing:
>
> ConnectionError: ('Connection aborted.', HTTPException('Deadline exceeded
> while waiting for HTTP response from URL: https://issues.apache.org/
> jira/rest/api/2/serverInfo',))
>
> From Stackdriver's log-based metrics, I can spot that this problem started
> around July 24th. We're already using a much-higher-than-default URL fetch
> timeout, so it's possible that the problem is related to access
> credentials, IP blocks, outdated client libraries, or something else.
>
> On Wed, Aug 2, 2017 at 1:10 PM Bryan Cutler <cutlerb@gmail.com> wrote:
>
>> Thanks Hyukjin!  I didn't see your previous message..  It looks like your
>> manual run worked pretty well for the JIRAs I'm following, the only thing
>> is that it didn't mark them as "in progress", but that's not a big deal.
>> Otherwise that helps until we can find out why it's not doing this
>> automatically.  I'm not familiar with that script, can anyone run it to
>> apply to a single JIRA they are working on?
>>
>> On Wed, Aug 2, 2017 at 12:09 PM, Hyukjin Kwon <gurwls223@gmail.com>
>> wrote:
>>
>>> I was wondering about this too..
>>>
>>>
>>> Yes, actually, I have been manually adding some links by resembling the
>>> same steps in the script before.
>>>
>>> I was thinking it'd rather be nicer to run this manually once and then I
>>> ran this against single JIRA
>>>
>>> first - https://issues.apache.org/jira/browse/SPARK-21526 to show how
>>> it looks like and check if there
>>>
>>> is any issue or objection just in case.
>>>
>>>
>>> Will run this manually now once. I will revert all my action manually if
>>> there is any issue by doing this.
>>>
>>>
>>> 2017-08-03 3:50 GMT+09:00 Sean Owen <sowen@cloudera.com>:
>>>
>>>> Hyukjin mentioned this here earlier today and had run it manually, but
>>>> yeah I'm not sure where it normally runs or why it hasn't. Shane not sure
>>>> if you're the person to ask?
>>>>
>>>>
>>>> On Wed, Aug 2, 2017 at 7:47 PM Bryan Cutler <cutlerb@gmail.com> wrote:
>>>>
>>>>> Hi Devs,
>>>>>
>>>>> I've noticed a couple PRs recently have not been automatically linked
>>>>> to the related JIRAs.  This was one of mine (I linked it manually)
>>>>> https://issues.apache.org/jira/browse/SPARK-21583, but I've seen it
>>>>> happen elsewhere.  I think this is the script that does it, but it hasn't
>>>>> been changed recently https://github.com/apache/spark/blob/master/dev/
>>>>> github_jira_sync.py.  Anyone else seen this or know what's going on?
>>>>>
>>>>> Thanks,
>>>>> Bryan
>>>>>
>>>>
>>>
>>

Mime
View raw message