Merge lp:~leannogasawara/arsenal/kernel into lp:arsenal/trunk

[Leann Ogasawara]

[Leann Ogasawara (leannogasawara)]

Hi Bryce,

I've been hoarding patches for the kernel arsenal scripts which seems bad. It's better if I get these merged so others on the kernel team can review and use.

These patches only touch code under contrib/linux with the exception of one patch (rev 306):

revno: 306
committer: Leann Ogasawara <email address hidden>
branch nick: arsenal
timestamp: Mon 2009-11-09 18:24:08 -0800
message:
  contrib/arsenal_lib.py: Use display_name to set owner_firstname

[lp:~leannogasawara/arsenal/kernel]

308. By Leann Ogasawara on 2009-11-11

contrib/linux/retarget-new-bugs.py: check for kj-linux-meta tag
earlier. also transition bugs for linux-meta-fsl-imx51 and
linux-meta-mvl-dove.

[Bryce Harrington (bryce)]

Hi Leann,

Sorry, I didn't see the merge proposal until just now!

Thanks, it all looks good, and I've merged it into base.

review: Approve

=== modified file 'contrib/arsenal_lib.py'
--- contrib/arsenal_lib.py	2009-10-02 02:35:44 +0000
+++ contrib/arsenal_lib.py	2009-11-11 23:01:09 +0000
@@ -140,7 +140,7 @@
         self.attachments = bug.attachments
         self.description = bug.description.encode('utf-8')
         self.owner = bug.owner
-        self.owner_firstname = self.owner.name.split(' ')[0]
+        self.owner_firstname = self.owner.display_name.split(' ')[0]
         self.launchpad = launchpad
         return
 
 
=== modified file 'contrib/linux/process-confirmed-bugs.py'
--- contrib/linux/process-confirmed-bugs.py	2009-09-30 22:15:59 +0000
+++ contrib/linux/process-confirmed-bugs.py	2009-11-11 23:01:09 +0000
@@ -37,12 +37,16 @@
                 break
 
             bug = ArsenalBug(bugtask.bug, lp.launchpad)
+            if opt_dryrun:
+                print "== Dry Run =="
+
             print "https://bugs.edge.launchpad.net/ubuntu/+bug/%d" % bug.id, "  ", bugtask.status, "  ", bug.title
 
             if (bug.has_tag("review-request") or
                 bug.has_tag("kj-omit") or
                 bug.has_tag("ct-rev") or
                 bug.has_tag("workflow") or
+                bug.has_tag("xen") or
                 bug.has_tag("regression-proposed") or
                 bug.has_tag("regression-release") or
                 bug.has_tag("regression-update") or
@@ -50,9 +54,6 @@
                 print "Skipping, tagged: ", bug.bug.tags
                 continue
 
-            if opt_dryrun:
-                print "== Dry Run =="
-
             # Check for old Confirmed bugs
             days = bug.age_last_message()
             if days > days_expired:
@@ -76,7 +77,10 @@
                     print "  --> skipping, bug has an canonical-kernel-team assignee"
                     assigned = True
                 else:
-                    for person in canonical_kernel_team.getMembersByStatus(status="Approved"):
+                    ckt = list(canonical_kernel_team.getMembersByStatus(status="Approved"))
+                    ckt.extend(list(canonical_kernel_team.getMembersByStatus(status="Administrator")))
+
+                    for person in ckt:
                         if bugtask.assignee.name == person.name:
                             print "  --> skipping, assigned to", bugtask.assignee.name
                             assigned = True
 
=== modified file 'contrib/linux/process-incomplete-bugs.py'
--- contrib/linux/process-incomplete-bugs.py	2009-09-30 22:15:59 +0000
+++ contrib/linux/process-incomplete-bugs.py	2009-11-11 23:01:09 +0000
@@ -20,18 +20,13 @@
 d           = lp.load_project("ubuntu")
 canonical_kernel_team = lp.launchpad.people['canonical-kernel-team']
 bug_limit = 10 # set to 0 or False for no buglimit
+kernel_janitor = lp.launchpad.people['kernel-janitor']
 total_count = 0
 message_close_incomplete = """This bug report was marked as Incomplete and has not had any updated comments for quite some time.  As a result this bug is being closed.  Please reopen if this is still an issue in the current Ubuntu release http://www.ubuntu.com/getubuntu/download . Also, please be sure to provide any requested information that may have been missing.  To reopen the bug, click on the current status under the Status column and change the status back to "New".  Thanks.
 
 [This is an automated message.  Apologies if it has reached you inappropriately; please just reply to this message indicating so.]
 """
 
-message_not_owner_logs = """Thanks for running apport-collect to attach the requested debug files.  We are still waiting for the original bug reporter to attach their debug files so that we can confirm you are indeed seeing the same issue.  If the original bug reporter does not attach their debug files within 7 days we will have you open a separate bug report.  We'll keep you posted.  Thanks for your patience.
-"""
-
-message_open_new_bug = """The original bug reporter has not attached their log files.  Please open a new bug report and we'll track your issue there.  Refer to https://wiki.ubuntu.com/KernelTeam/KernelTeamBugPolicies for insturctions on how to properly file a bug against the Ubuntu kernel.  Thanks.
-"""
-
 try:
     for source_pkg in source_pkgs:
         count = 0
@@ -39,17 +34,22 @@
 
         src_pkg = d.getSourcePackage(name=source_pkg)
         for bugtask in src_pkg.searchTasks(status="Incomplete (with response)", tags=["needs-kernel-logs", "needs-reporter-kernel-logs"]):
+            print count, bug_limit
             if bug_limit and count >= bug_limit:
                     print "Hit max number of bugs to process, exiting."
                     break
 
             bug = ArsenalBug(bugtask.bug, lp.launchpad)
+            if opt_dryrun:
+                print "== Dry Run=="
+
             print "https://bugs.launchpad.net/ubuntu/+bug/%d" % bug.id, "  ", bugtask.status, "  ", bug.title
 
             if (bug.has_tag("review-request") or
                 bug.has_tag("kj-omit") or
                 bug.has_tag("ct-rev") or
                 bug.has_tag("workflow") or
+                bug.has_tag("xen") or
                 bug.has_tag("regression-proposed") or
                 bug.has_tag("regression-release") or
                 bug.has_tag("regression-update") or
@@ -65,7 +65,10 @@
                     print "  --> skipping, bug has an canonical-kernel-team assignee"
                     assigned = True
                 else:
-                    for person in canonical_kernel_team.getMembersByStatus(status="Approved"):
+                    ckt = list(canonical_kernel_team.getMembersByStatus(status="Approved"))
+                    ckt.extend(list(canonical_kernel_team.getMembersByStatus(status="Administrator")))
+
+                    for person in ckt:
                         if bugtask.assignee.name == person.name:
                             print "  --> skipping, assigned to", bugtask.assignee.name
                             assigned = True
@@ -73,8 +76,6 @@
                 if assigned:
                     continue
 
-            if opt_dryrun:
-                print "== Dry Run=="
             # If has tag needs-*, see if those are attached now
             # If so, remove tag.  If no more needs-* tags, set to Confirmed
             needs_kernel_logs = True
@@ -83,52 +84,54 @@
 
             # Check if apport-collect was ran
             days = 0
-            for msg in reversed(messages):
-                subject = str(msg.subject)
-                if subject.find("apport-collect data") > -1:
-                    # Handle the case were apport-collect was run by someone
-                    # other than the original bug reporter
-                    if (msg.owner_link != bug.bug.owner_link) and not bug.has_tag("needs-reporter-kernel-logs"):
-                        greeting = "Hi " + msg.owner.name.split(' ')[0] + ",\n\n"
-                        comment = greeting + message_not_owner_logs
-                        if opt_dryrun:
-                            print " --> tag remove needs-kernel-logs"
-                            print " --> tag needs-reporter-kernel-logs"
-                            print " --> Msg:", comment
+            try:
+		last_msg = ""
+                for msg in reversed(messages):
+		    if last_msg == "":
+			last_msg = msg.content
+                    subject = str(msg.subject)
+                    if subject.find("apport-collect data") > -1:
+                        # Handle the case were apport-collect was run by someone
+                        # other than the original bug reporter
+                        if (msg.owner_link != bug.bug.owner_link) and not bug.has_tag("needs-reporter-kernel-logs"):
+			    print " --> Non-reporter attached logs"
+			    print last_msg.encode('utf-8')
+                        elif bug.has_tag("needs-reporter-kernel-logs"):
+                            continue
+                        # Set bug status to Confirmed and remove needs-kernel-logs tag
                         else:
-                            bug.remove_tag("needs-kernel-logs")
-                            bug.append_tag("needs-reporter-kernel-logs")
-                            bug.append_comment(comment)
-                    elif bug.has_tag("needs-reporter-kernel-logs"):
-                        days = bug.age_last_message()
-                        continue
-                    # Set bug status to Confirmed and remove needs-kernel-logs tag
-                    else:
-                        if bug.has_tag("needs-kernel-logs"):
-                            print " --> remove tag needs-kernel-logs"
-                        if bug.has_tag("needs-reporter-kernel-logs"):
-                            print " --> remove tag needs-reporter-kernel-logs"
-                        print " --> Confirmed"
-                        if not opt_dryrun:
-                            try:
-                                bug.remove_tag("needs-kernel-logs")
-                                bug.remove_tag("needs-reporter-kernel-logs")
-                                bugtask.transitionToStatus(status = "Confirmed")
-                            except:
-                                print " ERROR:  Failed to update bug ", bug.id
-                                print "New status was:  Confirmed"
-                                raise
-                        needs_kernel_logs = False
-                    count += 1
-                    total_count += 1
-                    break
+                            if bug.has_tag("needs-kernel-logs"):
+                                print " --> remove tag needs-kernel-logs"
+                            if bug.has_tag("needs-reporter-kernel-logs"):
+                                print " --> remove tag needs-reporter-kernel-logs"
+                            if not bug.has_tag("kj-triage"):
+                                print " --> add tag kj-triage"
+                            print " --> Confirmed"
+                            if bugtask.importance == "Undecided":
+                                print " --> set Medium Importance"
+                            if not opt_dryrun:
+                                try:
+                                    bug.remove_tag("needs-kernel-logs")
+                                    bug.remove_tag("needs-reporter-kernel-logs")
+                                    bugtask.transitionToStatus(status = "Confirmed")
+                                    if bugtask.importance == "Undecided":
+                                        bugtask.transitionToImportance(importance = "Medium")
+                                except:
+                                    print " ERROR:  Failed to update bug ", bug.id
+                                    print "New status was:  Confirmed, Medium"
+                                    raise
+                            needs_kernel_logs = False
+                        count += 1
+                        total_count += 1
+                        break
+                    elif msg.owner.name == kernel_janitor.name:
+                        print " --> no logs since kj commented, skipping"
+			print last_msg.encode('utf-8')
+                        break
+            except HTTPError, e:
+                print e.content
+                continue
 
-            if days > 7:
-                print " --> non-reporter apport-collect over 7 days ago"
-                if opt_dryrun:
-                    print " --> Msg:", message_open_new_bug
-                else:
-                    bug.append_comment(message_open_new_bug)
                 count += 1
                 total_count += 1
             # Requires triager to review
@@ -153,11 +156,12 @@
                 print "Hit max number of bugs to process, exiting."
                 break
             bug = ArsenalBug(bugtask.bug, lp.launchpad)
-            print "https://bugs.launchpad.net/ubuntu/+bug/%d" % bug.id, "  ", bugtask.status, "  ", bug.title
 
             if opt_dryrun:
                 print "== Dry Run=="
 
+            print "https://bugs.launchpad.net/ubuntu/+bug/%d" % bug.id, "  ", bugtask.status, "  ", bug.title
+
             if len(bug.bug.bug_tasks) > 2:
                 print " --> Skipping; bug_tasks ==", len(bug.bug.bug_tasks)
 #                if opt_dryrun:
 
=== modified file 'contrib/linux/process-new-bugs.py'
--- contrib/linux/process-new-bugs.py	2009-09-30 22:15:59 +0000
+++ contrib/linux/process-new-bugs.py	2009-11-11 23:01:09 +0000
@@ -87,7 +87,7 @@
     },
 ]
 
-message_all = """Please be sure to confirm this issue exists with the latest development release of Ubuntu.  ISO CD images are available from http://cdimage.ubuntu.com/releases/ .  Please then run following command from a Terminal (Applications->Accessories->Terminal).  It will automatically gather and attach updated debug information to this report.
+message_all = """Please be sure to confirm this issue exists with the latest development release of Ubuntu.  ISO CD images are available from http://cdimage.ubuntu.com/releases/karmic .  If the issue remains, please run the following command from a Terminal (Applications->Accessories->Terminal).  It will automatically gather and attach updated debug information to this report.
 
 apport-collect -p %s %d
 
@@ -101,7 +101,7 @@
 Thanks in advance.
 """
 
-message_old_bug = """This bug was reported a while ago and there hasn't been any activity in it recently. We were wondering if this is still an issue? Can you try with the latest development release of Ubuntu?  ISO CD images are available from http://cdimage.ubuntu.com/releases/ .
+message_old_bug = """This bug was reported a while ago and there hasn't been any activity in it recently. We were wondering if this is still an issue? Can you try with the latest development release of Ubuntu?  ISO CD images are available from http://cdimage.ubuntu.com/releases/karmic .
 
 If it remains an issue, could you run the following command from a Terminal (Applications->Accessories->Terminal).  It will automatically gather and attach updated debug information to this report.
 
@@ -112,7 +112,7 @@
 Thanks in advance.
 """
 
-message_sound_bug = """Please be sure to confirm this issue exists with the latest development release of Ubuntu.  ISO CD images are available from http://cdimage.ubuntu.com/releases/ . Please then run the following command from a Terminal (Applications->Accessories->Terminal).  It will automatically gather and attach updated debug information to this report.
+message_sound_bug = """Please be sure to confirm this issue exists with the latest development release of Ubuntu.  ISO CD images are available from http://cdimage.ubuntu.com/releases/karmic . If the issue remains, please run the following command from a Terminal (Applications->Accessories->Terminal).  It will automatically gather and attach updated debug information to this report.
 
 apport-collect -p alsa-base %d
 
@@ -121,7 +121,7 @@
 Thanks in advance.
 """
 
-message_suspend_hibernate_bug ="""Please be sure to confirm this issue exists with the latest development release of Ubuntu.  ISO CD images are available from http://cdimage.ubuntu.com/releases/ .  However, note you can only test Suspend when using a LiveCD.  If you could run the following command from a Terminal (Applications->Accessories->Terminal) it will automatically gather and attach updated debug information to this report.
+message_suspend_hibernate_bug ="""Please be sure to confirm this issue exists with the latest development release of Ubuntu.  ISO CD images are available from http://cdimage.ubuntu.com/releases/karmic .  However, note you can only test Suspend, not Hibernate, when using a LiveCD.  If the issue remains, run the following command from a Terminal (Applications->Accessories->Terminal) it will automatically gather and attach updated debug information to this report.
 
 apport-collect -p %s %d
 
@@ -220,24 +220,35 @@
         count=0
         print "Processing ", source_pkg
         s = d.getSourcePackage(name = source_pkg)
+
         # Eventually update the searchTasks for tag negated search to avoid
         # checking for "kj-omit", "workflow", "review-request", and "kj-triage" below
-        for bugtask in s.searchTasks(status="New"):
+        new_tasks = s.searchTasks(status="New")
+        for bugtask in new_tasks:
             if bug_limit and count >= bug_limit:
                 print "Hit max number of bugs to process, exiting."
                 break
 
             bug = ArsenalBug(bugtask.bug, lp.launchpad)
+            if opt_dryrun:
+                print "== Dry Run =="
+
             print "https://bugs.launchpad.net/ubuntu/+bug/%d" % bug.id, "  ", bugtask.status, "  ", str(bug.title)
             reply = ArsenalReply(bug)
             logs_complete = False
 
             # TODO:  These bugs cause launchpad to issue a 500 error for some reason
             if bug.id == 337926 or bug.id == 377090:
-                continue
-
-            # TODO:  If a bug is nomnated, skip it
+                print "skipping ", bug.id
+                continue
+
+            if "failed to install/upgrade" in bug.title:
+                print " --> skipping"
+                continue
+
+            # TODO:  If a bug is nominated, skip it
             if 0 == 1:
+                print "nominated ", bug.id
                 continue
 
             # Skip bugs we don't want to handle automatically
@@ -246,6 +257,7 @@
                 bug.has_tag("ct-rev") or
                 bug.has_tag("review-request") or
                 bug.has_tag("kj-triage") or
+                bug.has_tag("xen") or
                 bug.has_tag("regression-proposed") or
                 bug.has_tag("regression-release") or
                 bug.has_tag("regression-update") or
@@ -261,7 +273,10 @@
                     print "  --> skipping, bug has an canonical-kernel-team assignee"
                     assigned = True
                 else:
-                    for person in canonical_kernel_team.getMembersByStatus(status="Approved"):
+                    ckt = list(canonical_kernel_team.getMembersByStatus(status="Approved"))
+                    ckt.extend(list(canonical_kernel_team.getMembersByStatus(status="Administrator")))
+
+                    for person in ckt:
                         if bugtask.assignee.name == person.name:
                             print "  --> skipping, assigned to", bugtask.assignee.name
                             assigned = True
@@ -307,9 +322,6 @@
                     total_count += 1
                     continue
 
-            if opt_dryrun:
-                print "== Dry Run =="
-
             if bug.has_tag("apport-package") or bug.has_tag("apport-kerneloops"):
                 logs_complete = True
 
@@ -377,7 +389,7 @@
             if bug.age_last_message() > days_old and not bug.has_tag("apport-package"):
                 print "  --> Old bug over ", bug.age_last_message(), " days"
                 # After Karmic, s/linux-image-`uname -r`/source_pkg/
-                reply.set_message(message_old_bug % ("linux-image-`uname -r`", bug.id))
+                reply.set_message(message_old_bug % (source_pkg, bug.id))
                 if opt_dryrun:
                     print "  --> tag needs-kernel-logs"
                     print "  --> tag needs-upstream-testing"
@@ -396,12 +408,12 @@
             elif len(bug.attachments) == 0:
                 print "  --> [No attachments]"
                 # After Karmic, s/linux-image-`uname -r`/source_pkg/
-                if regex_subj['sound'].search(bug.title):
+                if regex_subj['sound'].search(bug.title) or bug.has_tag("kernel-sound"):
                     reply.set_message(message_sound_bug % (bug.id))
                 elif regex_subj['suspend'].search(bug.title) or regex_subj['hibernate'].search(bug.title):
-                    reply.set_message(message_suspend_hibernate_bug % ("linux-image-`uname -r`", bug.id))
+                    reply.set_message(message_suspend_hibernate_bug % (source_pkg, bug.id))
                 else:
-                    reply.set_message(message_all % ("linux-image-`uname -r`", bug.id))
+                    reply.set_message(message_all % (source_pkg, bug.id))
 
                 if opt_dryrun:
                     print "  --> tag needs-kernel-logs"
@@ -445,9 +457,9 @@
                         logs_complete = True
                     else:
                         if regex_subj['suspend'].search(bug.title) or regex_subj['hibernate'].search(bug.title):
-                            reply.set_message(message_suspend_hibernate_bug % ("linux-image-`uname -r`", bug.id))
+                            reply.set_message(message_suspend_hibernate_bug % (source_pkg, bug.id))
                         else:
-                            reply.set_message(message_all % ("linux-image-`uname -r`", bug.id))
+                            reply.set_message(message_all % (source_pkg, bug.id))
                 else:
                     if (has_file['Amixer'] and has_file['AplayDevice'] and has_file['Codec0']) or has_file['unknown']:
                         logs_complete = True
 
=== modified file 'contrib/linux/process-triaged-bugs.py'
--- contrib/linux/process-triaged-bugs.py	2009-09-30 22:15:59 +0000
+++ contrib/linux/process-triaged-bugs.py	2009-11-11 23:01:09 +0000
@@ -16,7 +16,7 @@
 source_pkgs = sys.argv[1:]
 lp          = LaunchpadService()
 d           = lp.load_project("ubuntu")
-
+bug_limit = 5
 total_count = 0
 message_old_triaged = """This bug report was marked as Triaged a while ago but has not had any updated comments for quite some time.  Please let us know if this issue remains in the current Ubuntu release, http://www.ubuntu.com/getubuntu/download .  If the issue remains, click on the current status under the Status column and change the status back to "New".  Thanks.
 
@@ -27,9 +27,19 @@
         count = 0
         print "Processing ", source_pkg
 
-        src_pkg = d.getSourcePackage(name=source_pkg)
+	try:
+	    src_pkg = d.getSourcePackage(name=source_pkg)
+	except HTTPError, e:
+	    print e.content
+	    print "continuing on . . ."
         for bugtask in src_pkg.searchTasks(status="Triaged"):
             bug = ArsenalBug(bugtask.bug, lp.launchpad)
+            if bug_limit and count >= bug_limit:
+                    print "Hit max number of bugs to process, exiting."
+                    break
+
+            if opt_dryrun:
+                print "== Dry Run =="
 
             print "https://bugs.edge.launchpad.net/ubuntu/+bug/%d" % bug.id, "  ", bugtask.status, "  ", bug.title
 
@@ -37,6 +47,7 @@
                 bug.has_tag("kj-omit") or
                 bug.has_tag("ct-rev") or
                 bug.has_tag("workflow") or
+                bug.has_tag("xen") or
                 bug.has_tag("regression-proposed") or
                 bug.has_tag("regression-release") or
                 bug.has_tag("regression-update") or
@@ -53,16 +64,13 @@
             # Expire bugs which have gone more than 120 days without an
             # updated comment
             if days > days_expired:
-                if opt_dryrun:
-                    print "== Dry Run =="
-                    if not bug.has_tag("kj-triage"):
-                       print " --> tagged kj-triage"
-                    print " --> Incomplete"
-                    print " --> Msg was: ", message_old_triaged
-                else:
+                if not bug.has_tag("kj-triage"):
+                   print " --> tagged kj-triage"
+                print " --> Incomplete"
+                print " --> Msg was: ", message_old_triaged
+                if not opt_dryrun:
                     bug.append_comment(message_old_triaged)
-                    if not bug.has_tag("kj-triage"):
-                        bug.append_tag("kj-triage")
+                    bug.append_tag("kj-triage")
                     bugtask.transitionToStatus(status = "Incomplete")
 
                 count += 1
 
=== modified file 'contrib/linux/retarget-new-bugs.py'
--- contrib/linux/retarget-new-bugs.py	2009-09-30 22:15:59 +0000
+++ contrib/linux/retarget-new-bugs.py	2009-11-11 23:01:09 +0000
@@ -18,14 +18,18 @@
     def transition(self, from_task, to_task, comment):
         old_task = distro.getSourcePackage(name=from_task)
         new_task = distro.getSourcePackage(name=to_task)
-        collection = old_task.searchTasks(status="New")
+        collection = old_task.searchTasks()
         count = 1
         for bug_task in collection:
-            bug = ArsenalBug(bug_task.bug, arsenal.launchpad)
+            bug = ArsenalBug(bug_task.bug, lp.launchpad)
             print count, " - ", bug.id, "  ", bug_task.status, "  ", bug.title
             count = count + 1
             skip = False
 
+            if bug.has_tag("kj-linux-meta"):
+                print "Tagged kj-linux-meta, skipping."
+                continue
+
             # check if bug already has to_task open
             if len(bug_task.bug.bug_tasks) > 1:
                 for task in bug_task.bug.bug_tasks:
@@ -49,10 +53,12 @@
 
 This bug was reported against the linux-meta package when it likely should have been reported against the linux package instead.  We are automatically transitioning this to the linux kernel package so that the appropriate teams are notified and made aware of this issue.
 
-If this bug really is a bug in the linux-meta package you can move it back to linux-meta and set the Status to Confirmed, or contact us on the #ubuntu-kernel channel on the FreeNode IRC server.  Thanks.
+If this bug really is a bug in the linux-meta package you can move it back to linux-meta and tag it kj-linux-meta, or contact us on the #ubuntu-kernel channel on the FreeNode IRC server.  Thanks.
 '''
 
     bug_task = BugTask()
     bug_task.transition("linux-meta", "linux", comment)
+    bug_task.transition("linux-meta-fsl-imx51", "linux-fsl-imx51", comment)
+    bug_task.transition("linux-meta-mvl-dove", "linux-mvl-dove", comment)
 
 # vi:set ts=4 sw=4 expandtab: