Merge lp:~sinzui/launchpad/bug-subselect-timeout into lp:launchpad

[Curtis Hovey]

Only match the context when searching for structures a person is subscribed to.

The bug query clause for structural subscribers times out. This is
possibly caused by the superfluous joins against bugtaskflat.

--------------------------------------------------------------------

RULES

    Pre-implementation: wgrant
    * Structure the query to be fast. Try In() and Row() to match a CTE table
      of just the relevant structural subscriptions.
    * Do not match impossible conditions...when looking for distro or
      distro packages, do not match on product or product series.

QA

    * Visit https://bugs.qastaging.launchpad.net/ubuntu/+bugs?field.structural_subscriber=foundations-bugs
    * Verify it does not timeout

LINT

    lib/lp/bugs/model/bugtasksearch.py

TEST

    ./bin/test -vvc -t structural lp.bugs.model.tests.test_bugtasksearch

IMPLEMENTATION

Replaced the LEFT JOINS with subselects using In() and Row() as needed. I
tried a single Row() that matched the SS columns to BugTaskFlat columns,
but the tests failed -- I suspect that NULLs in Row() did not match the NULLs
in the SELECT. NILL != NULL. Maybe I botched this. Note the odd rule that
matches DSP SSs to DS SP bugs. This looks historical, but may also be
convenience because SPs can be short-lived and probably causes users to
not ask why they stopped getting email when a new series started. After I
got the tests to pass, I changed the code to build only the subqueries that
make sense for the bug params being used.
    lib/lp/bugs/model/bugtasksearch.py

[j.c.sackett (jcsackett)]

This looks alright to me, but I would wait for William's input as the peculiarities of bug search performance is definitely not something I'm up on.

review: Approve

[William Grant (wgrant)]

There's a general issue here that your preconditions for applying the restrictions are a little too strict. I'd suggest perhaps maintaining a map of {target type: [potentially relevant subscription targets]}, or just examining all subscription targets unconditionally if that performs reasonably.

I tried doing this in a single ROW ... IN clause as I suggested on the call, but didn't get very far before it tried to strangle me, mostly due to the fact that (distro, spn) tasks must be found by any (distro, NULL) subscription. It'd be possible with a bit of UNIONing, but I'm not sure it's worth it at this stage.

Specific comments inline below.

105 + Select(
106 + SS.distributionID,
107 + tables=[SS],
108 + where=(SS.sourcepackagenameID == None))))

Given how long the structsub code is, it might be worth dropping the newline in the middle there, and similarly compressing the other blocks. Even cutting just a few lines makes the whole thing seem a lot more manageable.

109 + ss_clauses.append(In(
110 + Row(BugTaskFlat.distribution_id,
111 + BugTaskFlat.sourcepackagename_id),
112 + Select(
113 + ((SS.distributionID,
114 + SS.sourcepackagenameID),),
115 + tables=[SS])))

I don't think you need the double bracketing in the Select. Removing that also lets you get it onto one line. Same in a couple of other places.

125 + if params.distroseries is not None:
126 + distroseries_id = params.distroseries.id
127 + parent_distro_id = params.distroseries.distributionID
128 + else:
129 + distroseries_id = 0
130 + parent_distro_id = 0

I've never understood this bit. It looks like it will just ignore DSP structsubs for SP tasks unless queried in a distroseries context?

141 + if is_person_search or params.product is not None:
142 + ss_clauses.append(In(
143 + BugTaskFlat.product_id,
144 + Select(
145 + SS.productID,
146 + tables=[SS])))

What about project groups? They show multiple products, but this will ignore all of the product subscriptions. Also, the entire Select() expression can be one line.

153 + if is_person_search or params.project is not None:

This will break in one obscure case: if searching in a product, it won't return any bugs if the only structural subscription is through the project group.

163 + if is_person_search or params.milestone is not None:

This needs to apply in basically all cases. Milestone subscriptions affect all targets.

183 + # Remove bugtasks from deactivated products.
184 + # This is needed for searches where people are the context.
185 + if is_person_search:

This also breaks in the case of project group searches.

review: Needs Fixing (code)

[Curtis Hovey (sinzui)]

Thank you for the review William.

I simplified the rules for appending ss clauses. Milestones are always present as you suggested. The two if-statements
check if the query is constrained to product or distro related bugs. This allowed me to remove the is_person_search variable that you pointed out was also applying to project groups. I reduced the number of vertical lines. I also revised the comment about why we search for DSP's when we know the user has constrained the search to a distroseries.

[William Grant (wgrant)]

That turned out much nicer than I thought it could, thanks! One last nitpick: "if params.product is None and params.productseries is None" can probably be further restricted with "params.project is None", although it's correct either way.

review: Approve (code)

=== modified file 'lib/lp/bugs/doc/bugtask-search.txt'
--- lib/lp/bugs/doc/bugtask-search.txt	2012-08-16 05:18:54 +0000
+++ lib/lp/bugs/doc/bugtask-search.txt	2012-09-28 22:06:42 +0000
@@ -18,114 +18,6 @@
     True
 
 
-== Searching by bug supervisor ==
-
-The 'bug_supervisor' parameter allows you to search bugtasks that a certain
-person is responsible for. A person can be a bug supervisor for a product,
-a distribution, or a distribution source package. No Privileges Person
-isn't a bug supervisor, so no bugs are found for him:
-
-    >>> from lp.registry.interfaces.person import IPersonSet
-    >>> no_priv = getUtility(IPersonSet).getByName('no-priv')
-    >>> no_priv_bug_supervisor = BugTaskSearchParams(
-    ...     user=None, bug_supervisor=no_priv)
-    >>> found_bugtasks = bugtask_set.search(no_priv_bug_supervisor)
-    >>> found_bugtasks.count()
-    0
-
-== Product bugs ==
-
-Firefox has a few bugs:
-
-    >>> from lp.registry.interfaces.product import IProductSet
-    >>> firefox = getUtility(IProductSet).getByName('firefox')
-    >>> firefox_bugs = firefox.searchTasks(all_public)
-    >>> firefox_public_bugs = firefox_bugs.count()
-    >>> firefox_public_bugs > 0
-    True
-
-== Distribution and package bugs ==
-
-Ubuntu does too:
-
-    >>> from lp.registry.interfaces.distribution import IDistributionSet
-    >>> ubuntu = getUtility(IDistributionSet).getByName("ubuntu")
-    >>> all_public = BugTaskSearchParams(user=None)
-    >>> ubuntu_bugs = ubuntu.searchTasks(all_public)
-    >>> ubuntu_bugs.count() > 0
-    True
-
-and in particular, mozilla-firefox in Ubuntu has 'em:
-
-    >>> ubuntu_firefox = ubuntu.getSourcePackage("mozilla-firefox")
-    >>> ubuntu_firefox_bugs = ubuntu_firefox.searchTasks(all_public)
-    >>> ubuntu_firefox_bugs.count() > 0
-    True
-
-== Person bugs ==
-
-To get all related tasks to a person call searchTasks() on the person
-object:
-
-    >>> from lp.registry.interfaces.person import IPersonSet
-    >>> user = getUtility(IPersonSet).getByName('name16')
-    >>> user_bugs = user.searchTasks(None, user=None)
-    >>> user_bugs.count() > 0
-    True
-
-== Dupes and Conjoined tasks ==
-
-You can set flags to omit duplicates:
-
-    >>> no_dupes = BugTaskSearchParams(user=None, omit_dupes=True)
-    >>> firefox = getUtility(IProductSet).getByName('firefox')
-    >>> sans_dupes = firefox.searchTasks(no_dupes)
-    >>> sans_dupes.count() < firefox_public_bugs
-    True
-
-and also series-targeted bugs:
-
-    >>> no_targeted = BugTaskSearchParams(user=None, omit_targeted=True)
-    >>> sans_targeted = ubuntu.searchTasks(no_targeted)
-    >>> sans_targeted.count() < ubuntu_bugs
-    True
-
-=== Product bug supervisor ===
-
-If No Privileges is specified as Firefox's bug supervisor, searching for his
-bugs return all of Firefox's bugs.
-
-    >>> login('[e-mail]')
-    >>> firefox.bug_supervisor = no_priv
-
-    >>> found_bugtasks = bugtask_set.search(no_priv_bug_supervisor)
-    >>> found_bugtasks.count() == firefox_bugs.count()
-    True
-
-    >>> found_targets = set(
-    ...     bugtask.target.bugtargetdisplayname for bugtask in found_bugtasks)
-    >>> for target_name in sorted(found_targets):
-    ...     print target_name
-    Mozilla Firefox
-
-
-=== Distribution bug supervisor ===
-
-If someone is bug supervisor for Firefox, Firefox in Ubuntu, and Ubuntu,
-all bugs in Firefox and Ubuntu are returned. Bugs in the Ubuntu Firefox
-package are included in the Ubuntu bugs, so they won't be returned
-twice.
-
-    >>> all_public = BugTaskSearchParams(user=None)
-    >>> ubuntu_bugs = ubuntu.searchTasks(all_public)
-    >>> ubuntu_bugs.count() > 0
-    True
-
-    >>> ubuntu.bug_supervisor = no_priv
-    >>> found_bugtasks = bugtask_set.search(no_priv_bug_supervisor)
-    >>> found_bugtasks.count() == firefox_bugs.count() + ubuntu_bugs.count()
-    True
-
 == Searching using bug full-text index ==
 
 The searchtext parameter does an extensive and expensive search (it
@@ -136,6 +28,8 @@
 
 For example, there are no bugs with the word 'Fnord' in Firefox.
 
+    >>> from lp.registry.interfaces.product import IProductSet
+    >>> firefox = getUtility(IProductSet).getByName('firefox')
     >>> text_search = BugTaskSearchParams(user=None, searchtext=u'Fnord')
     >>> found_bugtasks = firefox.searchTasks(text_search)
     >>> found_bugtasks.count()
@@ -144,6 +38,7 @@
 But if we put that word in the bug #4 description, it will be found.
 
     >>> from lp.bugs.interfaces.bug import IBugSet
+    >>> login('[e-mail]')
     >>> bug_four = getUtility(IBugSet).get(4)
     >>> bug_four.description += (
     ...     '\nThat happens pretty often with the Fnord Highlighter '
@@ -212,322 +107,6 @@
     >>> transaction.abort()
 
 
-== Searching by bug reporter ==
-
-The 'bug_reporter' parameter allows you to search for bugs reported by a
-certain person.
-
-    >>> foo_bar = getUtility(IPersonSet).getByEmail('[e-mail]')
-    >>> reported_by_foo_bar = BugTaskSearchParams(
-    ...     user=None, bug_reporter=foo_bar)
-    >>> reported_by_foo_bar.setDistribution(ubuntu)
-    >>> found_bugtasks = bugtask_set.search(reported_by_foo_bar)
-    >>> for bugtask in found_bugtasks:
-    ...     print "#%s in %s reported by %s" % (
-    ...         bugtask.bug.id, bugtask.bugtargetname,
-    ...         bugtask.bug.owner.displayname)
-    #9 in thunderbird (Ubuntu) reported by Foo Bar
-    #10 in linux-source-2.6.15 (Ubuntu) reported by Foo Bar
-
-
-== Searching for nominated bugs ==
-
-We can search for bugs nominated to a distribution series by using the
-nominated_for parameter.
-
-    >>> ubuntu = getUtility(IDistributionSet).getByName('ubuntu')
-    >>> warty = ubuntu.getSeries('warty')
-
-    >>> from lp.bugs.model.bugnomination import BugNomination
-    >>> print list(BugNomination.selectBy(distroseries=warty))
-    []
-
-    >>> from lp.bugs.interfaces.bug import CreateBugParams
-    >>> nominated_for_warty = BugTaskSearchParams(
-    ...     user=None, nominated_for=warty)
-    >>> list(ubuntu.searchTasks(nominated_for_warty))
-    []
-
-    >>> nominated_bug = ubuntu.createBug(
-    ...     CreateBugParams(owner=no_priv, title='Test nominated bug',
-    ...                     comment='Something'))
-    >>> BugNomination(
-    ...     owner=no_priv, distroseries=warty, bug=nominated_bug)
-    <BugNomination at ...>
-
-    >>> for bugtask in ubuntu.searchTasks(nominated_for_warty):
-    ...     print bugtask.bug.title
-    Test nominated bug
-
-The same parameter is used to search for bugs nominated to a product
-series.
-
-    >>> firefox = getUtility(IProductSet).getByName('firefox')
-    >>> firefox_trunk = firefox.getSeries('trunk')
-    >>> print list(BugNomination.selectBy(productseries=firefox_trunk))
-    []
-    >>> nominated_for_trunk = BugTaskSearchParams(
-    ...     user=None, nominated_for=firefox_trunk)
-    >>> list(firefox.searchTasks(nominated_for_trunk))
-    []
-
-    >>> nominated_bug = firefox.createBug(
-    ...     CreateBugParams(owner=no_priv, title='Bug to be fixed in trunk',
-    ...                     comment='Something'))
-    >>> BugNomination(
-    ...     owner=no_priv, productseries=firefox_trunk, bug=nominated_bug)
-    <BugNomination at ...>
-
-    >>> for bugtask in firefox.searchTasks(nominated_for_trunk):
-    ...     print bugtask.bug.title
-    Bug to be fixed in trunk
-
-== Filter by Upstream Status ==
-
-Add an Ubuntu bugtask for a bug that is confirmed upstream.
-
-    >>> from lp.bugs.interfaces.bugtask import (
-    ...     BugTaskImportance,
-    ...     BugTaskStatus,
-    ...     )
-    >>> from lp.bugs.model.tests.test_bugtask import (
-    ...     BugTaskSearchBugsElsewhereTest)
-    >>> def bugTaskInfo(bugtask):
-    ...     return '%i %i %s %s' % (
-    ...         bugtask.id, bugtask.bug.id, bugtask.bugtargetdisplayname,
-    ...         bugtask.bug.title)
-    >>> test_helper = BugTaskSearchBugsElsewhereTest(helper_only=True)
-    >>> bug_twelve =  getUtility(IBugSet).get(12)
-    >>> task_open_upstream = bugtask_set.createTask(
-    ...     bug_twelve, foo_bar, ubuntu,
-    ...     status=BugTaskStatus.NEW, importance=BugTaskImportance.MEDIUM)
-    >>> test_helper.assertBugTaskIsOpenUpstream(task_open_upstream)
-
-Pass the resolved_upstream flag to include only bugtasks linked to
-watches that are rejected, fixed committed or fix released, or bugtasks
-related to upstream bugtasks (i.e. filed on the same bug) that are fix
-committed or fix released.
-
-    >>> test_helper.setUpBugsResolvedUpstreamTests()
-    >>> params = BugTaskSearchParams(
-    ...     resolved_upstream=True, orderby='id', user=None)
-    >>> closed_elsewhere_tasks = ubuntu.searchTasks(params)
-    >>> for bugtask in closed_elsewhere_tasks:
-    ...     test_helper.assertBugTaskIsResolvedUpstream(bugtask)
-    ...     print bugTaskInfo(bugtask)
-    17 1 mozilla-firefox (Ubuntu) Firefox does not support SVG
-    26 2 Ubuntu Blackhole Trash folder
-    23 9 thunderbird (Ubuntu) Thunderbird crashes
-
-
-Pass the open_upstream flag to include only bugtasks linked to those
-watches or those upstream bugtasks that have the status "unconfirmed",
-"needs info", "confirmed", "in progress" or "unknown". Note that a bug
-may be associated with three or more bugtasks. If one upstream task
-has a state associated with "open upstream", and another upstream task
-has a state associated with "resolved upstream", the bug is included
-in the results of the "open upstream" filter as well as the "resolved
-upstream" filter.
-
-(In the examples below, the last bugtask is ellipsized because its ID
- is generated here and therefore sampledata-dependent.)
-
-    >>> params = BugTaskSearchParams(
-    ...     open_upstream=True, orderby='id', user=None)
-    >>> open_elsewhere_tasks = ubuntu.searchTasks(params)
-    >>> for bugtask in open_elsewhere_tasks:
-    ...     test_helper.assertBugTaskIsOpenUpstream(bugtask)
-    ...     print bugTaskInfo(bugtask)
-    17  1 mozilla-firefox (Ubuntu) Firefox does not support SVG
-    26  2 Ubuntu Blackhole Trash folder
-    ... ... Ubuntu Copy, Cut and Delete operations should work on selections
-
-
-We can also filter our search to include only bugs that are not known to
-affect upstream, i.e., bugs that don't have an IUpstreamBugTask.
-
-    >>> params = BugTaskSearchParams(
-    ...     has_no_upstream_bugtask=True, orderby='id', user=None)
-    >>> tasks_with_no_upstreams = ubuntu.searchTasks(params)
-    >>> for bugtask in tasks_with_no_upstreams:
-    ...     test_helper.assertShouldBeShownOnNoUpstreamTaskSearch(bugtask)
-    ...     print bugTaskInfo(bugtask)
-    25 10 linux-source-2.6.15 (Ubuntu) another test bug
-    ... ... Ubuntu Test nominated bug
-
-If we combine upstream-related filters, we get the union of the results
-of the single filters.
-
-    >>> params = BugTaskSearchParams(
-    ...     has_no_upstream_bugtask=True, resolved_upstream=True,
-    ...     orderby='id', user=None)
-    >>> tasks_with_no_upstreams = ubuntu.searchTasks(params)
-    >>> for bugtask in tasks_with_no_upstreams:
-    ...     print bugTaskInfo(bugtask)
-    17  1 mozilla-firefox (Ubuntu) Firefox does not support SVG
-    26  2 Ubuntu Blackhole Trash folder
-    23  9 thunderbird (Ubuntu) Thunderbird crashes
-    25 10 linux-source-2.6.15 (Ubuntu) another test bug
-    ... ... Ubuntu Test nominated bug
-
-
-    >>> test_helper.tearDownBugsElsewhereTests()
-
-The search filter can also return bugs that are related to CVE reports:
-
-    >>> from lp.bugs.interfaces.cve import ICveSet
-    >>> def getCves(bugtask):
-    ...     bug, cve = getUtility(ICveSet).getBugCvesForBugTasks([bugtask])[0]
-    ...     return cve.sequence
-    >>> params = BugTaskSearchParams(
-    ...     has_cve=True, orderby='id', user=None)
-    >>> tasks_with_cves = ubuntu.searchTasks(params)
-    >>> for bugtask in tasks_with_cves:
-    ...     print bugTaskInfo(bugtask), getCves(bugtask)
-    17 1 mozilla-firefox (Ubuntu) Firefox does not support SVG 1999-8979
-    26 2 Ubuntu Blackhole Trash folder 1999-2345
-
-
-== Searching by bug commenter ==
-
-The 'bug_commenter' parameter allows you to search bugtasks on which a certain
-person has commented. No Privileges Person hasn't commented on any bugs, so no
-bugs are found for him:
-
-    >>> from lp.registry.interfaces.person import IPersonSet
-    >>> transaction.abort()
-
-    >>> no_priv = getUtility(IPersonSet).getByName('no-priv')
-    >>> no_priv_bug_commenter = BugTaskSearchParams(
-    ...     user=None, bug_commenter=no_priv)
-    >>> found_bugtasks = bugtask_set.search(no_priv_bug_commenter)
-    >>> found_bugtasks.count()
-    0
-
-If No Privileges Person comments on some bugs, those bugs can then be found by
-a bug commenter search. There will be one bug task instance returned for each
-bug target that the task is registered against, so in the test below three
-comments will produce eight found bug tasks (three for bug 1, five for bug 2).
-
-    >>> bug_one = getUtility(IBugSet).get(1)
-    >>> bug_one.newMessage(no_priv, 'No subject', 'some comment')
-    <Message at ...>
-
-    >>> bug_two = getUtility(IBugSet).get(2)
-    >>> bug_two.newMessage(no_priv, 'No subject', 'another comment')
-    <Message at ...>
-
-    >>> bug_two.newMessage(no_priv, 'No subject', 'yet another comment')
-    <Message at ...>
-
-    >>> for (bug_id, target) in sorted((bugtask.bug.id, bugtask.bugtargetname)
-    ...                                for bugtask in found_bugtasks):
-    ...     print bug_id, target
-    1 firefox
-    1 mozilla-firefox (Debian)
-    1 mozilla-firefox (Ubuntu)
-    2 Ubuntu Hoary
-    2 mozilla-firefox (Debian Woody)
-    2 mozilla-firefox (Debian)
-    2 tomcat
-    2 ubuntu
-
-If No Privileges Person reports a bug and does not comment on it, that bug
-will not be included in the results returned by the bug commenter search.
-
-    >>> from lp.bugs.interfaces.bug import CreateBugParams
-    >>> from lp.registry.interfaces.product import IProductSet
-
-    >>> firefox = getUtility(IProductSet).getByName('firefox')
-    >>> firefox.createBug(
-    ...     CreateBugParams(no_priv, "Some bug", "Some comment"))
-    <Bug at ...>
-
-    >>> for (bug_id, target) in sorted((bugtask.bug.id, bugtask.bugtargetname)
-    ...                                for bugtask in found_bugtasks):
-    ...     print bug_id, target
-    1 firefox
-    1 mozilla-firefox (Debian)
-    1 mozilla-firefox (Ubuntu)
-    2 Ubuntu Hoary
-    2 mozilla-firefox (Debian Woody)
-    2 mozilla-firefox (Debian)
-    2 tomcat
-    2 ubuntu
-
-
-== Search for BugTasks assigned to milestones ==
-
-BugTaskSet.search() can return bugtasks associated with milestones.
-No BugTask is associated yet with firefox milestone 1.0.
-
-    >>> product_milestone = firefox.getMilestone('1.0')
-    >>> params = BugTaskSearchParams(milestone=product_milestone, user=None)
-    >>> milestone_tasks = bugtask_set.search(params)
-    >>> print milestone_tasks.count()
-    0
-
-Similary, no BugTasks are associated with the project firexfox belongs to.
-
-    >>> mozilla = firefox.project
-    >>> project_milestone = mozilla.getMilestone('1.0')
-    >>> params = BugTaskSearchParams(milestone=project_milestone, user=None)
-    >>> milestone_tasks = bugtask_set.search(params)
-    >>> print milestone_tasks.count()
-    0
-
-When a BugTask is associated with a milestone, it is returned in a search
-for bugs of this milestone.
-
-    >>> bugtask = firefox.searchTasks(BugTaskSearchParams(user=None))[0]
-    >>> print bugTaskInfo(bugtask)
-    2 1 Mozilla Firefox Firefox does not support SVG
-    >>> bugtask.milestone = product_milestone
-    >>> params = BugTaskSearchParams(milestone=product_milestone, user=None)
-    >>> milestone_tasks = bugtask_set.search(params)
-    >>> for bugtask in milestone_tasks:
-    ...     print bugTaskInfo(bugtask)
-    2 1 Mozilla Firefox Firefox does not support SVG
-
-This BugTask is also a BugTask of the milestone of the mozilla project.
-
-    >>> params = BugTaskSearchParams(milestone=project_milestone, user=None)
-    >>> milestone_tasks = bugtask_set.search(params)
-    >>> for bugtask in milestone_tasks:
-    ...     print bugTaskInfo(bugtask)
-    2 1 Mozilla Firefox Firefox does not support SVG
-
-If a bug has one bugtask associated with a product and another bugtask
-associated with a product series, and if both tasks are assigned to the
-same milestone...
-
-    >>> firefox_1_0 = firefox.getSeries("1.0")
-    >>> productseries_task = bugtask_set.createTask(
-    ...     bug_one, no_priv, firefox_1_0)
-    >>> productseries_task.milestone = product_milestone
-    >>> print bugTaskInfo(productseries_task)
-    40 1 Mozilla Firefox 1.0 Firefox does not support SVG
-
-...both of them are returned, by a search for bugs associated with the
-product milestone...
-
-    >>> params = BugTaskSearchParams(milestone=product_milestone, user=None)
-    >>> milestone_tasks = bugtask_set.search(params)
-    >>> for bugtask in milestone_tasks:
-    ...     print bugTaskInfo(bugtask)
-    2 1 Mozilla Firefox Firefox does not support SVG
-    40 1 Mozilla Firefox 1.0 Firefox does not support SVG
-
-...as well as by a search for bugs associated with the project milestone.
-
-    >>> params = BugTaskSearchParams(milestone=project_milestone, user=None)
-    >>> milestone_tasks = bugtask_set.search(params)
-    >>> for bugtask in milestone_tasks:
-    ...     print bugTaskInfo(bugtask)
-    2 1 Mozilla Firefox Firefox does not support SVG
-    40 1 Mozilla Firefox 1.0 Firefox does not support SVG
-
-
 == Bugs with partner packages ==
 
 Bugs may also be targeted to partner packages.  First turn "cdrkit" into
@@ -535,6 +114,8 @@
 
     >>> from zope.security.proxy import removeSecurityProxy
     >>> from lp.soyuz.interfaces.component import IComponentSet
+    >>> from lp.registry.interfaces.distribution import IDistributionSet
+    >>> ubuntu = getUtility(IDistributionSet).getByName("ubuntu")
     >>> proxied_cdrkit = ubuntu.getSourcePackage("cdrkit")
     >>> cdrkit = removeSecurityProxy(proxied_cdrkit)
     >>> cdrkit.component = getUtility(IComponentSet)['partner']
@@ -550,6 +131,8 @@
 We can file a bug against it and see that show up in a search:
 
     >>> from lp.bugs.interfaces.bug import CreateBugParams
+    >>> from lp.registry.interfaces.person import IPersonSet
+    >>> no_priv = getUtility(IPersonSet).getByName('no-priv')
     >>> bug = cdrkit.createBug(
     ...     CreateBugParams(owner=no_priv, title='Bug to be fixed in trunk',
     ...                     comment='Something'))
@@ -558,239 +141,6 @@
     1
 
 
-== Searching by tags ==
-
-It is possible to search for bugs by their tags. Tags in the search
-parameters can be combined using either ''any'' or ''all''.
-
-First, we create some test bugs.
-
-    >>> firefox = getUtility(IProductSet).get(4)
-    >>> foobar = getUtility(IPersonSet).get(16)
-
-The first bug is tagged with both 'test-tag-1' and 'test-tag-2'.
-
-    >>> params = CreateBugParams(
-    ...     title="test bug a", comment="test bug a", owner=foobar)
-    >>> test_bug_a = firefox.createBug(params)
-    >>> test_bug_a.tags = ['test-tag-1', 'test-tag-2']
-
-The second bug is tagged with only 'test-tag-1'.
-
-    >>> params = CreateBugParams(
-    ...     title="test bug b", comment="test bug b", owner=foobar)
-    >>> test_bug_b = firefox.createBug(params)
-    >>> test_bug_b.tags = ['test-tag-1']
-
-Searching for bugs with any of the tags returns both of them.
-
-    >>> from operator import attrgetter
-    >>> from lp.services.searchbuilder import all, any
-
-    >>> def search_tasks_and_print_bugs(user=None, **args):
-    ...     params = BugTaskSearchParams(user=user, **args)
-    ...     tasks = firefox.searchTasks(params)
-    ...     bugs = (task.bug for task in tasks)
-    ...     bugs = sorted(bugs, key=attrgetter('id'))
-    ...     for bug in bugs:
-    ...         print "%s [%s]" % (bug.title, ", ".join(bug.tags))
-
-    >>> search_tasks_and_print_bugs(
-    ...     tag=any('test-tag-1', 'test-tag-2'))
-    test bug a [test-tag-1, test-tag-2]
-    test bug b [test-tag-1]
-
-Searching for bugs with all of the tags returns only test bug a.
-
-    >>> search_tasks_and_print_bugs(
-    ...     tag=all('test-tag-1', 'test-tag-2'))
-    test bug a [test-tag-1, test-tag-2]
-
-Search for the absence of a tag is possible by prefixing the tag name
-with a minus.
-
-    >>> search_tasks_and_print_bugs(tag=any('-test-tag-2'))
-    Firefox does not support SVG []
-    Reflow problems with complex page layouts [layout-test]
-    Firefox install instructions should be complete [doc]
-    Firefox crashes when Save As dialog for a nonexistent window is closed []
-    Some bug []
-    test bug b [test-tag-1]
-
-The any() and all() search combinators are taken into consideration
-when searching for the absence of tags too. The following search says
-"give me bugs that don't have the test-tag-2 tag set *OR* that don't
-have the layout-test tag set". Only test-bug-1 is elimininated because
-it has no tags other than those requested in the search.
-
-    >>> search_tasks_and_print_bugs(
-    ...     tag=any('-test-tag-1', '-test-tag-2'))
-    Firefox does not support SVG []
-    Reflow problems with complex page layouts [layout-test]
-    Firefox install instructions should be complete [doc]
-    Firefox crashes when Save As dialog for a nonexistent window is closed []
-    Some bug []
-    test bug b [test-tag-1]
-
-Whereas the following search says "give me bugs that don't have the
-test-tag-2 tag set *AND* that don't have the layout-test tag set".
-
-    >>> search_tasks_and_print_bugs(
-    ...     tag=all('-test-tag-2', '-layout-test'))
-    Firefox does not support SVG []
-    Firefox install instructions should be complete [doc]
-    Firefox crashes when Save As dialog for a nonexistent window is closed []
-    Some bug []
-    test bug b [test-tag-1]
-
-Searching for the presence of any tags at all is also possible using a
-wildcard. If prefixed with a minus it searches for the absence of
-tags.
-
-    >>> search_tasks_and_print_bugs(tag=all('*'))
-    Reflow problems with complex page layouts [layout-test]
-    Firefox install instructions should be complete [doc]
-    test bug a [test-tag-1, test-tag-2]
-    test bug b [test-tag-1]
-
-    >>> search_tasks_and_print_bugs(tag=all('-*'))
-    Firefox does not support SVG []
-    Firefox crashes when Save As dialog for a nonexistent window is closed []
-    Some bug []
-
-Searching for the presence and absence of tags finds no
-matches. Unsurprisingly.
-
-    >>> search_tasks_and_print_bugs(tag=all('*', '-*'))
-
-Wildcards can be combined with non-wildcard tags. The following finds
-all bugs with tags, but without test-tag-1:
-
-    >>> search_tasks_and_print_bugs(tag=all('*', '-test-tag-1'))
-    Reflow problems with complex page layouts [layout-test]
-    Firefox install instructions should be complete [doc]
-
-The following is very similar; it finds all bugs with tags, *or*
-without test-tag-1:
-
-    >>> search_tasks_and_print_bugs(tag=any('*', '-test-tag-1'))
-    Firefox does not support SVG []
-    Reflow problems with complex page layouts [layout-test]
-    Firefox install instructions should be complete [doc]
-    Firefox crashes when Save As dialog for a nonexistent window is closed []
-    Some bug []
-    test bug a [test-tag-1, test-tag-2]
-    test bug b [test-tag-1]
-
-The following finds all untagged bugs and bugs with the doc tag.
-
-    >>> search_tasks_and_print_bugs(tag=any('-*', 'doc'))
-    Firefox does not support SVG []
-    Firefox install instructions should be complete [doc]
-    Firefox crashes when Save As dialog for a nonexistent window is closed []
-    Some bug []
-
-
-== Searching by date_closed ==
-
-It's possible to limit the search by date_closed, to get only bugs
-closed after a certain date. greater_than is used to search for bugs
-closed after a certain date.
-
-    >>> import pytz
-    >>> from datetime import datetime, timedelta
-    >>> from lp.services.searchbuilder import greater_than
-    >>> product = factory.makeProduct()
-    >>> utc_now = datetime(2008, 9, 4, 12, 0, 0, tzinfo=pytz.timezone('UTC'))
-    >>> not_closed_bug = factory.makeBug(target=product, title="Not closed")
-    >>> bug_closed_a_day_ago = factory.makeBug(
-    ...     target=product, date_closed=utc_now-timedelta(days=1),
-    ...     title="Closed a day ago")
-    >>> bug_closed_a_week_ago = factory.makeBug(
-    ...     target=product, date_closed=utc_now-timedelta(days=7),
-    ...     title="Closed a week ago")
-
-    >>> search_params = BugTaskSearchParams(
-    ...     user=None, orderby="-date_closed",
-    ...     date_closed=greater_than(utc_now))
-    >>> list(product.searchTasks(search_params))
-    []
-
-    >>> search_params.date_closed = greater_than(utc_now - timedelta(days=2))
-    >>> for bug_task in product.searchTasks(search_params):
-    ...     print bug_task.bug.title
-    Closed a day ago
-
-
-== Searching for bug with attachments ==
-
-It's possible to search for bugs with an attachment of a certain type.
-
-    >>> from StringIO import StringIO
-    >>> from lp.services.librarian.interfaces import ILibraryFileAliasSet
-    >>> from lp.services.messages.interfaces.message import IMessageSet
-    >>> from lp.bugs.interfaces.bugattachment import (
-    ...     BugAttachmentType,
-    ...     IBugAttachmentSet,
-    ...     )
-    >>> product = factory.makeProduct()
-    >>> patch_bug = factory.makeBug(target=product)
-    >>> filecontent = 'Some diff data'
-    >>> filealias = getUtility(ILibraryFileAliasSet).create(
-    ...     name='patch.diff', size=len(filecontent),
-    ...     file=StringIO(filecontent), contentType='text/plain')
-    >>> message = getUtility(IMessageSet).fromText(
-    ...     subject="title", content="added a patch.")
-    >>> attachmentset = getUtility(IBugAttachmentSet)
-    >>> attachment = attachmentset.create(
-    ...     bug=patch_bug, filealias=filealias, title='Patch',
-    ...     message=message, attach_type=BugAttachmentType.PATCH)
-    >>> patch_bug.attachments.count()
-    1
-
-We've added an attachment to our new bug with an attachment type
-PATCH. Searching for bugs with that attachment type we get one result.
-
-    >>> search_params = BugTaskSearchParams(
-    ...     user=None, attachmenttype=BugAttachmentType.PATCH)
-    >>> product.searchTasks(search_params).count()
-    1
-
-    >>> filecontent = 'Some more diff data'
-    >>> filealias = getUtility(ILibraryFileAliasSet).create(
-    ...     name='patch.diff', size=len(filecontent),
-    ...     file=StringIO(filecontent), contentType='text/plain')
-    >>> message = getUtility(IMessageSet).fromText(
-    ...     subject="title", content="added another patch.")
-    >>> attachmentset = getUtility(IBugAttachmentSet)
-    >>> attachment = attachmentset.create(
-    ...     bug=patch_bug, filealias=filealias, title='Patch 2',
-    ...     message=message, attach_type=BugAttachmentType.PATCH)
-    >>> patch_bug.attachments.count()
-    2
-
-We've added another patch to the bug. Searching for bugs with patch
-attachments still returns a single result, since even though a new
-attachment was added, there is still only one bug with attachments.
-
-    >>> product.searchTasks(search_params).count()
-    1
-
-== Searching for bugs affecting a user ==
-
-We can search for bugs which a user marked as affecting them.
-
-    >>> affecting_bug = factory.makeBug(title='A bug affecting a user')
-    >>> affected_user = factory.makePerson(name='affected-user')
-    >>> affecting_bug.markUserAffected(affected_user)
-    >>> target = affecting_bug.bugtasks[0].target
-    >>> affecting_tasks = target.searchTasks(
-    ...     None, user=None, affected_user=affected_user)
-    >>> for task in affecting_tasks:
-    ...     print task.bug.title
-    A bug affecting a user
-
-
 == Searching for bugs related to hardware ==
 
 We can search for bugs which are related to a given hardware device or
@@ -825,7 +175,7 @@
     9 Foo Bar
     10 Foo Bar
     2 Sample Person
-    19 No Privileges Person
+    16 No Privileges Person
 
 Similary, we can search for device drivers appearing in HWDB submissions
 of a bug reporter.
@@ -871,6 +221,8 @@
 The PCI device (0x10de, 0x0455) is not controlled in any HWDB submission
 by the sd driver, so we'll get an empty result set for this query.
 
+    >>> from lp.registry.interfaces.product import IProductSet
+    >>> firefox = getUtility(IProductSet).getByName('firefox')
     >>> search_params = BugTaskSearchParams(
     ...     user=None, hardware_bus=HWBus.PCI, hardware_vendor_id='0x10de',
     ...     hardware_product_id='0x0455', hardware_driver_name='sd',
@@ -880,7 +232,8 @@
 
 We can also search for device owners which are subscribed to a bug.
 
-    >>> sample_person = getUtility(IPersonSet).getByEmail('[e-mail]')
+    >>> sample_person = getUtility(IPersonSet).getByEmail(
+    ...     '[e-mail]')
     >>> search_params = BugTaskSearchParams(
     ...     user=None, hardware_bus=HWBus.PCI, hardware_vendor_id='0x10de',
     ...     hardware_product_id='0x0455',
@@ -909,14 +262,14 @@
     >>> from lp.hardwaredb.interfaces.hwdb import IHWSubmissionSet
     >>> hw_submission = getUtility(IHWSubmissionSet).getBySubmissionKey(
     ...     'sample-submission')
-    >>> bug_19 = getUtility(IBugSet).get(19)
-    >>> bug_19.linkHWSubmission(hw_submission)
+    >>> bug_16 = getUtility(IBugSet).get(16)
+    >>> bug_16.linkHWSubmission(hw_submission)
     >>> search_params = BugTaskSearchParams(
     ...     user=None, hardware_bus=HWBus.PCI, hardware_vendor_id='0x10de',
     ...     hardware_product_id='0x0455', hardware_is_linked_to_bug=True)
     >>> for bugtask in ubuntu.searchTasks(search_params):
     ...     print bugtask.bug.id
-    19
+    16
 
 If a device appears in a private submission, related bugs are shown
 only if the user running the request is the owner of the submission
@@ -930,14 +283,15 @@
     ...     hardware_is_linked_to_bug=True)
     >>> for bugtask in ubuntu.searchTasks(search_params):
     ...     print bugtask.bug.id
-    19
+    16
 
+    >>> foo_bar = getUtility(IPersonSet).getByEmail('[e-mail]')
     >>> search_params = BugTaskSearchParams(
     ...     user=foo_bar, hardware_bus=HWBus.PCI, hardware_vendor_id='0x10de',
     ...     hardware_product_id='0x0455', hardware_is_linked_to_bug=True)
     >>> for bugtask in ubuntu.searchTasks(search_params):
     ...     print bugtask.bug.id
-    19
+    16
 
 Other users cannot see that a bug is related to a device from a
 private submission.
@@ -954,55 +308,6 @@
     >>> ubuntu.searchTasks(search_params).count()
     0
 
-== Searching for bugs affecting me ==
-
-The user searching for bugs can search for bugs affecting him.
-
-We search for bugs affecting foo_bar, then check that all the results
-return True for isUserAffected(foo_bar).
-
-    >>> search_params = BugTaskSearchParams(
-    ...     user=foo_bar, affects_me=True)
-    >>> print reduce(
-    ...     lambda x, y: x and y,
-    ...     [task.bug.isUserAffected(foo_bar)
-    ...      for task in firefox.searchTasks(search_params)])
-    True
-
-
-== Searching for bugs linked to branches ==
-
-We can search for bugs having branches linked to them.
-
-    >>> from lp.bugs.interfaces.bugtasksearch import BugBranchSearch
-    >>> search_params = BugTaskSearchParams(
-    ...     user=None, linked_branches=BugBranchSearch.BUGS_WITH_BRANCHES)
-    >>> for task in firefox.searchTasks(search_params):
-    ...     print task.bug.id, task.bug.linked_branches.count()
-    4 2
-    5 1
-
-Similarly, we can search for bugs that do not have any linked branches.
-
-    >>> search_params = BugTaskSearchParams(
-    ...     user=None, linked_branches=BugBranchSearch.BUGS_WITHOUT_BRANCHES)
-    >>> for task in firefox.searchTasks(search_params):
-    ...     print task.bug.id, task.bug.linked_branches.count()
-    1 0
-    6 0
-    18 0
-    20 0
-    21 0
-
-And we can search for bugs linked to a specific branch.
-
-    >>> search_params = BugTaskSearchParams(
-    ...     user=None, linked_branches=1)
-    >>> for task in firefox.searchTasks(search_params):
-    ...     print task.bug.id
-    4
-    5
-
 
 == Ordering search results ==
 
@@ -1015,6 +320,11 @@
 
 Here is the list of bugs for Ubuntu.
 
+    >>> def bugTaskInfo(bugtask):
+    ...     return '%i %i %s %s' % (
+    ...         bugtask.id, bugtask.bug.id, bugtask.bugtargetdisplayname,
+    ...         bugtask.bug.title)
+
     >>> params = BugTaskSearchParams(
     ...     orderby='-number_of_duplicates', user=None)
     >>> ubuntu_tasks = ubuntu.searchTasks(params)
@@ -1024,7 +334,7 @@
     23 9 thunderbird (Ubuntu) Thunderbird crashes
     25 10 linux-source-2.6.15 (Ubuntu) another test bug
     26 2 Ubuntu Blackhole Trash folder
-    41 19 cdrkit (Ubuntu) Bug to be fixed in trunk
+    36 16 cdrkit (Ubuntu) Bug to be fixed in trunk
 
 None of these bugs have any duplicates.
 
@@ -1041,7 +351,8 @@
     >>> bug_ten.markAsDuplicate(bug_nine)
     >>> flush_database_updates()
 
-Searching again reveals bug #9 at the top of the list, since it now has a duplicate.
+Searching again reveals bug #9 at the top of the list, since it now has
+a duplicate.
 
     >>> ubuntu_tasks = ubuntu.searchTasks(params)
     >>> for bugtask in ubuntu_tasks:
@@ -1050,7 +361,7 @@
     17 1 mozilla-firefox (Ubuntu) Firefox does not support SVG
     25 10 linux-source-2.6.15 (Ubuntu) another test bug
     26 2 Ubuntu Blackhole Trash folder
-    41 19 cdrkit (Ubuntu) Bug to be fixed in trunk
+    36 16 cdrkit (Ubuntu) Bug to be fixed in trunk
 
 
 === Ordering by number of comments ===
@@ -1066,11 +377,11 @@
     ...     bug = bugtask.bug
     ...     print '%s %s [%s comments]' % (
     ...     bug.id, bug.title, bug.message_count)
-    2 Blackhole Trash folder [5 comments]
-    1 Firefox does not support SVG [3 comments]
+    2 Blackhole Trash folder [3 comments]
+    1 Firefox does not support SVG [2 comments]
     10 another test bug [2 comments]
     9 Thunderbird crashes [1 comments]
-    19 Bug to be fixed in trunk [1 comments]
+    16 Bug to be fixed in trunk [1 comments]
 
 
 === Ordering by bug heat ===
@@ -1090,7 +401,7 @@
     ...     bug = bugtask.bug
     ...     print '%s %s [heat: %s]' % (
     ...     bug.id, bug.title, bug.heat)
-    19 Bug to be fixed in trunk [heat: 19]
+    16 Bug to be fixed in trunk [heat: 16]
     10 another test bug [heat: 10]
     9 Thunderbird crashes [heat: 9]
     2 Blackhole Trash folder [heat: 2]
@@ -1111,7 +422,7 @@
     >>> ubuntu_tasks = ubuntu.searchTasks(params)
     >>> for bugtask in ubuntu_tasks:
     ...     print bugTaskInfo(bugtask)
-    41 19 cdrkit (Ubuntu) Bug to be fixed in trunk
+    36 16 cdrkit (Ubuntu) Bug to be fixed in trunk
     26 2 Ubuntu Blackhole Trash folder
     25 10 linux-source-2.6.15 (Ubuntu) another test bug
     23 9 thunderbird (Ubuntu) Thunderbird crashes
@@ -1119,6 +430,7 @@
 
 If we add a patch attachment to bug 2 and bug 10, they are listed first.
 
+    >>> bug_two = getUtility(IBugSet).get(2)
     >>> patch_attachment_bug_2 = factory.makeBugAttachment(
     ...     bug=bug_two, is_patch=True)
     >>> transaction.commit()
@@ -1131,237 +443,6 @@
     ...     print bugTaskInfo(bugtask)
     26 2 Ubuntu Blackhole Trash folder
     25 10 linux-source-2.6.15 (Ubuntu) another test bug
-    41 19 cdrkit (Ubuntu) Bug to be fixed in trunk
+    36 16 cdrkit (Ubuntu) Bug to be fixed in trunk
     23 9 thunderbird (Ubuntu) Thunderbird crashes
     17 1 mozilla-firefox (Ubuntu) Firefox does not support SVG
-
-
-== Searching using a flat interface ==
-
-An alternative, simplified interface for searching bug tasks is available
-by passing all search parameters as function arguments to searchTasks.
-This interface corresponds to the search options available from the web
-search form and the public API.
-
-    >>> def print_bugtasks(bugtasks):
-    ...     for bugtask in bugtasks:
-    ...         print '%s %s %s %s %s' % (
-    ...             bugtask.bug.id, bugtask.bugtargetdisplayname,
-    ...             bugtask.bug.title, bugtask.status.title.upper(),
-    ...             bugtask.importance.title.upper())
-
-
-When we call the method on Ubuntu, without any parameters, the result is
-identical to calling searchTasks.
-
-    >>> print_bugtasks(ubuntu.searchTasks(None, user=None))
-    1 mozilla-firefox (Ubuntu) Firefox does not support SVG NEW MEDIUM
-    9 thunderbird (Ubuntu) Thunderbird crashes CONFIRMED MEDIUM
-    2 Ubuntu Blackhole Trash folder NEW MEDIUM
-    19 cdrkit (Ubuntu) Bug to be fixed in trunk NEW UNDECIDED
-
-If we want to restrict the search using certain parameters pass them to
-the function directly. Here we search Ubuntu again, but only bugs for
-the firefox package.
-
-    >>> from lp.registry.interfaces.sourcepackagename import (
-    ...     ISourcePackageNameSet)
-    >>> source_package_name_set = getUtility(ISourcePackageNameSet)
-    >>> firefox_source_package = source_package_name_set['mozilla-firefox']
-    >>> print_bugtasks(ubuntu.searchTasks(
-    ...     None, user=None, sourcepackagename=firefox_source_package))
-    1 mozilla-firefox (Ubuntu) Firefox does not support SVG NEW MEDIUM
-
-Or search for bugs on a distribution source package directly.
-
-    >>> print_bugtasks(ubuntu_firefox.searchTasks(None, user=None))
-    1 mozilla-firefox (Ubuntu) Firefox does not support SVG NEW MEDIUM
-
-Or we can search a certain milestone (only getting bugs targeted to it).
-
-    >>> firefox_milestone_1 = firefox.getMilestone('1.0')
-    >>> print_bugtasks(firefox_milestone_1.searchTasks(None, user=None))
-    1 Mozilla Firefox Firefox does not support SVG NEW LOW
-    1 Mozilla Firefox 1.0 Firefox does not support SVG NEW UNDECIDED
-
-We can restrict our search for firefox bugs with a text search.
-
-    >>> print_bugtasks(firefox.searchTasks(
-    ...     None, user=None, search_text=u'instructions'))
-    5 Mozilla Firefox Firefox install instructions should be complete NEW CRITICAL
-
-Or restrict our search (over the mozilla project this time) to a list of
-relevant importance values.
-
-    >>> print_bugtasks(mozilla.searchTasks(
-    ...     None, user=None,
-    ...     importance=[BugTaskImportance.LOW, BugTaskImportance.MEDIUM]))
-    4 Mozilla Firefox Reflow problems with complex page layouts NEW MEDIUM
-    1 Mozilla Firefox Firefox does not support SVG NEW LOW
-
-
-== Searching by structural subscriber ==
-
-The 'structural_subscriber' search parameter allows one to search all the bug
-tasks to which a person is structurally subscribed. A person can be a
-structural subscriber for a product, a product series, a project, a milestone,
-a distribution, a distribution series and a distribution source package. No
-Privileges Person isn't a structural subscriber, so no bug tasks are found:
-
-    >>> from lp.registry.interfaces.person import IPersonSet
-    >>> no_priv = getUtility(IPersonSet).getByName('no-priv')
-    >>> no_priv_struct_sub = BugTaskSearchParams(
-    ...     user=None, structural_subscriber=no_priv)
-    >>> found_bugtasks = bugtask_set.search(no_priv_struct_sub)
-    >>> found_bugtasks.count()
-    0
-
-Create a new person and make them a subscriber to all Firefox (product) bug
-reports.  Subsequently, we confirm that they are subscribed to all of the
-Firefox bug tasks.
-
-    >>> product_struct_subber = factory.makePerson(
-    ...     name='product-struct-subber')
-    >>> firefox.addBugSubscription(product_struct_subber,
-    ...     product_struct_subber)
-    <...StructuralSubscription object at ...>
-    >>> product_struct_sub_search = BugTaskSearchParams(
-    ...     user=None, structural_subscriber=product_struct_subber)
-    >>> found_bugtasks = bugtask_set.search(product_struct_sub_search)
-    >>> found_bugtasks.count()
-    7
-
-Create a new person and subscribe them to all of the bug tasks for a product
-series.  We then test to see that they are subscribed to all of the bug tasks
-for the product series in which they are interested.
-
-    >>> product_series = firefox.getSeries('1.0')
-    >>> all_targeted = BugTaskSearchParams(user=None, omit_targeted=False)
-    >>> series_tasks = product_series.searchTasks(all_targeted)
-    >>> series_struct_subber = factory.makePerson(
-    ...     name='series-struct-subber')
-    >>> product_series.addBugSubscription(series_struct_subber,
-    ...     series_struct_subber)
-    <...StructuralSubscription object at ...>
-    >>> series_struct_sub_search = BugTaskSearchParams(
-    ...     user=None, structural_subscriber=series_struct_subber)
-    >>> found_bugtasks = bugtask_set.search(series_struct_sub_search)
-    >>> found_bugtasks.count()
-    2
-
-Create a new product which will be a part of a project group.  A bug is
-created for the product which should then show up for structural subscribers
-of the project group.  Then a new person is created who is subscribed to all
-of the bug reports about the project. Search for bug tasks that this new
-person is subscribed.
-
-    >>> product = factory.makeProduct()
-    >>> bug = factory.makeBug(target=product)
-    >>> project = factory.makeProject()
-    >>> product.project = project
-    >>> project_struct_subber = factory.makePerson(
-    ...     name='project-struct-subber')
-    >>> project.addBugSubscription(project_struct_subber,
-    ...     project_struct_subber)
-    <...StructuralSubscription object at ...>
-    >>> project_struct_sub_search = BugTaskSearchParams(
-    ...     user=None, structural_subscriber=project_struct_subber)
-    >>> found_bugtasks = bugtask_set.search(project_struct_sub_search)
-    >>> found_bugtasks.count()
-    1
-
-We will also subscribe this project subscriber to a product that is a part of
-the project and ensure that duplicate bug tasks do not appear in the search
-results.
-
-    >>> product2 = factory.makeProduct()
-    >>> bug = factory.makeBug(target=product2)
-    >>> product2.project = project
-    >>> product2.addBugSubscription(project_struct_subber,
-    ...     project_struct_subber)
-    <...StructuralSubscription object at ...>
-    >>> project_struct_sub_search = BugTaskSearchParams(
-    ...     user=None, structural_subscriber=project_struct_subber)
-    >>> found_bugtasks = bugtask_set.search(project_struct_sub_search)
-    >>> found_bugtasks.count()
-    2
-
-Create a new person and subscribe them to all of bug tasks targeted to a
-milestone.  We then test to see that they are subscribed to all of the
-bug tasks for the milestone in which they are interested.
-
-    >>> milestone_struct_subber = factory.makePerson(
-    ...     name='milestone-struct-subber')
-    >>> product_milestone.addBugSubscription(milestone_struct_subber,
-    ...     milestone_struct_subber)
-    <...StructuralSubscription object at ...>
-    >>> milestone_struct_sub_search = BugTaskSearchParams(
-    ...     user=None, structural_subscriber=milestone_struct_subber)
-    >>> found_bugtasks = bugtask_set.search(milestone_struct_sub_search)
-    >>> found_bugtasks.count()
-    2
-
-Create another new person and subscribe them to all the Ubuntu bug reports -
-crazy I know.  Then test to see that this poor person is subscribed to all
-bugs with an Ubuntu bug task.
-
-    >>> distro_struct_subber = factory.makePerson(
-    ...     name='distro-struct-subber')
-    >>> ubuntu.addBugSubscription(distro_struct_subber,
-    ...     distro_struct_subber)
-    <...StructuralSubscription object at ...>
-    >>> distro_struct_sub_search = BugTaskSearchParams(
-    ...     user=None, structural_subscriber=distro_struct_subber)
-    >>> found_bugtasks = bugtask_set.search(distro_struct_sub_search)
-    >>> found_bugtasks.count()
-    5
-
-Create a new person who will only be subscribed to an Ubuntu series, which is
-something more reasonable than all of Ubuntu.  Test to ensure that this person
-is subscribed to all of the bug tasks about that distro series.
-
-    >>> ubuntu = getUtility(IDistributionSet).getByName('ubuntu')
-    >>> hoary = ubuntu.getSeries('hoary')
-    >>> distro_series_struct_subber = factory.makePerson(
-    ...     name='distro-series-struct-subber')
-    >>> hoary.addBugSubscription(distro_series_struct_subber,
-    ...     distro_series_struct_subber)
-    <...StructuralSubscription object at ...>
-    >>> distro_series_struct_sub_search = BugTaskSearchParams(
-    ...     user=None, structural_subscriber=distro_series_struct_subber)
-    >>> found_bugtasks = bugtask_set.search(distro_series_struct_sub_search)
-    >>> all_targeted = BugTaskSearchParams(user=None, omit_targeted=False)
-    >>> hoary_bugtasks = hoary.searchTasks(all_targeted)
-    >>> found_bugtasks.count() == hoary_bugtasks.count()
-    True
-
-Create a new person and make them a subscriber to all Ubuntu Firefox (a
-distribution source package) bug reports.  Test to see that the new person is
-subscribed to all of the Ubuntu Firefox bug tasks.
-
-    >>> package_struct_subber = factory.makePerson(
-    ...     name='package-struct-subber')
-    >>> ubuntu_firefox.addBugSubscription(package_struct_subber,
-    ...     package_struct_subber)
-    <...StructuralSubscription object at ...>
-    >>> package_struct_sub_search = BugTaskSearchParams(
-    ...     user=None, structural_subscriber=package_struct_subber)
-    >>> found_bugtasks = bugtask_set.search(package_struct_sub_search)
-    >>> found_bugtasks.count()
-    1
-
-We'll also subscribe the person who is currently subscribed to a package's bug
-reports, package_struct_subber, to the bug reports of a product series to
-ensure that the structural_subscriber search is returning the set of both bug
-tasks.
-
-    >>> product_series.addBugSubscription(package_struct_subber,
-    ...     package_struct_subber)
-    <...StructuralSubscription object at ...>
-    >>> package_struct_sub_search = BugTaskSearchParams(
-    ...     user=None, structural_subscriber=package_struct_subber)
-    >>> found_bugtasks = bugtask_set.search(package_struct_sub_search)
-    >>> combined_bugtasks_count = (ubuntu_firefox_bugs.count () +
-    ...    series_tasks.count())
-    >>> found_bugtasks.count() == combined_bugtasks_count
-    True

=== modified file 'lib/lp/bugs/model/bugtasksearch.py'
--- lib/lp/bugs/model/bugtasksearch.py	2012-09-24 05:17:00 +0000
+++ lib/lp/bugs/model/bugtasksearch.py	2012-09-28 22:06:42 +0000
@@ -87,11 +87,9 @@
 from lp.services.database.lpstorm import IStore
 from lp.services.database.sqlbase import sqlvalues
 from lp.services.database.stormexpr import (
-    Array,
     ArrayAgg,
     ArrayIntersects,
     get_where_for_reference,
-    NullCount,
     Unnest,
     )
 from lp.services.propertycache import get_property_cache
@@ -478,8 +476,6 @@
             BugSubscription.person == params.subscriber))
 
     if params.structural_subscriber is not None:
-        # See bug 787294 for the story that led to the query elements
-        # below.  Please change with care.
         with_clauses.append(
             '''ss as (SELECT * from StructuralSubscription
             WHERE StructuralSubscription.subscriber = %s)'''
@@ -488,79 +484,65 @@
         class StructuralSubscriptionCTE(StructuralSubscription):
             __storm_table__ = 'ss'
 
-        join_tables.append(
-            (Product, LeftJoin(Product, And(
-                            BugTaskFlat.product_id == Product.id,
-                            Product.active))))
-        ProductSub = ClassAlias(StructuralSubscriptionCTE)
-        join_tables.append((
-            ProductSub,
-            LeftJoin(
-                ProductSub,
-                BugTaskFlat.product_id == ProductSub.productID)))
-        ProductSeriesSub = ClassAlias(StructuralSubscriptionCTE)
-        join_tables.append((
-            ProductSeriesSub,
-            LeftJoin(
-                ProductSeriesSub,
-                BugTaskFlat.productseries_id ==
-                    ProductSeriesSub.productseriesID)))
-        ProjectSub = ClassAlias(StructuralSubscriptionCTE)
-        join_tables.append((
-            ProjectSub,
-            LeftJoin(
-                ProjectSub,
-                Product.projectID == ProjectSub.projectID)))
-        DistributionSub = ClassAlias(StructuralSubscriptionCTE)
-        join_tables.append((
-            DistributionSub,
-            LeftJoin(
-                DistributionSub,
-                And(BugTaskFlat.distribution_id ==
-                        DistributionSub.distributionID,
-                    Or(
-                        DistributionSub.sourcepackagenameID ==
-                            BugTaskFlat.sourcepackagename_id,
-                        DistributionSub.sourcepackagenameID == None)))))
-        if params.distroseries is not None:
-            parent_distro_id = params.distroseries.distributionID
-        else:
-            parent_distro_id = 0
-        DistroSeriesSub = ClassAlias(StructuralSubscriptionCTE)
-        join_tables.append((
-            DistroSeriesSub,
-            LeftJoin(
-                DistroSeriesSub,
-                Or(BugTaskFlat.distroseries_id ==
-                        DistroSeriesSub.distroseriesID,
-                    # There is a mismatch between BugTask and
-                    # StructuralSubscription. SS does not support
-                    # distroseries. This clause works because other
-                    # joins ensure the match bugtask is the right
-                    # series.
-                    And(parent_distro_id == DistroSeriesSub.distributionID,
-                        BugTaskFlat.sourcepackagename_id ==
-                            DistroSeriesSub.sourcepackagenameID)))))
-        MilestoneSub = ClassAlias(StructuralSubscriptionCTE)
-        join_tables.append((
-            MilestoneSub,
-            LeftJoin(
-                MilestoneSub,
-                BugTaskFlat.milestone_id == MilestoneSub.milestoneID)))
-        extra_clauses.append(
-            NullCount(Array(
-                ProductSub.id, ProductSeriesSub.id, ProjectSub.id,
-                DistributionSub.id, DistroSeriesSub.id, MilestoneSub.id)) < 6)
-        has_duplicate_results = True
+        SS = ClassAlias(StructuralSubscriptionCTE)
+        # Milestones apply to all structural subscription searches.
+        ss_clauses = [
+            In(BugTaskFlat.milestone_id, Select(SS.milestoneID, tables=[SS]))]
+        if (params.project is None
+            and params.product is None and params.productseries is None):
+            # This search is *not* contrained to project related bugs, so
+            # include distro, distroseries, DSP and SP subscriptions.
+            ss_clauses.append(In(
+                BugTaskFlat.distribution_id,
+                Select(SS.distributionID, tables=[SS],
+                       where=(SS.sourcepackagenameID == None))))
+            ss_clauses.append(In(
+                Row(BugTaskFlat.distribution_id,
+                    BugTaskFlat.sourcepackagename_id),
+                Select((SS.distributionID, SS.sourcepackagenameID),
+                       tables=[SS])))
+            ss_clauses.append(In(
+                BugTaskFlat.distroseries_id,
+                Select(SS.distroseriesID, tables=[SS],
+                       where=(SS.sourcepackagenameID == None))))
+            # Users expect to find their DSP subscriptions when searching
+            # distroseries. We only include these when we need to.
+            if params.distroseries is not None:
+                distroseries_id = params.distroseries.id
+                parent_distro_id = params.distroseries.distributionID
+            else:
+                distroseries_id = 0
+                parent_distro_id = 0
+            ss_clauses.append(In(
+                Row(BugTaskFlat.distroseries_id,
+                    BugTaskFlat.sourcepackagename_id),
+                Select((distroseries_id, SS.sourcepackagenameID), tables=[SS],
+                       where=And(
+                           SS.distributionID == parent_distro_id,
+                           SS.sourcepackagenameID != None))))
+        if params.distribution is None and params.distroseries is None:
+            # This search is *not* contrained to distro related bugs so
+            # include products, productseries, and project group subscriptions.
+            project_match = True
+            if params.project is not None:
+                project_match = Product.project == params.project
+            ss_clauses.append(In(
+                BugTaskFlat.product_id,
+                Select(SS.productID, tables=[SS])))
+            ss_clauses.append(In(
+                BugTaskFlat.productseries_id,
+                Select(SS.productseriesID, tables=[SS])))
+            ss_clauses.append(In(
+                BugTaskFlat.product_id,
+                Select(Product.id, tables=[SS, Product],
+                       where=And(
+                           SS.projectID == Product.projectID,
+                           project_match,
+                           Product.active))))
+        extra_clauses.append(Or(*ss_clauses))
 
-    # Remove bugtasks from deactivated products, if necessary.
-    # We don't have to do this if
-    # 1) We're searching on bugtasks for a specific product
-    # 2) We're searching on bugtasks for a specific productseries
-    # 3) We're searching on bugtasks for a distribution
-    # 4) We're searching for bugtasks for a distroseries
-    # because in those instances we don't have arbitrary products which
-    # may be deactivated showing up in our search.
+    # Remove bugtasks from deactivated products. This is needed for searches
+    # where people or project groups are the context.
     if (params.product is None and
         params.distribution is None and
         params.productseries is None and
@@ -618,15 +600,10 @@
         if tag_clause is not None:
             extra_clauses.append(tag_clause)
 
-    # XXX Tom Berger 2008-02-14:
-    # We use StructuralSubscription to determine
-    # the bug supervisor relation for distribution source
-    # packages, following a conversion to use this object.
-    # We know that the behaviour remains the same, but we
-    # should change the terminology, or re-instate
-    # PackageBugSupervisor, since the use of this relation here
-    # is not for subscription to notifications.
-    # See bug #191809
+    # XXX sinzui 2012-09-26:
+    # This uses StructuralSubscription to assume a bug supervisor relationship
+    # for distribution source packages to preserve historical behaviour.
+    # This also duplicates params.structural_subscriber code and behaviour.
     if params.bug_supervisor:
         extra_clauses.append(Or(
             In(

=== modified file 'lib/lp/bugs/model/tests/test_bugtasksearch.py'
--- lib/lp/bugs/model/tests/test_bugtasksearch.py	2012-09-21 02:03:56 +0000
+++ lib/lp/bugs/model/tests/test_bugtasksearch.py	2012-09-28 22:06:42 +0000
@@ -2377,6 +2377,29 @@
         self.assertContentEqual(bug1.bugtasks, tasks)
 
 
+class TargetLessTestCase(TestCaseWithFactory):
+    """Test that do not call setTarget() in the BugTaskSearchParams."""
+
+    layer = DatabaseFunctionalLayer
+
+    def test_project_group_structural_subscription(self):
+        # Search results can be limited to bugs without a bug target to which
+        # a given person has a structural subscription.
+        subscriber = self.factory.makePerson()
+        product = self.factory.makeProduct()
+        self.factory.makeBug(target=product)
+        with person_logged_in(product.owner):
+            project_group = self.factory.makeProject(owner=product.owner)
+            product.project = project_group
+        with person_logged_in(subscriber):
+            project_group.addBugSubscription(subscriber, subscriber)
+        params = BugTaskSearchParams(
+            user=None, structural_subscriber=subscriber)
+        bugtask_set = getUtility(IBugTaskSet)
+        found_bugtasks = bugtask_set.search(params)
+        self.assertEqual(1, found_bugtasks.count())
+
+
 class BaseGetBugPrivacyFilterTermsTests:
 
     layer = DatabaseFunctionalLayer