lp:~bac/charmworld/reed-search

Branch merges

Propose for merging

No branches dependent on this one.

Related source package recipes

No recipes using this branch. (?)

Create packaging recipe

Related snap packages

Related bugs

Related blueprints

508. By Brad Crittenden on 2014-05-01

Merge Reed's ngram. Update package dependencies.

507. By Brad Crittenden on 2014-04-30

Add in annotations from mongo when returning search results from the API search.

Approved by Juju Gui Bot, j.c.sackett.

506. By Brad Crittenden on 2014-04-22

Handle old bundles in production.

Some bundles in production do not have promulgated flags. Process them
anyway.

https://codereview.appspot.com/90420044/

R=bac.

Approved by Juju Gui Bot.

505. By Brad Crittenden on 2014-04-22

Fix migration wrt invalid constraints.

The migration 026 for rewriting contraints to be space-separated worked for
staging. When run on production it had errors due to some of the bundles on
production being older than the more restrictive bundle proofing
requirements. Those bundles were in essence grandfathered in and never
subject to the stricter rules. At least one bundle had an obsolete 'cpu'
constraint and that caused the migration to fail since it was not checking for
the thrown ValueError in this situation.

The migration now checks for ValueError and will remove the affected bundle
from the target database and proceed.

The functions are now noted to throw ValueError.

Test was added for this situation and logging checked.

https://codereview.appspot.com/90420043/

R=jcsackett.

Approved by Juju Gui Bot.

504. By Brad Crittenden on 2014-04-22

Fix 026 exodus.

A basket, the thing ingested from Launchpad, contains one or more bundles. If
there are multiple bundles they may establish an inheritance relationship
between the bundles in that basket. The inheritance is done before the basket
is split into individual bundles and stored -- flattened in other words.

The 'inherits' line is not removed, however.

When the exodus happens, the re-use of 'store_bundles' attempts to flatten
again, but the relationship to the other bundle is invalid as the 'basket' now
just contains the one pre-processed bundle.

The fix for this exodus is to simply manually delete the 'inherits' key from
the bundle_data before proceeding.

The exodus has been tested against a full ingest of all charms and bundles.
The unit test has been modified to include an 'inherits' key in the data,
which is then ignored.

https://codereview.appspot.com/90300043/

R=jcsackett.

Approved by Juju Gui Bot.

503. By Brad Crittenden on 2014-04-21

Exodus to remove comma-sep constraints.

Comma-separated constraints in bundles are deprecated. Currently newly
processed bundles are converted to space-separated on ingestion but there are
a number of old bundles in the db that need fixing.

As this is an exodus, and only one can be present at a time, the 025 exodus is
deleted.

https://codereview.appspot.com/89770044/

R=jcsackett.

Approved by Juju Gui Bot.

502. By Brad Crittenden on 2014-04-17

Fix spurious error.

All fixes are:

Fix spurious test failure when version==0.

Print bundle revid in debug message.

Handle HTTPErrors from get_branch_tips and ValueErrors from parse_branch.

Move implicit parse_branch test from test_lp to an explicit one in
test_utils.

Add Cache-Control to Launchpad API call when getting branch tips. Also use
correct distribution name rather than alias.

https://codereview.appspot.com/88980043/

R=sinzui.

Approved by Juju Gui Bot.

501. By Brad Crittenden on 2014-04-14

Do not index charm annotations in bundles.

Charms in bundles have "annotations" which include information relating to the
widget layout when placed in the GUI ('gui-x' and 'gui-y'). This information
is ancilliary to the actual bundle and should not be searchable.

The presence of such data has also caused operational problems. In
production, some bundles are not being indexed due to parsing errors of the x
and y coordinates.

An example failure is shown here:
2014-04-11 19:12:30,096 DEBUG [pyelasticsearch][MainThread] got response {u'items': [{u'index': {u'_type': u'bundle', u'_id': u'~charmers/muletrain/wiki', u'error': u'MapperParsingException[failed to parse [data.data.services.mediawiki.annotations.gui-y]]; nested: NumberFormatException[For input string: "-117.5"]; ', u'_index': u'charms-78512'}}], u'took': 4}

Unfortunately, this ingestion failure cannot be reproduced on staging or
locally. Further, it appears pyelasticsearch is not returning an error in
this situation but is instead silently failing to index the bundle.

To QA the branch, in one window run 'make run'. In another, ingest some
bundles:

% bin/ingest-queued --prefix="~charmers/charms/bundles"

Afterwards, go to 'localhost:2464' in your browser. Search for 'bundles' and
see the ones that got ingested. Next search for 'gui-x' and see no results.

https://codereview.appspot.com/87710043/

R=jcsackett.

Approved by Juju Gui Bot.

500. By Brad Crittenden on 2014-04-03

Bundle proof errors if juju-gui included.

Updated error message from review but failed to land in previous commit.

https://codereview.appspot.com/83270048/

R=hatch, juju-gui-bot.

Approved by Juju Gui Bot.

499. By Brad Crittenden on 2014-04-02

Bundle proof errors if juju-gui included.

The juju-gui should not be a part of a bundle, though the juju-gui export
mechanism has historically included it when doing the export. Because it has
been in the exports, and most bundles start out as an export that then gets
tuned, a lot of existing bundles have juju-gui in them.

https://codereview.appspot.com/83270048/

R=hatch.

Approved by Juju Gui Bot.