lp:~ev/daisy/filter-out-and-monitor-bad-data

Created by Evan on 2013-06-18 and last modified on 2013-06-18
Get this branch:
bzr branch lp:~ev/daisy/filter-out-and-monitor-bad-data
Only Evan can upload to this branch. If you are Evan please log in for upload directions.

Branch merges

Related bugs

Related blueprints

Branch information

Owner:
Evan
Project:
Daisy
Status:
Development

Recent revisions

377. By Evan on 2013-06-18

Monitor whether we're getting anything useful out of attempting to repair the SAS. Only filter out reports that have absolutely no chance of producing something useful. A missing DistroRelease field means it will only ever count towards the total for 'All releases' and show up in the counts on the problem pages, but that's still something. Only filter out reports that are either missing the ProblemType and Package or the ProblemType and some sort of signature.

376. By Evan on 2013-06-18

Comment.

375. By Evan on 2013-06-18

Don't create fields that don't exist in the db.

374. By Evan on 2013-06-18

Reports like 074e85be-7cdd-11e2-b545-2c768aafd08c only have the SystemIdentifier field, which we insert. They're effectively empty, so don't accept them.

373. By Evan on 2013-06-18

Don't just throw everything into the Stacktrace CF. We have a lot of that data in the OOPS CF, and we're not actually using anything from Stacktrace except the '*Stacktrace*' columns.

372. By Evan on 2013-06-18

Add some more instrumentation to submit_core.

371. By Evan on 2013-06-18

Don't use the MissingSAS or KernelCrash rows in the Counters CF anymore - this is what statsd and graphite are for. Do not proceed if the report is missing the DistroRelease or ProblemType keys. Increment statsd meters when we are dropping invalid reports.

370. By Evan on 2013-06-17

Tune the buffer_size down to 1K to give the cluster a bit more breathing room.

369. By Evan on 2013-06-14

Make build_errors_by_release faster by caching FirstError. This will require multiple runs, but it already need that because FirstError isn't accurate until it is entirely populated.

368. By Evan on 2013-06-12

This is now BucketVersionSystems2.

Branch metadata

Branch format:
Branch format 7
Repository format:
Bazaar repository format 2a (needs bzr 1.16 or later)
Stacked on:
lp:daisy
This branch contains Public information 
Everyone can see this information.

Subscribers