Merge lp:~brian-murray/oops-repository/bucketsystems into lp:~ev/oops-repository/whoopsie-daisy

[Brian Murray]

Per our earlier discussion this adds bucketsystems to the schema, provides a method for adding systems to a bucket, and a way to query bucket versions.

[Brian Murray (brian-murray)]

One thing I'm uncertain of is limiting the column_count to 20. I sent you an email regarding the querying I did of the bucketversions column family and it seems that the first version appears in the first 9 columns more than 90% of the time an within the first 20 100% of the time.

[Evan (ev)]

On Thu, Feb 7, 2013 at 6:11 PM, Brian Murray <email address hidden> wrote:
> +def update_bucket_systems(config, bucket, system):
> + pool = connection_pool(config)
> + bucketsystems_cf = pycassa.ColumnFamily(pool, 'BucketSystems')
> + try:
> + systems = bucketsystems_cf.get(bucket)
> + if system not in systems:
> + bucketsystems_cf.insert(bucket, {system: ''})
> + except NotFoundException:
> + bucketsystems_cf.insert(bucket, {system: ''})

Shouldn't this just be bucketsystems_cf.insert() without the try block
or get() method? If there are already columns in bucketsystems_cf,
calling insert() will just update the column with that system
identifier.

[Brian Murray (brian-murray)]

On Thu, Feb 07, 2013 at 06:22:23PM -0000, Evan Dandrea wrote:
> On Thu, Feb 7, 2013 at 6:11 PM, Brian Murray <email address hidden> wrote:
> > +def update_bucket_systems(config, bucket, system):
> > + pool = connection_pool(config)
> > + bucketsystems_cf = pycassa.ColumnFamily(pool, 'BucketSystems')
> > + try:
> > + systems = bucketsystems_cf.get(bucket)
> > + if system not in systems:
> > + bucketsystems_cf.insert(bucket, {system: ''})
> > + except NotFoundException:
> > + bucketsystems_cf.insert(bucket, {system: ''})
>
> Shouldn't this just be bucketsystems_cf.insert() without the try block
> or get() method? If there are already columns in bucketsystems_cf,
> calling insert() will just update the column with that system
> identifier.

Ah, I hadn't thought about that. I'll go ahead and test that then,
thanks.

--
Brian Murray

[lp:~brian-murray/oops-repository/bucketsystems]

57. By Brian Murray on 2013-02-07

just use an insert in update_bucket_systems

=== modified file 'oopsrepository/oopses.py'
--- oopsrepository/oopses.py	2012-12-17 09:27:35 +0000
+++ oopsrepository/oopses.py	2013-02-07 18:43:22 +0000
@@ -187,6 +187,15 @@
                                              retry_counter_mutations=True)
     bucketversions_cf.add(bucketid, version)
 
+def query_bucket_versions(config, bucketid):
+    pool = connection_pool(config)
+    bucketversions_cf = pycassa.ColumnFamily(pool, 'BucketVersions')
+    try:
+        bucket_versions = bucketversions_cf.get(bucketid, column_count=20)
+    except NotFoundException:
+        bucket_versions = None
+    return bucket_versions 
+
 def update_bucket_metadata(config, bucketid, source, version, comparator, release=''):
     # We only update the first and last seen version fields. We do not update
     # the current version field as talking to Launchpad is an expensive
@@ -219,3 +228,8 @@
     if metadata:
         metadata['Source'] = source
         bucketmetadata_cf.insert(bucketid, metadata)
+
+def update_bucket_systems(config, bucket, system):
+    pool = connection_pool(config)
+    bucketsystems_cf = pycassa.ColumnFamily(pool, 'BucketSystems')
+    bucketsystems_cf.insert(bucket, {system: ''})
 
=== modified file 'oopsrepository/schema.py'
--- oopsrepository/schema.py	2012-12-03 18:04:54 +0000
+++ oopsrepository/schema.py	2013-02-07 18:43:22 +0000
@@ -73,6 +73,10 @@
             workaround_1779(mgr.create_column_family, keyspace, 'BucketVersions',
                 comparator_type=UTF8_TYPE,
                 default_validation_class=CounterColumnType())
+        if 'BucketSystems' not in cfs:
+            workaround_1779(mgr.create_column_family, keyspace, 'BucketSystems',
+                comparator_type=UTF8_TYPE,
+                default_validation_class=UTF8_TYPE)
         if 'Counters' not in cfs:
             workaround_1779(mgr.create_column_family, keyspace, 'Counters',
                 comparator_type=UTF8_TYPE,