Merge lp:~nigelbabu/harvest/cache-add into lp:harvest

[Nigel Babu]

Try to add some caching and reducing the DB load for '/'

[Daniel Holbach (dholbach)]

Looks great and makes loading '/' less heavy, it's just not clear yet if it causes problems with users logging in. Can somebody provide some insight into that?

[lp:~nigelbabu/harvest/cache-add]

316. By Nigel Babu on 2011-08-17

Don't cache the entire view, just the queries

317. By Nigel Babu on 2011-08-17

Fix the bugs in my previous approach

318. By Nigel Babu on 2011-08-17

Use only one cache get call

[Michael Hall (mhall119)]

Opportunity.objects.only_valid().count()

is not the same as:

Opportunity.objects.filter(valid=True, last_updated=F('opportunitylist__last_updated')).count()

It adds an extra filter: opportunitylist__active=True

[Daniel Holbach (dholbach)]

> Opportunity.objects.only_valid().count()
>
> is not the same as:
>
> Opportunity.objects.filter(valid=True,
> last_updated=F('opportunitylist__last_updated')).count()
>
> It adds an extra filter: opportunitylist__active=True

I know, but it's intentional. We use the same check in a couple of others places, but we missed to change it here as well.

[Daniel Holbach (dholbach)]

Good work!

review: Approve

=== modified file 'harvest/common/views.py'
--- harvest/common/views.py	2010-12-08 08:15:50 +0000
+++ harvest/common/views.py	2011-08-17 17:06:24 +0000
@@ -4,19 +4,35 @@
 from django.http import HttpResponseRedirect
 from django.shortcuts import render_to_response
 from django.template import RequestContext
+from django.core.cache import cache
 
 from opportunities.models import SourcePackage, OpportunityList, Opportunity, Note
 
 def index(request):
-    
+    values = cache.get_many(['user_count', 'source_packages',
+        'opp_lists', 'opp', 'notes'])
+    if 'user_count' not in values:
+        values['user_count'] = User.objects.count()
+        cache.set('user_count', values['user_count'], 60 * 5)
+    if 'source_packages' not in values:
+        values['source_packages'] = SourcePackage.objects.count()
+        cache.set('source_packages', values['source_packages'], 60 * 5)
+    if 'opp_lists' not in values:
+        values['opp_lists'] = OpportunityList.objects.filter(active=True).count()
+        cache.set('opp_lists', values['opp_lists'], 60 * 5)
+    if 'opp' not in values:
+        values['opp'] = Opportunity.objects.only_valid().count()
+        cache.set('opp', values['opp'], 60 * 5)
+    if 'notes' not in values:
+        values['notes'] = Note.objects.count()
+        cache.set('notes', values['notes'], 60 * 5)
     context = { 
         'pageSection': 'home', 
-        'users': User.objects.count(),
-        'sourcepackages': SourcePackage.objects.count(),
-        'opportunitylists': OpportunityList.objects.filter(active=True).count(),
-        'opportunities': Opportunity.objects.filter(valid=True, 
-                                last_updated=F('opportunitylist__last_updated')).count(),
-        'notes': Note.objects.count(),
+        'users': values['user_count'],
+        'sourcepackages': values['source_packages'],
+        'opportunitylists': values['opp_lists'],
+        'opportunities': values['opp'],
+        'notes': values['notes'],
     }
     return render_to_response('index.html', context,
                               context_instance=RequestContext(request))

=== modified file 'harvest/settings.py'
--- harvest/settings.py	2011-06-28 13:42:41 +0000
+++ harvest/settings.py	2011-08-17 17:06:24 +0000
@@ -162,4 +162,4 @@
 VERSION_NAME_STRING = utils.get_harvest_version_name(
                         os.path.join(PROJECT_PATH, "version"),
                         DEBUG)
-
+CACHE_BACKEND = 'locmem://'