Merge lp:~james-w/pkgme-service/log-oopses into lp:pkgme-service

[James Westby]

Add the configuration to have django errors logged to a file.

Hi,

This adds the configuration for django's logging framework to log
everything at WARNING or higher to a log file.

It also does a couple of things that will help with using this:

  * Adds /pkgme/+oops URL that generates an exception on purpose.
  * Adds a template for 404 so that they don't turn in to 500s
  * Adds a template for 500 so that we know what the real exception is, not
    the one that says "You don't have a 500.html template."

This will require some co-ordination with IS around deployment so that the
config is in place first, and the config needs to be created by puppet
so that it can point to the right log dir. I'll propose a branch to add
this to the template.

Thanks,

James

[lp:~james-w/pkgme-service/log-oopses]

39. By James Westby on 2012-01-24

Also add the stanza to the dev_config template.

40. By James Westby on 2012-01-24

Reset debug.

[Jonathan Lange (jml)]

Looks good.

Why do we set the log dir for Django in the example credentials, but set the output directory for pkgme in the production.cfg?

Is there some wiki page we should update for tricky deployments?

review: Approve

[James Westby (james-w)]

On Wed, 25 Jan 2012 13:57:44 -0000, Jonathan Lange <email address hidden> wrote:
> Review: Approve
>
> Looks good.
>
> Why do we set the log dir for Django in the example credentials, but set the output directory for pkgme in the production.cfg?

We have to set the log dir in the production_credentials as it will
change depending on the instance (production/staging), so it should be
managed by puppet.

The fact that the output directory is the same for pkgme is a bit of an
oddity I guess.

> Is there some wiki page we should update for tricky deployments?

https://wiki.canonical.com/InformationInfrastructure/WebOps/CA/PkgmeServiceProductionStatus

I'll update it when the branch lands and I have the revno.

Thanks,

James

=== modified file '.bzrignore'
--- .bzrignore	2012-01-13 01:03:04 +0000
+++ .bzrignore	2012-01-24 21:19:23 +0000
@@ -9,3 +9,4 @@
 sourcecode/*
 django_project/production_paths.py
 django_project/production_credentials.cfg
+django.log

=== modified file 'dev_config/templates/production_credentials.cfg.erb'
--- dev_config/templates/production_credentials.cfg.erb	2012-01-18 23:45:14 +0000
+++ dev_config/templates/production_credentials.cfg.erb	2012-01-24 21:19:23 +0000
@@ -25,3 +25,9 @@
 myapps_consumer_key = 
 myapps_token = 
 myapps_token_secret = 
+
+[django_file_logging_handler]
+level = WARNING
+class = logging.handlers.WatchedFileHandler
+formatter = verbose
+filename = <%= basedir %>/django.log

=== modified file 'django_project/dev.cfg'
--- django_project/dev.cfg	2012-01-09 22:52:57 +0000
+++ django_project/dev.cfg	2012-01-24 21:19:23 +0000
@@ -15,6 +15,12 @@
     djkombu
 databases = databases
 
+[django_file_logging_handler]
+level = WARNING
+class = logging.handlers.WatchedFileHandler
+formatter = verbose
+filename = django.log
+
 [databases]
 default = default_database
 

=== modified file 'django_project/main.cfg'
--- django_project/main.cfg	2012-01-10 14:33:32 +0000
+++ django_project/main.cfg	2012-01-24 21:19:23 +0000
@@ -33,3 +33,38 @@
     django.contrib.messages.middleware.MessageMiddleware
 root_urlconf = django_project.urls
 test_runner = djcelery.contrib.test_runner.CeleryTestSuiteRunner
+logging = django_logging
+
+[django_logging]
+version = 1
+disable_existing_loggers = True
+formatters = django_logging_formatters
+handlers = django_logging_handlers
+loggers = django_logging_loggers
+root = django_logging_root
+
+[django_logging_root]
+level = WARNING
+
+[django_logging_loggers]
+django = django_logger
+
+[django_logger]
+handlers = file
+level = WARNING
+
+[django_logging_handlers]
+file = django_file_logging_handler
+null = django_null_logging_handler
+
+[django_null_logging_handler]
+level = DEBUG
+class = django.utils.log.NullHandler
+
+[django_logging_formatters]
+verbose = django_verbose_logging_formatter
+
+[django_verbose_logging_formatter]
+# Double escaped as configglue interpolates once, and we need the rest to survive
+# until django interpolates.
+format = %%(levelname)s %%(asctime)s %%(module)s %%(process)d %%(thread)d %%(message)s

=== modified file 'django_project/production_credentials.cfg.example'
--- django_project/production_credentials.cfg.example	2012-01-09 22:52:57 +0000
+++ django_project/production_credentials.cfg.example	2012-01-24 21:19:23 +0000
@@ -31,3 +31,9 @@
 myapps_consumer_key =
 myapps_token =
 myapps_token_secret =
+
+[django_file_logging_handler]
+level = WARNING
+class = logging.handlers.WatchedFileHandler
+formatter = verbose
+filename = /var/log/pkgme-service/django.log

=== added directory 'src/djpkgme/templates'
=== added file 'src/djpkgme/templates/404.html'
--- src/djpkgme/templates/404.html	1970-01-01 00:00:00 +0000
+++ src/djpkgme/templates/404.html	2012-01-24 21:19:23 +0000
@@ -0,0 +1,5 @@
+<html>
+    <body>
+        <h1>Not Found</h1>
+    </body>
+</html>

=== added file 'src/djpkgme/templates/500.html'
--- src/djpkgme/templates/500.html	1970-01-01 00:00:00 +0000
+++ src/djpkgme/templates/500.html	2012-01-24 21:19:23 +0000
@@ -0,0 +1,5 @@
+<html>
+<body>
+    <h1>Error</h1>
+</body>
+</html>

=== modified file 'src/djpkgme/urls.py'
--- src/djpkgme/urls.py	2011-08-31 19:37:43 +0000
+++ src/djpkgme/urls.py	2012-01-24 21:19:23 +0000
@@ -5,5 +5,6 @@
 
 urlpatterns = patterns('',
     url(r'^$', views.index),
+    url(r'^\+oops$', views.oops),
     url(r'^api/', include(djpkgme.api.urls)),
 )

=== modified file 'src/djpkgme/views.py'
--- src/djpkgme/views.py	2011-11-28 17:20:42 +0000
+++ src/djpkgme/views.py	2012-01-24 21:19:23 +0000
@@ -1,8 +1,9 @@
-from django.http import HttpResponseRedirect, HttpResponse
-
-from .tasks import GuessPackagingDetailsTask
+from django.http import HttpResponse
 
 ALL_CLEAR = 'Hello world!'
 
 def index(request):
     return HttpResponse(ALL_CLEAR)
+
+def oops(request):
+    raise AssertionError("Manually generated OOPS")