Merge lp:~jtv/maas/1.2-bug-1069734 into lp:maas/1.2

Proposed by Jeroen T. Vermeulen on 2012-10-25
Status: Merged
Approved by: Jeroen T. Vermeulen on 2012-10-26
Approved revision: 1273
Merged at revision: 1272
Proposed branch: lp:~jtv/maas/1.2-bug-1069734
Merge into: lp:maas/1.2
Diff against target: 1207 lines (+691/-306)
12 files modified
etc/cron.d/maas-gc (+2/-6)
setup.py (+0/-2)
src/maasserver/api.py (+2/-2)
src/maasserver/management/commands/gc.py (+0/-24)
src/maasserver/migrations/0039_add_filestorage_content.py (+233/-0)
src/maasserver/migrations/0040_make_filestorage_data_not_null.py (+203/-0)
src/maasserver/migrations/0041_remove_filestorage_data.py (+213/-0)
src/maasserver/models/filestorage.py (+15/-92)
src/maasserver/testing/factory.py (+4/-4)
src/maasserver/tests/test_api.py (+4/-2)
src/maasserver/tests/test_commands.py (+0/-11)
src/maasserver/tests/test_filestorage.py (+15/-163)
To merge this branch: bzr merge lp:~jtv/maas/1.2-bug-1069734
Reviewer Review Type Date Requested Status
Raphaël Badin (community) 2012-10-25 Approve on 2012-10-26
Review via email: mp+131334@code.launchpad.net

Commit message

Move file storage from the filesystem into the database, so that it's no longer local to a single app server.

Functionally the change is hidden inside FileStorage, but the need for garbage collection goes out the window as well. We should remove the maas-gc cron script altogether, but for that, the packaging branch needs updating as well. Keeping the file in disabled form for now decouples the two changes.

Description of the change

This is the change for the 1.2 branch. We'll want to apply the change to trunk as well.

Jeroen

To post a comment you must log in.
Raphaël Badin (rvb) wrote :

Looks generally good… but I'm wondering about how you tested this, hence "Needs information".

[0]

560 + rmtree(
561 + os.path.join(settings.MEDIA_ROOT, upload_dir),
562 + ignore_errors=True)

I think this is a bit dangerous, because it uses a user setting. Imagine the case where a user is installing MAAS (with a version which has this migration) but changes MEDIA_ROOT to a custom location. This migration, without any good reason, will issue a rm -rf ${MEDIA_ROOT}/storage. I really wonder if we shouldn't leave that directory alone. The flip side would be that old installations will have a 'storage' directory with a few files in it but new installations (which is probably what we should focus on) will be fine (and the rf -rf ... won't be run).

[1]

Testing: migrations cannot be unit tested but I'd be more comfortable with this change knowing that the migration has been tested on a instance with real data in FileStorage. How did you test this?

review: Needs Information
Raphaël Badin (rvb) wrote :

All right, thanks for the explanations about the manual testing you did. Maybe something we could do is create utilities methods in migrations (like you've done here) and test these… but I guess the utilities would have to be written in a very special way so that one could test them without calling the orm at all.

review: Approve
lp:~jtv/maas/1.2-bug-1069734 updated on 2012-10-26
1273. By Jeroen T. Vermeulen on 2012-10-26

Only remove old storage directory if MEDIA_ROOT is set, and exists.

Jeroen T. Vermeulen (jtv) wrote :

Thanks. Also as discussed, I've updated the rmtree code. The deletion no longer happens if MEDIA_ROOT is not set, or if it does not identify an existing directory.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'etc/cron.d/maas-gc'
--- etc/cron.d/maas-gc 2012-08-03 16:33:05 +0000
+++ etc/cron.d/maas-gc 2012-10-26 07:56:20 +0000
@@ -1,8 +1,4 @@
1# Perform daily background cleanups in MAAS.1# Perform daily background cleanups in MAAS.
2#2#
3# The "maas gc" command is for garbage-collection, such as deleting uploaded3# This currently does nothing: maas-gc is no longer needed.
4# files from Juju's file storage API, and in the future commissioning logs,4#0 0 * * * root /usr/sbin/maas gc &> /dev/null
5# that have been superseded by newer ones. (This isn't done immediately
6# when the files are overwritten because (1) the transaction that overwrites
7# them may fail, and (2) a file may still be in use when it's overwritten.)
80 0 * * * root /usr/sbin/maas gc &> /dev/null
95
=== modified file 'setup.py'
--- setup.py 2012-10-01 22:56:46 +0000
+++ setup.py 2012-10-26 07:56:20 +0000
@@ -65,8 +65,6 @@
65 'etc/maas/commissioning-user-data',65 'etc/maas/commissioning-user-data',
66 'contrib/maas-http.conf',66 'contrib/maas-http.conf',
67 'contrib/maas_local_settings.py']),67 'contrib/maas_local_settings.py']),
68 ('/etc/cron.d',
69 ['etc/cron.d/maas-gc']),
70 ('/usr/share/maas',68 ('/usr/share/maas',
71 ['contrib/wsgi.py',69 ['contrib/wsgi.py',
72 'etc/celeryconfig.py',70 'etc/celeryconfig.py',
7371
=== modified file 'src/maasserver/api.py'
--- src/maasserver/api.py 2012-10-11 13:55:44 +0000
+++ src/maasserver/api.py 2012-10-26 07:56:20 +0000
@@ -87,7 +87,6 @@
87from functools import partial87from functools import partial
88import httplib88import httplib
89from inspect import getdoc89from inspect import getdoc
90import simplejson as json
91import sys90import sys
92from textwrap import dedent91from textwrap import dedent
9392
@@ -175,6 +174,7 @@
175from piston.utils import rc174from piston.utils import rc
176from provisioningserver.enum import POWER_TYPE175from provisioningserver.enum import POWER_TYPE
177from provisioningserver.kernel_opts import KernelParameters176from provisioningserver.kernel_opts import KernelParameters
177import simplejson as json
178178
179179
180class OperationsResource(Resource):180class OperationsResource(Resource):
@@ -926,7 +926,7 @@
926 db_file = FileStorage.objects.get(filename=filename)926 db_file = FileStorage.objects.get(filename=filename)
927 except FileStorage.DoesNotExist:927 except FileStorage.DoesNotExist:
928 raise MAASAPINotFound("File not found")928 raise MAASAPINotFound("File not found")
929 return HttpResponse(db_file.data.read(), status=httplib.OK)929 return HttpResponse(db_file.content, status=httplib.OK)
930930
931931
932class AnonFilesHandler(AnonymousOperationsHandler):932class AnonFilesHandler(AnonymousOperationsHandler):
933933
=== removed file 'src/maasserver/management/commands/gc.py'
--- src/maasserver/management/commands/gc.py 2012-04-16 10:00:51 +0000
+++ src/maasserver/management/commands/gc.py 1970-01-01 00:00:00 +0000
@@ -1,24 +0,0 @@
1# Copyright 2012 Canonical Ltd. This software is licensed under the
2# GNU Affero General Public License version 3 (see the file LICENSE).
3
4"""Custom django command: garabge-collect."""
5
6from __future__ import (
7 absolute_import,
8 print_function,
9 unicode_literals,
10 )
11
12__metaclass__ = type
13__all__ = [
14 'Command',
15 ]
16
17
18from django.core.management.base import BaseCommand
19from maasserver.models import FileStorage
20
21
22class Command(BaseCommand):
23 def handle(self, *args, **options):
24 FileStorage.objects.collect_garbage()
250
=== added file 'src/maasserver/migrations/0039_add_filestorage_content.py'
--- src/maasserver/migrations/0039_add_filestorage_content.py 1970-01-01 00:00:00 +0000
+++ src/maasserver/migrations/0039_add_filestorage_content.py 2012-10-26 07:56:20 +0000
@@ -0,0 +1,233 @@
1# -*- coding: utf-8 -*-
2from base64 import b64encode
3import datetime
4import os.path
5
6from django.conf import settings
7from django.db import models
8from south.db import db
9from south.v2 import SchemaMigration
10
11
12def get_unmigrated_filestorages(orm):
13 """Find FileStorage objects whose data needs migrating."""
14 return orm['maasserver.FileStorage'].objects.filter(content=None)
15
16
17def read_file(storage):
18 """Read file contents from a FileStorage."""
19 return storage.data.read()
20
21
22def copy_files_into_database(orm):
23 """Copy file contents into the "content" field."""
24 for storage in get_unmigrated_filestorages(orm):
25 raw_content = read_file(storage)
26 storage.content = b64encode(raw_content).decode('ascii')
27 storage.save()
28
29
30class Migration(SchemaMigration):
31
32 def forwards(self, orm):
33 # Adding field 'FileStorage.content'
34 db.add_column(u'maasserver_filestorage', 'content',
35 self.gf('metadataserver.fields.BinaryField')(null=True),
36 keep_default=False)
37
38 # Changing field 'FileStorage.filename'
39 db.alter_column(u'maasserver_filestorage', 'filename', self.gf('django.db.models.fields.CharField')(unique=True, max_length=255))
40
41 # Effecting data migration. Not deleting the old files yet; the
42 # database transaction might still abort for whatever reason.
43 copy_files_into_database(orm)
44
45 def backwards(self, orm):
46 # Deleting field 'FileStorage.content'
47 db.delete_column(u'maasserver_filestorage', 'content')
48
49
50 # Changing field 'FileStorage.filename'
51 db.alter_column(u'maasserver_filestorage', 'filename', self.gf('django.db.models.fields.CharField')(max_length=200, unique=True))
52
53 models = {
54 'auth.group': {
55 'Meta': {'object_name': 'Group'},
56 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
57 'name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '80'}),
58 'permissions': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Permission']", 'symmetrical': 'False', 'blank': 'True'})
59 },
60 'auth.permission': {
61 'Meta': {'ordering': "('content_type__app_label', 'content_type__model', 'codename')", 'unique_together': "(('content_type', 'codename'),)", 'object_name': 'Permission'},
62 'codename': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
63 'content_type': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['contenttypes.ContentType']"}),
64 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
65 'name': ('django.db.models.fields.CharField', [], {'max_length': '50'})
66 },
67 'auth.user': {
68 'Meta': {'object_name': 'User'},
69 'date_joined': ('django.db.models.fields.DateTimeField', [], {'default': 'datetime.datetime.now'}),
70 'email': ('django.db.models.fields.EmailField', [], {'unique': 'True', 'max_length': '75', 'blank': 'True'}),
71 'first_name': ('django.db.models.fields.CharField', [], {'max_length': '30', 'blank': 'True'}),
72 'groups': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Group']", 'symmetrical': 'False', 'blank': 'True'}),
73 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
74 'is_active': ('django.db.models.fields.BooleanField', [], {'default': 'True'}),
75 'is_staff': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
76 'is_superuser': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
77 'last_login': ('django.db.models.fields.DateTimeField', [], {'default': 'datetime.datetime.now'}),
78 'last_name': ('django.db.models.fields.CharField', [], {'max_length': '30', 'blank': 'True'}),
79 'password': ('django.db.models.fields.CharField', [], {'max_length': '128'}),
80 'user_permissions': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Permission']", 'symmetrical': 'False', 'blank': 'True'}),
81 'username': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '30'})
82 },
83 'contenttypes.contenttype': {
84 'Meta': {'ordering': "('name',)", 'unique_together': "(('app_label', 'model'),)", 'object_name': 'ContentType', 'db_table': "'django_content_type'"},
85 'app_label': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
86 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
87 'model': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
88 'name': ('django.db.models.fields.CharField', [], {'max_length': '100'})
89 },
90 u'maasserver.bootimage': {
91 'Meta': {'unique_together': "((u'architecture', u'subarchitecture', u'release', u'purpose'),)", 'object_name': 'BootImage'},
92 'architecture': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
93 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
94 'purpose': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
95 'release': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
96 'subarchitecture': ('django.db.models.fields.CharField', [], {'max_length': '255'})
97 },
98 u'maasserver.componenterror': {
99 'Meta': {'object_name': 'ComponentError'},
100 'component': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '40'}),
101 'created': ('django.db.models.fields.DateTimeField', [], {}),
102 'error': ('django.db.models.fields.CharField', [], {'max_length': '1000'}),
103 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
104 'updated': ('django.db.models.fields.DateTimeField', [], {})
105 },
106 u'maasserver.config': {
107 'Meta': {'object_name': 'Config'},
108 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
109 'name': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
110 'value': ('maasserver.fields.JSONObjectField', [], {'null': 'True'})
111 },
112 u'maasserver.dhcplease': {
113 'Meta': {'object_name': 'DHCPLease'},
114 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
115 'ip': ('django.db.models.fields.IPAddressField', [], {'unique': 'True', 'max_length': '15'}),
116 'mac': ('maasserver.fields.MACAddressField', [], {}),
117 'nodegroup': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['maasserver.NodeGroup']"})
118 },
119 u'maasserver.filestorage': {
120 'Meta': {'object_name': 'FileStorage'},
121 'content': ('metadataserver.fields.BinaryField', [], {'null': 'True'}),
122 'data': ('django.db.models.fields.files.FileField', [], {'max_length': '255'}),
123 'filename': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '255'}),
124 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'})
125 },
126 u'maasserver.macaddress': {
127 'Meta': {'object_name': 'MACAddress'},
128 'created': ('django.db.models.fields.DateTimeField', [], {}),
129 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
130 'mac_address': ('maasserver.fields.MACAddressField', [], {'unique': 'True'}),
131 'node': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['maasserver.Node']"}),
132 'updated': ('django.db.models.fields.DateTimeField', [], {})
133 },
134 u'maasserver.node': {
135 'Meta': {'object_name': 'Node'},
136 'after_commissioning_action': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
137 'architecture': ('django.db.models.fields.CharField', [], {'default': "u'i386/generic'", 'max_length': '31'}),
138 'cpu_count': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
139 'created': ('django.db.models.fields.DateTimeField', [], {}),
140 'distro_series': ('django.db.models.fields.CharField', [], {'default': 'None', 'max_length': '10', 'null': 'True', 'blank': 'True'}),
141 'error': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '255', 'blank': 'True'}),
142 'hardware_details': ('maasserver.fields.XMLField', [], {'default': 'None', 'null': 'True', 'blank': 'True'}),
143 'hostname': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '255', 'blank': 'True'}),
144 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
145 'memory': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
146 'netboot': ('django.db.models.fields.BooleanField', [], {'default': 'True'}),
147 'nodegroup': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['maasserver.NodeGroup']", 'null': 'True'}),
148 'owner': ('django.db.models.fields.related.ForeignKey', [], {'default': 'None', 'to': "orm['auth.User']", 'null': 'True', 'blank': 'True'}),
149 'power_parameters': ('maasserver.fields.JSONObjectField', [], {'default': "u''", 'blank': 'True'}),
150 'power_type': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '10', 'blank': 'True'}),
151 'status': ('django.db.models.fields.IntegerField', [], {'default': '0', 'max_length': '10'}),
152 'system_id': ('django.db.models.fields.CharField', [], {'default': "u'node-b588ce50-1ea0-11e2-946f-002608dc6120'", 'unique': 'True', 'max_length': '41'}),
153 'tags': ('django.db.models.fields.related.ManyToManyField', [], {'to': u"orm['maasserver.Tag']", 'symmetrical': 'False'}),
154 'token': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['piston.Token']", 'null': 'True'}),
155 'updated': ('django.db.models.fields.DateTimeField', [], {})
156 },
157 u'maasserver.nodegroup': {
158 'Meta': {'object_name': 'NodeGroup'},
159 'api_key': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '18'}),
160 'api_token': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['piston.Token']", 'unique': 'True'}),
161 'cluster_name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '100', 'blank': 'True'}),
162 'created': ('django.db.models.fields.DateTimeField', [], {}),
163 'dhcp_key': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '255', 'blank': 'True'}),
164 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
165 'name': ('django.db.models.fields.CharField', [], {'max_length': '80', 'blank': 'True'}),
166 'status': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
167 'updated': ('django.db.models.fields.DateTimeField', [], {}),
168 'uuid': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '36'})
169 },
170 u'maasserver.nodegroupinterface': {
171 'Meta': {'unique_together': "((u'nodegroup', u'interface'),)", 'object_name': 'NodeGroupInterface'},
172 'broadcast_ip': ('django.db.models.fields.GenericIPAddressField', [], {'default': 'None', 'max_length': '39', 'null': 'True', 'blank': 'True'}),
173 'created': ('django.db.models.fields.DateTimeField', [], {}),
174 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
175 'interface': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '255', 'blank': 'True'}),
176 'ip': ('django.db.models.fields.GenericIPAddressField', [], {'max_length': '39'}),
177 'ip_range_high': ('django.db.models.fields.GenericIPAddressField', [], {'default': 'None', 'max_length': '39', 'null': 'True', 'blank': 'True'}),
178 'ip_range_low': ('django.db.models.fields.GenericIPAddressField', [], {'default': 'None', 'max_length': '39', 'null': 'True', 'blank': 'True'}),
179 'management': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
180 'nodegroup': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['maasserver.NodeGroup']"}),
181 'router_ip': ('django.db.models.fields.GenericIPAddressField', [], {'default': 'None', 'max_length': '39', 'null': 'True', 'blank': 'True'}),
182 'subnet_mask': ('django.db.models.fields.GenericIPAddressField', [], {'default': 'None', 'max_length': '39', 'null': 'True', 'blank': 'True'}),
183 'updated': ('django.db.models.fields.DateTimeField', [], {})
184 },
185 u'maasserver.sshkey': {
186 'Meta': {'unique_together': "((u'user', u'key'),)", 'object_name': 'SSHKey'},
187 'created': ('django.db.models.fields.DateTimeField', [], {}),
188 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
189 'key': ('django.db.models.fields.TextField', [], {}),
190 'updated': ('django.db.models.fields.DateTimeField', [], {}),
191 'user': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['auth.User']"})
192 },
193 u'maasserver.tag': {
194 'Meta': {'object_name': 'Tag'},
195 'comment': ('django.db.models.fields.TextField', [], {'blank': 'True'}),
196 'created': ('django.db.models.fields.DateTimeField', [], {}),
197 'definition': ('django.db.models.fields.TextField', [], {}),
198 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
199 'name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '256'}),
200 'updated': ('django.db.models.fields.DateTimeField', [], {})
201 },
202 u'maasserver.userprofile': {
203 'Meta': {'object_name': 'UserProfile'},
204 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
205 'user': ('django.db.models.fields.related.OneToOneField', [], {'to': "orm['auth.User']", 'unique': 'True'})
206 },
207 'piston.consumer': {
208 'Meta': {'object_name': 'Consumer'},
209 'description': ('django.db.models.fields.TextField', [], {}),
210 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
211 'key': ('django.db.models.fields.CharField', [], {'max_length': '18'}),
212 'name': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
213 'secret': ('django.db.models.fields.CharField', [], {'max_length': '32'}),
214 'status': ('django.db.models.fields.CharField', [], {'default': "'pending'", 'max_length': '16'}),
215 'user': ('django.db.models.fields.related.ForeignKey', [], {'blank': 'True', 'related_name': "'consumers'", 'null': 'True', 'to': "orm['auth.User']"})
216 },
217 'piston.token': {
218 'Meta': {'object_name': 'Token'},
219 'callback': ('django.db.models.fields.CharField', [], {'max_length': '255', 'null': 'True', 'blank': 'True'}),
220 'callback_confirmed': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
221 'consumer': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['piston.Consumer']"}),
222 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
223 'is_approved': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
224 'key': ('django.db.models.fields.CharField', [], {'max_length': '18'}),
225 'secret': ('django.db.models.fields.CharField', [], {'max_length': '32'}),
226 'timestamp': ('django.db.models.fields.IntegerField', [], {'default': '1351168636L'}),
227 'token_type': ('django.db.models.fields.IntegerField', [], {}),
228 'user': ('django.db.models.fields.related.ForeignKey', [], {'blank': 'True', 'related_name': "'tokens'", 'null': 'True', 'to': "orm['auth.User']"}),
229 'verifier': ('django.db.models.fields.CharField', [], {'max_length': '10'})
230 }
231 }
232
233 complete_apps = ['maasserver']
0234
=== added file 'src/maasserver/migrations/0040_make_filestorage_data_not_null.py'
--- src/maasserver/migrations/0040_make_filestorage_data_not_null.py 1970-01-01 00:00:00 +0000
+++ src/maasserver/migrations/0040_make_filestorage_data_not_null.py 2012-10-26 07:56:20 +0000
@@ -0,0 +1,203 @@
1# -*- coding: utf-8 -*-
2import datetime
3
4from django.db import models
5from south.db import db
6from south.v2 import SchemaMigration
7
8
9class Migration(SchemaMigration):
10
11 def forwards(self, orm):
12
13 # Changing field 'FileStorage.content'
14 # Disallow NULLs. The previous migration should have
15 # initialized the column for all existing rows.
16 db.alter_column(u'maasserver_filestorage', 'content', self.gf('metadataserver.fields.BinaryField')(null=False))
17
18 def backwards(self, orm):
19
20 # Changing field 'FileStorage.content'
21 db.alter_column(u'maasserver_filestorage', 'content', self.gf('metadataserver.fields.BinaryField')(null=True))
22
23 models = {
24 'auth.group': {
25 'Meta': {'object_name': 'Group'},
26 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
27 'name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '80'}),
28 'permissions': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Permission']", 'symmetrical': 'False', 'blank': 'True'})
29 },
30 'auth.permission': {
31 'Meta': {'ordering': "('content_type__app_label', 'content_type__model', 'codename')", 'unique_together': "(('content_type', 'codename'),)", 'object_name': 'Permission'},
32 'codename': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
33 'content_type': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['contenttypes.ContentType']"}),
34 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
35 'name': ('django.db.models.fields.CharField', [], {'max_length': '50'})
36 },
37 'auth.user': {
38 'Meta': {'object_name': 'User'},
39 'date_joined': ('django.db.models.fields.DateTimeField', [], {'default': 'datetime.datetime.now'}),
40 'email': ('django.db.models.fields.EmailField', [], {'unique': 'True', 'max_length': '75', 'blank': 'True'}),
41 'first_name': ('django.db.models.fields.CharField', [], {'max_length': '30', 'blank': 'True'}),
42 'groups': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Group']", 'symmetrical': 'False', 'blank': 'True'}),
43 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
44 'is_active': ('django.db.models.fields.BooleanField', [], {'default': 'True'}),
45 'is_staff': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
46 'is_superuser': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
47 'last_login': ('django.db.models.fields.DateTimeField', [], {'default': 'datetime.datetime.now'}),
48 'last_name': ('django.db.models.fields.CharField', [], {'max_length': '30', 'blank': 'True'}),
49 'password': ('django.db.models.fields.CharField', [], {'max_length': '128'}),
50 'user_permissions': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Permission']", 'symmetrical': 'False', 'blank': 'True'}),
51 'username': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '30'})
52 },
53 'contenttypes.contenttype': {
54 'Meta': {'ordering': "('name',)", 'unique_together': "(('app_label', 'model'),)", 'object_name': 'ContentType', 'db_table': "'django_content_type'"},
55 'app_label': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
56 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
57 'model': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
58 'name': ('django.db.models.fields.CharField', [], {'max_length': '100'})
59 },
60 u'maasserver.bootimage': {
61 'Meta': {'unique_together': "((u'architecture', u'subarchitecture', u'release', u'purpose'),)", 'object_name': 'BootImage'},
62 'architecture': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
63 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
64 'purpose': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
65 'release': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
66 'subarchitecture': ('django.db.models.fields.CharField', [], {'max_length': '255'})
67 },
68 u'maasserver.componenterror': {
69 'Meta': {'object_name': 'ComponentError'},
70 'component': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '40'}),
71 'created': ('django.db.models.fields.DateTimeField', [], {}),
72 'error': ('django.db.models.fields.CharField', [], {'max_length': '1000'}),
73 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
74 'updated': ('django.db.models.fields.DateTimeField', [], {})
75 },
76 u'maasserver.config': {
77 'Meta': {'object_name': 'Config'},
78 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
79 'name': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
80 'value': ('maasserver.fields.JSONObjectField', [], {'null': 'True'})
81 },
82 u'maasserver.dhcplease': {
83 'Meta': {'object_name': 'DHCPLease'},
84 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
85 'ip': ('django.db.models.fields.IPAddressField', [], {'unique': 'True', 'max_length': '15'}),
86 'mac': ('maasserver.fields.MACAddressField', [], {}),
87 'nodegroup': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['maasserver.NodeGroup']"})
88 },
89 u'maasserver.filestorage': {
90 'Meta': {'object_name': 'FileStorage'},
91 'content': ('metadataserver.fields.BinaryField', [], {}),
92 'data': ('django.db.models.fields.files.FileField', [], {'max_length': '255'}),
93 'filename': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '255'}),
94 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'})
95 },
96 u'maasserver.macaddress': {
97 'Meta': {'object_name': 'MACAddress'},
98 'created': ('django.db.models.fields.DateTimeField', [], {}),
99 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
100 'mac_address': ('maasserver.fields.MACAddressField', [], {'unique': 'True'}),
101 'node': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['maasserver.Node']"}),
102 'updated': ('django.db.models.fields.DateTimeField', [], {})
103 },
104 u'maasserver.node': {
105 'Meta': {'object_name': 'Node'},
106 'after_commissioning_action': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
107 'architecture': ('django.db.models.fields.CharField', [], {'default': "u'i386/generic'", 'max_length': '31'}),
108 'cpu_count': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
109 'created': ('django.db.models.fields.DateTimeField', [], {}),
110 'distro_series': ('django.db.models.fields.CharField', [], {'default': 'None', 'max_length': '10', 'null': 'True', 'blank': 'True'}),
111 'error': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '255', 'blank': 'True'}),
112 'hardware_details': ('maasserver.fields.XMLField', [], {'default': 'None', 'null': 'True', 'blank': 'True'}),
113 'hostname': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '255', 'blank': 'True'}),
114 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
115 'memory': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
116 'netboot': ('django.db.models.fields.BooleanField', [], {'default': 'True'}),
117 'nodegroup': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['maasserver.NodeGroup']", 'null': 'True'}),
118 'owner': ('django.db.models.fields.related.ForeignKey', [], {'default': 'None', 'to': "orm['auth.User']", 'null': 'True', 'blank': 'True'}),
119 'power_parameters': ('maasserver.fields.JSONObjectField', [], {'default': "u''", 'blank': 'True'}),
120 'power_type': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '10', 'blank': 'True'}),
121 'status': ('django.db.models.fields.IntegerField', [], {'default': '0', 'max_length': '10'}),
122 'system_id': ('django.db.models.fields.CharField', [], {'default': "u'node-766d0f92-1ea5-11e2-9dee-002608dc6120'", 'unique': 'True', 'max_length': '41'}),
123 'tags': ('django.db.models.fields.related.ManyToManyField', [], {'to': u"orm['maasserver.Tag']", 'symmetrical': 'False'}),
124 'token': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['piston.Token']", 'null': 'True'}),
125 'updated': ('django.db.models.fields.DateTimeField', [], {})
126 },
127 u'maasserver.nodegroup': {
128 'Meta': {'object_name': 'NodeGroup'},
129 'api_key': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '18'}),
130 'api_token': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['piston.Token']", 'unique': 'True'}),
131 'cluster_name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '100', 'blank': 'True'}),
132 'created': ('django.db.models.fields.DateTimeField', [], {}),
133 'dhcp_key': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '255', 'blank': 'True'}),
134 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
135 'name': ('django.db.models.fields.CharField', [], {'max_length': '80', 'blank': 'True'}),
136 'status': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
137 'updated': ('django.db.models.fields.DateTimeField', [], {}),
138 'uuid': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '36'})
139 },
140 u'maasserver.nodegroupinterface': {
141 'Meta': {'unique_together': "((u'nodegroup', u'interface'),)", 'object_name': 'NodeGroupInterface'},
142 'broadcast_ip': ('django.db.models.fields.GenericIPAddressField', [], {'default': 'None', 'max_length': '39', 'null': 'True', 'blank': 'True'}),
143 'created': ('django.db.models.fields.DateTimeField', [], {}),
144 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
145 'interface': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '255', 'blank': 'True'}),
146 'ip': ('django.db.models.fields.GenericIPAddressField', [], {'max_length': '39'}),
147 'ip_range_high': ('django.db.models.fields.GenericIPAddressField', [], {'default': 'None', 'max_length': '39', 'null': 'True', 'blank': 'True'}),
148 'ip_range_low': ('django.db.models.fields.GenericIPAddressField', [], {'default': 'None', 'max_length': '39', 'null': 'True', 'blank': 'True'}),
149 'management': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
150 'nodegroup': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['maasserver.NodeGroup']"}),
151 'router_ip': ('django.db.models.fields.GenericIPAddressField', [], {'default': 'None', 'max_length': '39', 'null': 'True', 'blank': 'True'}),
152 'subnet_mask': ('django.db.models.fields.GenericIPAddressField', [], {'default': 'None', 'max_length': '39', 'null': 'True', 'blank': 'True'}),
153 'updated': ('django.db.models.fields.DateTimeField', [], {})
154 },
155 u'maasserver.sshkey': {
156 'Meta': {'unique_together': "((u'user', u'key'),)", 'object_name': 'SSHKey'},
157 'created': ('django.db.models.fields.DateTimeField', [], {}),
158 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
159 'key': ('django.db.models.fields.TextField', [], {}),
160 'updated': ('django.db.models.fields.DateTimeField', [], {}),
161 'user': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['auth.User']"})
162 },
163 u'maasserver.tag': {
164 'Meta': {'object_name': 'Tag'},
165 'comment': ('django.db.models.fields.TextField', [], {'blank': 'True'}),
166 'created': ('django.db.models.fields.DateTimeField', [], {}),
167 'definition': ('django.db.models.fields.TextField', [], {}),
168 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
169 'name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '256'}),
170 'updated': ('django.db.models.fields.DateTimeField', [], {})
171 },
172 u'maasserver.userprofile': {
173 'Meta': {'object_name': 'UserProfile'},
174 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
175 'user': ('django.db.models.fields.related.OneToOneField', [], {'to': "orm['auth.User']", 'unique': 'True'})
176 },
177 'piston.consumer': {
178 'Meta': {'object_name': 'Consumer'},
179 'description': ('django.db.models.fields.TextField', [], {}),
180 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
181 'key': ('django.db.models.fields.CharField', [], {'max_length': '18'}),
182 'name': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
183 'secret': ('django.db.models.fields.CharField', [], {'max_length': '32'}),
184 'status': ('django.db.models.fields.CharField', [], {'default': "'pending'", 'max_length': '16'}),
185 'user': ('django.db.models.fields.related.ForeignKey', [], {'blank': 'True', 'related_name': "'consumers'", 'null': 'True', 'to': "orm['auth.User']"})
186 },
187 'piston.token': {
188 'Meta': {'object_name': 'Token'},
189 'callback': ('django.db.models.fields.CharField', [], {'max_length': '255', 'null': 'True', 'blank': 'True'}),
190 'callback_confirmed': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
191 'consumer': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['piston.Consumer']"}),
192 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
193 'is_approved': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
194 'key': ('django.db.models.fields.CharField', [], {'max_length': '18'}),
195 'secret': ('django.db.models.fields.CharField', [], {'max_length': '32'}),
196 'timestamp': ('django.db.models.fields.IntegerField', [], {'default': '1351170650L'}),
197 'token_type': ('django.db.models.fields.IntegerField', [], {}),
198 'user': ('django.db.models.fields.related.ForeignKey', [], {'blank': 'True', 'related_name': "'tokens'", 'null': 'True', 'to': "orm['auth.User']"}),
199 'verifier': ('django.db.models.fields.CharField', [], {'max_length': '10'})
200 }
201 }
202
203 complete_apps = ['maasserver']
0204
=== added file 'src/maasserver/migrations/0041_remove_filestorage_data.py'
--- src/maasserver/migrations/0041_remove_filestorage_data.py 1970-01-01 00:00:00 +0000
+++ src/maasserver/migrations/0041_remove_filestorage_data.py 2012-10-26 07:56:20 +0000
@@ -0,0 +1,213 @@
1# -*- coding: utf-8 -*-
2import datetime
3import os.path
4from shutil import rmtree
5
6from django.conf import settings
7from django.db import models
8from south.db import db
9from south.v2 import SchemaMigration
10
11# The sub-directory of MEDIA_ROOT where FileStorage used to store its
12# files. This duplicates the value of the now-removed
13# FileStorage.upload_dir.
14upload_dir = 'storage'
15
16
17class Migration(SchemaMigration):
18
19 def forwards(self, orm):
20 # Deleting field 'FileStorage.data'
21 db.delete_column(u'maasserver_filestorage', 'data')
22
23 # Cleaning up any obsolete FileStorage files.
24 if settings.MEDIA_ROOT and os.path.isdir(settings.MEDIA_ROOT):
25 rmtree(
26 os.path.join(settings.MEDIA_ROOT, upload_dir),
27 ignore_errors=True)
28
29 def backwards(self, orm):
30
31 # User chose to not deal with backwards NULL issues for 'FileStorage.data'
32 raise RuntimeError("Cannot reverse this migration. 'FileStorage.data' and its values cannot be restored.")
33
34 models = {
35 'auth.group': {
36 'Meta': {'object_name': 'Group'},
37 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
38 'name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '80'}),
39 'permissions': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Permission']", 'symmetrical': 'False', 'blank': 'True'})
40 },
41 'auth.permission': {
42 'Meta': {'ordering': "('content_type__app_label', 'content_type__model', 'codename')", 'unique_together': "(('content_type', 'codename'),)", 'object_name': 'Permission'},
43 'codename': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
44 'content_type': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['contenttypes.ContentType']"}),
45 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
46 'name': ('django.db.models.fields.CharField', [], {'max_length': '50'})
47 },
48 'auth.user': {
49 'Meta': {'object_name': 'User'},
50 'date_joined': ('django.db.models.fields.DateTimeField', [], {'default': 'datetime.datetime.now'}),
51 'email': ('django.db.models.fields.EmailField', [], {'unique': 'True', 'max_length': '75', 'blank': 'True'}),
52 'first_name': ('django.db.models.fields.CharField', [], {'max_length': '30', 'blank': 'True'}),
53 'groups': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Group']", 'symmetrical': 'False', 'blank': 'True'}),
54 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
55 'is_active': ('django.db.models.fields.BooleanField', [], {'default': 'True'}),
56 'is_staff': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
57 'is_superuser': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
58 'last_login': ('django.db.models.fields.DateTimeField', [], {'default': 'datetime.datetime.now'}),
59 'last_name': ('django.db.models.fields.CharField', [], {'max_length': '30', 'blank': 'True'}),
60 'password': ('django.db.models.fields.CharField', [], {'max_length': '128'}),
61 'user_permissions': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Permission']", 'symmetrical': 'False', 'blank': 'True'}),
62 'username': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '30'})
63 },
64 'contenttypes.contenttype': {
65 'Meta': {'ordering': "('name',)", 'unique_together': "(('app_label', 'model'),)", 'object_name': 'ContentType', 'db_table': "'django_content_type'"},
66 'app_label': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
67 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
68 'model': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
69 'name': ('django.db.models.fields.CharField', [], {'max_length': '100'})
70 },
71 u'maasserver.bootimage': {
72 'Meta': {'unique_together': "((u'architecture', u'subarchitecture', u'release', u'purpose'),)", 'object_name': 'BootImage'},
73 'architecture': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
74 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
75 'purpose': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
76 'release': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
77 'subarchitecture': ('django.db.models.fields.CharField', [], {'max_length': '255'})
78 },
79 u'maasserver.componenterror': {
80 'Meta': {'object_name': 'ComponentError'},
81 'component': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '40'}),
82 'created': ('django.db.models.fields.DateTimeField', [], {}),
83 'error': ('django.db.models.fields.CharField', [], {'max_length': '1000'}),
84 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
85 'updated': ('django.db.models.fields.DateTimeField', [], {})
86 },
87 u'maasserver.config': {
88 'Meta': {'object_name': 'Config'},
89 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
90 'name': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
91 'value': ('maasserver.fields.JSONObjectField', [], {'null': 'True'})
92 },
93 u'maasserver.dhcplease': {
94 'Meta': {'object_name': 'DHCPLease'},
95 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
96 'ip': ('django.db.models.fields.IPAddressField', [], {'unique': 'True', 'max_length': '15'}),
97 'mac': ('maasserver.fields.MACAddressField', [], {}),
98 'nodegroup': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['maasserver.NodeGroup']"})
99 },
100 u'maasserver.filestorage': {
101 'Meta': {'object_name': 'FileStorage'},
102 'content': ('metadataserver.fields.BinaryField', [], {}),
103 'filename': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '255'}),
104 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'})
105 },
106 u'maasserver.macaddress': {
107 'Meta': {'object_name': 'MACAddress'},
108 'created': ('django.db.models.fields.DateTimeField', [], {}),
109 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
110 'mac_address': ('maasserver.fields.MACAddressField', [], {'unique': 'True'}),
111 'node': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['maasserver.Node']"}),
112 'updated': ('django.db.models.fields.DateTimeField', [], {})
113 },
114 u'maasserver.node': {
115 'Meta': {'object_name': 'Node'},
116 'after_commissioning_action': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
117 'architecture': ('django.db.models.fields.CharField', [], {'default': "u'i386/generic'", 'max_length': '31'}),
118 'cpu_count': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
119 'created': ('django.db.models.fields.DateTimeField', [], {}),
120 'distro_series': ('django.db.models.fields.CharField', [], {'default': 'None', 'max_length': '10', 'null': 'True', 'blank': 'True'}),
121 'error': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '255', 'blank': 'True'}),
122 'hardware_details': ('maasserver.fields.XMLField', [], {'default': 'None', 'null': 'True', 'blank': 'True'}),
123 'hostname': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '255', 'blank': 'True'}),
124 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
125 'memory': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
126 'netboot': ('django.db.models.fields.BooleanField', [], {'default': 'True'}),
127 'nodegroup': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['maasserver.NodeGroup']", 'null': 'True'}),
128 'owner': ('django.db.models.fields.related.ForeignKey', [], {'default': 'None', 'to': "orm['auth.User']", 'null': 'True', 'blank': 'True'}),
129 'power_parameters': ('maasserver.fields.JSONObjectField', [], {'default': "u''", 'blank': 'True'}),
130 'power_type': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '10', 'blank': 'True'}),
131 'status': ('django.db.models.fields.IntegerField', [], {'default': '0', 'max_length': '10'}),
132 'system_id': ('django.db.models.fields.CharField', [], {'default': "u'node-da71dfcc-1ea5-11e2-8763-002608dc6120'", 'unique': 'True', 'max_length': '41'}),
133 'tags': ('django.db.models.fields.related.ManyToManyField', [], {'to': u"orm['maasserver.Tag']", 'symmetrical': 'False'}),
134 'token': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['piston.Token']", 'null': 'True'}),
135 'updated': ('django.db.models.fields.DateTimeField', [], {})
136 },
137 u'maasserver.nodegroup': {
138 'Meta': {'object_name': 'NodeGroup'},
139 'api_key': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '18'}),
140 'api_token': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['piston.Token']", 'unique': 'True'}),
141 'cluster_name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '100', 'blank': 'True'}),
142 'created': ('django.db.models.fields.DateTimeField', [], {}),
143 'dhcp_key': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '255', 'blank': 'True'}),
144 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
145 'name': ('django.db.models.fields.CharField', [], {'max_length': '80', 'blank': 'True'}),
146 'status': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
147 'updated': ('django.db.models.fields.DateTimeField', [], {}),
148 'uuid': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '36'})
149 },
150 u'maasserver.nodegroupinterface': {
151 'Meta': {'unique_together': "((u'nodegroup', u'interface'),)", 'object_name': 'NodeGroupInterface'},
152 'broadcast_ip': ('django.db.models.fields.GenericIPAddressField', [], {'default': 'None', 'max_length': '39', 'null': 'True', 'blank': 'True'}),
153 'created': ('django.db.models.fields.DateTimeField', [], {}),
154 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
155 'interface': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '255', 'blank': 'True'}),
156 'ip': ('django.db.models.fields.GenericIPAddressField', [], {'max_length': '39'}),
157 'ip_range_high': ('django.db.models.fields.GenericIPAddressField', [], {'default': 'None', 'max_length': '39', 'null': 'True', 'blank': 'True'}),
158 'ip_range_low': ('django.db.models.fields.GenericIPAddressField', [], {'default': 'None', 'max_length': '39', 'null': 'True', 'blank': 'True'}),
159 'management': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
160 'nodegroup': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['maasserver.NodeGroup']"}),
161 'router_ip': ('django.db.models.fields.GenericIPAddressField', [], {'default': 'None', 'max_length': '39', 'null': 'True', 'blank': 'True'}),
162 'subnet_mask': ('django.db.models.fields.GenericIPAddressField', [], {'default': 'None', 'max_length': '39', 'null': 'True', 'blank': 'True'}),
163 'updated': ('django.db.models.fields.DateTimeField', [], {})
164 },
165 u'maasserver.sshkey': {
166 'Meta': {'unique_together': "((u'user', u'key'),)", 'object_name': 'SSHKey'},
167 'created': ('django.db.models.fields.DateTimeField', [], {}),
168 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
169 'key': ('django.db.models.fields.TextField', [], {}),
170 'updated': ('django.db.models.fields.DateTimeField', [], {}),
171 'user': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['auth.User']"})
172 },
173 u'maasserver.tag': {
174 'Meta': {'object_name': 'Tag'},
175 'comment': ('django.db.models.fields.TextField', [], {'blank': 'True'}),
176 'created': ('django.db.models.fields.DateTimeField', [], {}),
177 'definition': ('django.db.models.fields.TextField', [], {}),
178 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
179 'name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '256'}),
180 'updated': ('django.db.models.fields.DateTimeField', [], {})
181 },
182 u'maasserver.userprofile': {
183 'Meta': {'object_name': 'UserProfile'},
184 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
185 'user': ('django.db.models.fields.related.OneToOneField', [], {'to': "orm['auth.User']", 'unique': 'True'})
186 },
187 'piston.consumer': {
188 'Meta': {'object_name': 'Consumer'},
189 'description': ('django.db.models.fields.TextField', [], {}),
190 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
191 'key': ('django.db.models.fields.CharField', [], {'max_length': '18'}),
192 'name': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
193 'secret': ('django.db.models.fields.CharField', [], {'max_length': '32'}),
194 'status': ('django.db.models.fields.CharField', [], {'default': "'pending'", 'max_length': '16'}),
195 'user': ('django.db.models.fields.related.ForeignKey', [], {'blank': 'True', 'related_name': "'consumers'", 'null': 'True', 'to': "orm['auth.User']"})
196 },
197 'piston.token': {
198 'Meta': {'object_name': 'Token'},
199 'callback': ('django.db.models.fields.CharField', [], {'max_length': '255', 'null': 'True', 'blank': 'True'}),
200 'callback_confirmed': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
201 'consumer': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['piston.Consumer']"}),
202 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
203 'is_approved': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
204 'key': ('django.db.models.fields.CharField', [], {'max_length': '18'}),
205 'secret': ('django.db.models.fields.CharField', [], {'max_length': '32'}),
206 'timestamp': ('django.db.models.fields.IntegerField', [], {'default': '1351170840L'}),
207 'token_type': ('django.db.models.fields.IntegerField', [], {}),
208 'user': ('django.db.models.fields.related.ForeignKey', [], {'blank': 'True', 'related_name': "'tokens'", 'null': 'True', 'to': "orm['auth.User']"}),
209 'verifier': ('django.db.models.fields.CharField', [], {'max_length': '10'})
210 }
211 }
212
213 complete_apps = ['maasserver']
0214
=== modified file 'src/maasserver/models/filestorage.py'
--- src/maasserver/models/filestorage.py 2012-08-24 10:28:29 +0000
+++ src/maasserver/models/filestorage.py 2012-10-26 07:56:20 +0000
@@ -15,22 +15,17 @@
15 ]15 ]
1616
1717
18from errno import ENOENT
19import os
20import time
21
22from django.conf import settings
23from django.core.files.base import ContentFile
24from django.core.files.storage import FileSystemStorage
25from django.db.models import (18from django.db.models import (
26 CharField,19 CharField,
27 FileField,
28 Manager,20 Manager,
29 Model,21 Model,
30 )22 )
31from maasserver import DefaultMeta23from maasserver import DefaultMeta
32from maasserver.models.cleansave import CleanSave24from maasserver.models.cleansave import CleanSave
33from maasserver.utils.orm import get_one25from metadataserver.fields import (
26 Bin,
27 BinaryField,
28 )
3429
3530
36class FileStorageManager(Manager):31class FileStorageManager(Manager):
@@ -47,108 +42,36 @@
47 original file will not be affected. Also, any ongoing reads from the42 original file will not be affected. Also, any ongoing reads from the
48 old file will continue without iterruption.43 old file will continue without iterruption.
49 """44 """
50 # The time, in seconds, that an unreferenced file is allowed to
51 # persist in order to satisfy ongoing requests.
52 grace_time = 12 * 60 * 60
53
54 def get_existing_storage(self, filename):
55 """Return an existing `FileStorage` of this name, or None."""
56 return get_one(self.filter(filename=filename))
5745
58 def save_file(self, filename, file_object):46 def save_file(self, filename, file_object):
59 """Save the file to the filesystem and persist to the database.47 """Save the file to the database.
60
61 The file will end up in MEDIA_ROOT/storage/
6248
63 If a file of that name already existed, it will be replaced by the49 If a file of that name already existed, it will be replaced by the
64 new contents.50 new contents.
65 """51 """
66 # This probably ought to read in chunks but large files are52 # This probably ought to read in chunks but large files are
67 # not expected. Also note that uploading a file with the same53 # not expected.
68 # name as an existing one will cause that file to be written54 content = Bin(file_object.read())
69 # with a new generated name, and the old one remains where it55 storage, created = self.get_or_create(
70 # is. See https://code.djangoproject.com/ticket/6157 - the56 filename=filename, defaults={'content': content})
71 # Django devs consider deleting things dangerous ... ha.57 if not created:
72 # HOWEVER - this operation would need to be atomic anyway so58 storage.content = content
73 # it's safest left how it is for now (reads can overlap with59 storage.save()
74 # writes from Juju).
75 content = ContentFile(file_object.read())
76
77 storage = self.get_existing_storage(filename)
78 if storage is None:
79 storage = FileStorage(filename=filename)
80 storage.data.save(filename, content)
81 return storage60 return storage
8261
83 def list_stored_files(self):
84 """Find the files stored in the filesystem."""
85 dirs, files = FileStorage.storage.listdir(FileStorage.upload_dir)
86 return [
87 os.path.join(FileStorage.upload_dir, filename)
88 for filename in files]
89
90 def list_referenced_files(self):
91 """Find the names of files that are referenced from `FileStorage`.
92
93 :return: All file paths within MEDIA ROOT (relative to MEDIA_ROOT)
94 that have `FileStorage` entries referencing them.
95 :rtype: frozenset
96 """
97 return frozenset(
98 file_storage.data.name
99 for file_storage in self.all())
100
101 def is_old(self, storage_filename):
102 """Is the named file in the filesystem storage old enough to be dead?
103
104 :param storage_filename: The name under which the file is stored in
105 the filesystem, relative to MEDIA_ROOT. This need not be the
106 same name as its filename as stored in the `FileStorage` object.
107 It includes the name of the upload directory.
108 """
109 file_path = os.path.join(settings.MEDIA_ROOT, storage_filename)
110 mtime = os.stat(file_path).st_mtime
111 expiry = mtime + self.grace_time
112 return expiry <= time.time()
113
114 def collect_garbage(self):
115 """Clean up stored files that are no longer accessible."""
116 # Avoid circular imports.
117 from maasserver.models import logger
118
119 try:
120 stored_files = self.list_stored_files()
121 except OSError as e:
122 if e.errno != ENOENT:
123 raise
124 logger.info(
125 "Upload directory does not exist yet. "
126 "Skipping garbage collection.")
127 return
128 referenced_files = self.list_referenced_files()
129 for path in stored_files:
130 if path not in referenced_files and self.is_old(path):
131 FileStorage.storage.delete(path)
132
13362
134class FileStorage(CleanSave, Model):63class FileStorage(CleanSave, Model):
135 """A simple file storage keyed on file name.64 """A simple file storage keyed on file name.
13665
137 :ivar filename: A unique file name to use for the data being stored.66 :ivar filename: A unique file name to use for the data being stored.
138 :ivar data: The file's actual data.67 :ivar content: The file's actual data.
139 """68 """
14069
141 class Meta(DefaultMeta):70 class Meta(DefaultMeta):
142 """Needed for South to recognize this model."""71 """Needed for South to recognize this model."""
14372
144 storage = FileSystemStorage()73 filename = CharField(max_length=255, unique=True, editable=False)
14574 content = BinaryField(null=False)
146 upload_dir = "storage"
147
148 # Unix filenames can be longer than this (e.g. 255 bytes), but leave
149 # some extra room for the full path, as well as a versioning suffix.
150 filename = CharField(max_length=200, unique=True, editable=False)
151 data = FileField(upload_to=upload_dir, storage=storage, max_length=255)
15275
153 objects = FileStorageManager()76 objects = FileStorageManager()
15477
15578
=== modified file 'src/maasserver/testing/factory.py'
--- src/maasserver/testing/factory.py 2012-10-02 22:28:07 +0000
+++ src/maasserver/testing/factory.py 2012-10-26 07:56:20 +0000
@@ -296,13 +296,13 @@
296 admin.save()296 admin.save()
297 return admin297 return admin
298298
299 def make_file_storage(self, filename=None, data=None):299 def make_file_storage(self, filename=None, content=None):
300 if filename is None:300 if filename is None:
301 filename = self.getRandomString(100)301 filename = self.getRandomString(100)
302 if data is None:302 if content is None:
303 data = self.getRandomString(1024).encode('ascii')303 content = self.getRandomString(1024).encode('ascii')
304304
305 return FileStorage.objects.save_file(filename, BytesIO(data))305 return FileStorage.objects.save_file(filename, BytesIO(content))
306306
307 def make_oauth_header(self, **kwargs):307 def make_oauth_header(self, **kwargs):
308 """Fake an OAuth authorization header.308 """Fake an OAuth authorization header.
309309
=== modified file 'src/maasserver/tests/test_api.py'
--- src/maasserver/tests/test_api.py 2012-10-11 13:41:57 +0000
+++ src/maasserver/tests/test_api.py 2012-10-26 07:56:20 +0000
@@ -2383,7 +2383,8 @@
2383class AnonymousFileStorageAPITest(FileStorageAPITestMixin, AnonAPITestCase):2383class AnonymousFileStorageAPITest(FileStorageAPITestMixin, AnonAPITestCase):
23842384
2385 def test_get_works_anonymously(self):2385 def test_get_works_anonymously(self):
2386 factory.make_file_storage(filename="foofilers", data=b"give me rope")2386 factory.make_file_storage(
2387 filename="foofilers", content=b"give me rope")
2387 response = self.make_API_GET_request("get", "foofilers")2388 response = self.make_API_GET_request("get", "foofilers")
23882389
2389 self.assertEqual(httplib.OK, response.status_code)2390 self.assertEqual(httplib.OK, response.status_code)
@@ -2453,7 +2454,8 @@
2453 self.assertEqual("file two", response.content)2454 self.assertEqual("file two", response.content)
24542455
2455 def test_get_file_succeeds(self):2456 def test_get_file_succeeds(self):
2456 factory.make_file_storage(filename="foofilers", data=b"give me rope")2457 factory.make_file_storage(
2458 filename="foofilers", content=b"give me rope")
2457 response = self.make_API_GET_request("get", "foofilers")2459 response = self.make_API_GET_request("get", "foofilers")
24582460
2459 self.assertEqual(httplib.OK, response.status_code)2461 self.assertEqual(httplib.OK, response.status_code)
24602462
=== modified file 'src/maasserver/tests/test_commands.py'
--- src/maasserver/tests/test_commands.py 2012-08-24 10:28:29 +0000
+++ src/maasserver/tests/test_commands.py 2012-10-26 07:56:20 +0000
@@ -14,13 +14,10 @@
1414
15from codecs import getwriter15from codecs import getwriter
16from io import BytesIO16from io import BytesIO
17import os
1817
19from django.conf import settings
20from django.contrib.auth.models import User18from django.contrib.auth.models import User
21from django.core.cache import cache19from django.core.cache import cache
22from django.core.management import call_command20from django.core.management import call_command
23from maasserver.models import FileStorage
24from maasserver.testing.factory import factory21from maasserver.testing.factory import factory
25from maasserver.utils.orm import get_one22from maasserver.utils.orm import get_one
26from maastesting.djangotestcase import DjangoTestCase23from maastesting.djangotestcase import DjangoTestCase
@@ -33,14 +30,6 @@
33 in a command's code, it should be extracted and unit-tested separately.30 in a command's code, it should be extracted and unit-tested separately.
34 """31 """
3532
36 def test_gc(self):
37 upload_dir = os.path.join(settings.MEDIA_ROOT, FileStorage.upload_dir)
38 os.makedirs(upload_dir)
39 self.addCleanup(os.removedirs, upload_dir)
40 call_command('gc')
41 # The test is that we get here without errors.
42 pass
43
44 def test_generate_api_doc(self):33 def test_generate_api_doc(self):
45 out = BytesIO()34 out = BytesIO()
46 stdout = getwriter("UTF-8")(out)35 stdout = getwriter("UTF-8")(out)
4736
=== modified file 'src/maasserver/tests/test_filestorage.py'
--- src/maasserver/tests/test_filestorage.py 2012-06-25 09:03:14 +0000
+++ src/maasserver/tests/test_filestorage.py 2012-10-26 07:56:20 +0000
@@ -14,45 +14,15 @@
1414
15import codecs15import codecs
16from io import BytesIO16from io import BytesIO
17import os
18import shutil
1917
20from django.conf import settings
21from maasserver.models import FileStorage18from maasserver.models import FileStorage
22from maasserver.testing.factory import factory19from maasserver.testing.factory import factory
23from maasserver.testing.testcase import TestCase20from maasserver.testing.testcase import TestCase
24from maastesting.utils import age_file
25from testtools.matchers import (
26 GreaterThan,
27 LessThan,
28 )
2921
3022
31class FileStorageTest(TestCase):23class FileStorageTest(TestCase):
32 """Testing of the :class:`FileStorage` model."""24 """Testing of the :class:`FileStorage` model."""
3325
34 def make_upload_dir(self):
35 """Create the upload directory, and arrange for eventual deletion.
36
37 The directory must not already exist. If it does, this method will
38 fail rather than arrange for deletion of a directory that may
39 contain meaningful data.
40
41 :return: Absolute path to the `FileStorage` upload directory. This
42 is the directory where the actual files are stored.
43 """
44 media_root = settings.MEDIA_ROOT
45 self.assertFalse(os.path.exists(media_root), "See media/README")
46 self.addCleanup(shutil.rmtree, media_root, ignore_errors=True)
47 os.mkdir(media_root)
48 upload_dir = os.path.join(media_root, FileStorage.upload_dir)
49 os.mkdir(upload_dir)
50 return upload_dir
51
52 def get_media_path(self, filename):
53 """Get the path to a given stored file, relative to MEDIA_ROOT."""
54 return os.path.join(FileStorage.upload_dir, filename)
55
56 def make_data(self, including_text='data'):26 def make_data(self, including_text='data'):
57 """Return arbitrary data.27 """Return arbitrary data.
5828
@@ -67,40 +37,24 @@
67 text = "%s %s" % (including_text, factory.getRandomString())37 text = "%s %s" % (including_text, factory.getRandomString())
68 return text.encode('ascii')38 return text.encode('ascii')
6939
70 def test_get_existing_storage_returns_None_if_none_found(self):
71 nonexistent_file = factory.getRandomString()
72 self.assertIsNone(
73 FileStorage.objects.get_existing_storage(nonexistent_file))
74
75 def test_get_existing_storage_finds_FileStorage(self):
76 self.make_upload_dir()
77 storage = factory.make_file_storage()
78 self.assertEqual(
79 storage,
80 FileStorage.objects.get_existing_storage(storage.filename))
81
82 def test_save_file_creates_storage(self):40 def test_save_file_creates_storage(self):
83 self.make_upload_dir()
84 filename = factory.getRandomString()41 filename = factory.getRandomString()
85 data = self.make_data()42 content = self.make_data()
86 storage = FileStorage.objects.save_file(filename, BytesIO(data))43 storage = FileStorage.objects.save_file(filename, BytesIO(content))
87 self.assertEqual(44 self.assertEqual(
88 (filename, data),45 (filename, content),
89 (storage.filename, storage.data.read()))46 (storage.filename, storage.content))
9047
91 def test_storage_can_be_retrieved(self):48 def test_storage_can_be_retrieved(self):
92 self.make_upload_dir()
93 filename = factory.getRandomString()49 filename = factory.getRandomString()
94 data = self.make_data()50 content = self.make_data()
95 factory.make_file_storage(filename=filename, data=data)51 factory.make_file_storage(filename=filename, content=content)
96 storage = FileStorage.objects.get(filename=filename)52 storage = FileStorage.objects.get(filename=filename)
97 self.assertEqual(53 self.assertEqual(
98 (filename, data),54 (filename, content),
99 (storage.filename, storage.data.read()))55 (storage.filename, storage.content))
10056
101 def test_stores_binary_data(self):57 def test_stores_binary_data(self):
102 self.make_upload_dir()
103
104 # This horrible binary data could never, ever, under any58 # This horrible binary data could never, ever, under any
105 # encoding known to man be interpreted as text(1). Switch the59 # encoding known to man be interpreted as text(1). Switch the
106 # bytes of the byte-order mark around and by design you get an60 # bytes of the byte-order mark around and by design you get an
@@ -114,121 +68,19 @@
11468
115 # And yet, because FileStorage supports binary data, it comes69 # And yet, because FileStorage supports binary data, it comes
116 # out intact.70 # out intact.
117 storage = factory.make_file_storage(filename="x", data=binary_data)71 storage = factory.make_file_storage(filename="x", content=binary_data)
118 self.assertEqual(binary_data, storage.data.read())72 self.assertEqual(binary_data, storage.content)
11973
120 def test_overwrites_file(self):74 def test_overwrites_file(self):
121 # If a file of the same name has already been stored, the75 # If a file of the same name has already been stored, the
122 # reference to the old data gets overwritten with one to the new76 # reference to the old data gets overwritten with one to the new
123 # data. They are actually different files on the filesystem.77 # data.
124 self.make_upload_dir()
125 filename = factory.make_name('filename')78 filename = factory.make_name('filename')
126 old_storage = factory.make_file_storage(79 old_storage = factory.make_file_storage(
127 filename=filename, data=self.make_data('old data'))80 filename=filename, content=self.make_data('old data'))
128 new_data = self.make_data('new-data')81 new_data = self.make_data('new-data')
129 new_storage = factory.make_file_storage(82 new_storage = factory.make_file_storage(
130 filename=filename, data=new_data)83 filename=filename, content=new_data)
131 self.assertNotEqual(old_storage.data.name, new_storage.data.name)84 self.assertEqual(old_storage.filename, new_storage.filename)
132 self.assertEqual(85 self.assertEqual(
133 new_data, FileStorage.objects.get(filename=filename).data.read())86 new_data, FileStorage.objects.get(filename=filename).content)
134
135 def test_list_stored_files_lists_files(self):
136 filename = factory.getRandomString()
137 factory.make_file(
138 location=self.make_upload_dir(), name=filename,
139 contents=self.make_data())
140 self.assertIn(
141 self.get_media_path(filename),
142 FileStorage.objects.list_stored_files())
143
144 def test_list_stored_files_includes_referenced_files(self):
145 self.make_upload_dir()
146 storage = factory.make_file_storage()
147 self.assertIn(
148 storage.data.name, FileStorage.objects.list_stored_files())
149
150 def test_list_referenced_files_lists_FileStorage_files(self):
151 self.make_upload_dir()
152 storage = factory.make_file_storage()
153 self.assertIn(
154 storage.data.name, FileStorage.objects.list_referenced_files())
155
156 def test_list_referenced_files_excludes_unreferenced_files(self):
157 filename = factory.getRandomString()
158 factory.make_file(
159 location=self.make_upload_dir(), name=filename,
160 contents=self.make_data())
161 self.assertNotIn(
162 self.get_media_path(filename),
163 FileStorage.objects.list_referenced_files())
164
165 def test_list_referenced_files_uses_file_name_not_FileStorage_name(self):
166 self.make_upload_dir()
167 filename = factory.getRandomString()
168 # The filename we're going to use is already taken. The file
169 # we'll be looking at will have to have a different name.
170 factory.make_file_storage(filename=filename)
171 storage = factory.make_file_storage(filename=filename)
172 # It's the name of the file, not the FileStorage.filename, that
173 # is in list_referenced_files.
174 self.assertIn(
175 storage.data.name, FileStorage.objects.list_referenced_files())
176
177 def test_is_old_returns_False_for_recent_file(self):
178 filename = factory.getRandomString()
179 path = factory.make_file(
180 location=self.make_upload_dir(), name=filename,
181 contents=self.make_data())
182 age_file(path, FileStorage.objects.grace_time - 60)
183 self.assertFalse(
184 FileStorage.objects.is_old(self.get_media_path(filename)))
185
186 def test_is_old_returns_True_for_old_file(self):
187 filename = factory.getRandomString()
188 path = factory.make_file(
189 location=self.make_upload_dir(), name=filename,
190 contents=self.make_data())
191 age_file(path, FileStorage.objects.grace_time + 1)
192 self.assertTrue(
193 FileStorage.objects.is_old(self.get_media_path(filename)))
194
195 def test_collect_garbage_deletes_garbage(self):
196 filename = factory.getRandomString()
197 path = factory.make_file(
198 location=self.make_upload_dir(), name=filename,
199 contents=self.make_data())
200 age_file(path, FileStorage.objects.grace_time + 1)
201 FileStorage.objects.collect_garbage()
202 self.assertFalse(
203 FileStorage.storage.exists(self.get_media_path(filename)))
204
205 def test_grace_time_is_generous_but_not_unlimited(self):
206 # Grace time for garbage collection is long enough that it won't
207 # expire while the request that wrote it is still being handled.
208 # But it won't keep a file around for ages. For instance, it'll
209 # be more than 20 seconds, but less than a day.
210 self.assertThat(FileStorage.objects.grace_time, GreaterThan(20))
211 self.assertThat(FileStorage.objects.grace_time, LessThan(24 * 60 * 60))
212
213 def test_collect_garbage_leaves_recent_files_alone(self):
214 filename = factory.getRandomString()
215 factory.make_file(
216 location=self.make_upload_dir(), name=filename,
217 contents=self.make_data())
218 FileStorage.objects.collect_garbage()
219 self.assertTrue(
220 FileStorage.storage.exists(self.get_media_path(filename)))
221
222 def test_collect_garbage_leaves_referenced_files_alone(self):
223 self.make_upload_dir()
224 storage = factory.make_file_storage()
225 age_file(storage.data.path, FileStorage.objects.grace_time + 1)
226 FileStorage.objects.collect_garbage()
227 self.assertTrue(FileStorage.storage.exists(storage.data.name))
228
229 def test_collect_garbage_tolerates_missing_upload_dir(self):
230 # When MAAS is freshly installed, the upload directory is still
231 # missing. But...
232 FileStorage.objects.collect_garbage()
233 # ...we get through garbage collection without breakage.
234 pass

Subscribers

People subscribed via source and target branches

to status/vote changes: